Naive Data

Linear regression is one of the widely used statistical techniques for analysis. It provides a relationship between the target and the predictors. The below graph shows the linear relationship between the “dependent” variable Y, whose values we wish to predict and the “independent” variable X from which we wish to predict it. There are two types of linear regression: 1. Simple Linear Regression : It denotes the relationship between one variable and the target variable. The above graph represents the Simple Linear Regression. 2. Multiple Linear Regression :     It denotes the relationship between 2 or more independent variables and the corresponding dependent variable. The independent variables can be continuous or categorical. The below graph shows the multiple linear regression. Properties of Linear Regression Below are the main properties of linear regression: There is a linear relationship between X a...

In today's world, in any industry, customers have multiple options to choose from like in Telecom, we have AT&T, Verizon, T- mobile, etc, in Media, we have Netflix, Amazon Prime, Apple TV, etc.  and similarly for others. In this highly competitive market, the industries face a large number of customer churn. Considering that obtaining a new customer costs 5-10 times more than maintaining a current one, hence,  retention of customers has now become even more important than acquiring a customer. Here we are going to take data from one of the telecom company, to understand the customer churn. What is the customer churn? If a client or subscriber ceases to do business with a company or service, its called Customer Churn.  Now let's take a deep dive to understand why the Customer Churn happens and how to predict it before happening. Understanding the data Before working on any model, we need to understand the data and try to a...

Photo by  Arseny Togulev This is the second part of the series on the Basics of Machine Learning. Please refer to this  link  for part 1. Assumptions of Simple Linear Regression Below are the main assumptions of linear regression: There is a  linear  relationship between X and Y. Error terms are  normally  distributed. Error terms are  independent  of each other. Change in one error term should not impact the other error terms. Error terms have  constant variance  i.e. variance shouldn’t increase or decrease as the error values change. Also, they shouldn’t follow any pattern. If the variance is not constant or homoscedastic then the inferences made on the model would be unreliable. Checking the Model Fit: We should always make sure that the model fit is really a good fit and is not by a chance. There are many ways to test it. T-Test: The t-distribution is also a kind of normal distribution; it is also symmetric and single-peaked, bu...

In this post, we are going to perform Exploratory Data Analysis to understand how data is used to minimize the risk of losing money while lending to customers. We are going to have a look at the dataset for one of the financial institution called LandingClub. Background LendingClub is a US peer-to-peer lending company, headquartered in San Francisco, California. It enables borrowers to create unsecured personal loans between $1,000 and $40,000. The standard loan period is three years. Investors can search and browse the loan listings on LendingClub website and select loans that they want to invest in based on the information supplied about the borrower, amount of loan, loan grade, and loan purpose. Investors make money from interest. LendingClub makes money by charging borrowers an origination fee and investors a service fee. Business Understanding When a loan application is received by the company, the company must make a decision to approve the lo...

The Artificial Neural Network (ANN) is inspired by the animal brain. Even though currently the ANNs are not as powerful as the brain yet, these are one of the most powerful learning models in the field of machine learning. Understanding the Human Brain and ANNs Let's understand first how the human brain works and how it influences the ANNs. The biological neuron receives signals through its dendrites which are either amplified or inhibited as they pass through the axons to the dendrites of other neurons. ANNs are also based on a similar concept, where ANNs are a collection of a large number of simple devices called Artificial Neurons. The network learns  to perform certain tasks ( like identifying a car ) by training the neurons to fire an action when a particular input ( like an image of a car) is provided. Perceptron Perceptron is one of the earlier proposed models, which takes a weighted sum of the inputs and applies an activation function to it...

  Photo by  Jens Lelie  on  Unsplash This is the second post in In Bytes series. Refer this link to read about the first post. Random Forest: Random forest is an ensemble made by the combination of a large number of decision trees. Ensemble: An ensemble means a group of things viewed as a whole rather than individually. In ensembles, a collection of models is used to make predictions, rather than individual models. In principle, ensembles can be made by combining all types of models. An ensemble can have a logistic regression, a neural network, and few decision trees working in unison. While choosing the model , we need to check for two things- Diversity and Acceptability. Diversity ensures that the models serve complementary purposes, which means that the individual models make predictions independent of each other. The advantages of this are different depending on the type of ensemble. Diversity ensures that even if some trees overfit, the other trees in the ensemb...

Photo by Franck V.   In this  post series we are going to learn the basics of Machine Learning. Let's starts with what is Machine Learning: “Machine Learning is the science of getting computers to learn and act like humans do, and improve their learning over time in autonomous fashion, by feeding them data and information in the form of observations and real-world interactions.”  Below are the top practical definitions from different sources: “Machine Learning at its most basic is the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world.” – Nvidia  “Machine learning is the science of getting computers to act without being explicitly programmed.” – Stanford “Machine learning is based on algorithms that can learn from data without relying on rules-based programming.”- McKinsey & Co. “Machine learning algorithms can figure out how to perform important tasks by generalizing from examples.”...

  Photo by Fabrice Villard on Unsplash This the first post in Byte Series, where we are going to have discuss on overall concept but in brief to act a refresher. Decision Trees: As the name goes, a decision tree uses a tree-like model to make predictions. It resembles an upside-down tree. Decision trees are easy to interpret; you can always go back and identify the various factors leading to a decision. A decision tree requires you to perform tests on attributes in order to split the data into multiple partitions. Types: Single and Multiway class:                If a test splits the data into more than two partitions, this is called a multiway decision tree. Classification and Regression:              In regression problems, a decision tree splits the data into multiple subsets. The difference                          ...

Photo by Chris Liverani This is the first post in the series Statistics Terms for Data Science. In this series , we are going to understand the Statistics Concepts which are used in Data Science. Probability: Probability is a measure of the chances of an event happening. You can't predict many events with complete certainty. We can only predict the chance of an event occurring i.e. how likely it will occur, using it. Probability will vary from 0 to 1, where 0 means that the event is impossible, and 1 implies a certain event. Random Variables: Random Variables , generally denoted by X, provides the association between the outcome of the experiment to something measurable.  For e.g. For a loan re-payment analysis, we come to know below outcome 1. Salary < 2 Lakhs ,fails to repay 2. Salary > 2 Lakhs , can repay So here we have two outcomes, fails to repay and can repay , so we can assign a value to each outcome i.e. we can define the Random Variable X as: This is useful in ...