Statistical Concepts

Range

• The range is the difference between the largest and smallest numbers in a set of data.
• Example: Find the range of the following data set: 3, 3, 5, 5, 8, 9, 9, 9, 13, 15.
  Range = Largest number - Smallest number
  Range = 15 - 3 = 12
  So, the range of this data set is 12.

Mean (Average)

• The mean is the average of all numbers in a given data set.
• The arithmetic mean is calculated by adding up all the numbers and dividing the sum by the total number of numbers in the list.
• Mean of observations = Sum of observations / Total number of observations
  Mathematically, Mean = \(\frac{Sum\ of\ observations}{Total\ number\ of observations}\)
• Find mean of: 3, 4, 5, 5, 8, 9, 9, 9, 13, 15
  sum of all data = 80
  total number of data = 10
  mean = \(\frac{80}{10}\) = 8

Median

• Median is the middle number when data is arranged in ascending order.
• If number of observation is even:
  Median =
  Example: Find the median of the data set: 3, 4, 5, 5, 8, 9, 9, 9, 13, 15
  Here, the number of observations is 10, so the average of the underlined numbers will be the median.
  Median = $\frac{\mathrm{8\; +\; 9}}{2}$
  Median = 8.5
• If number of observation is odd: Median = ($\frac{\mathrm{n\; +\; 1}}{2}$)th term
  Example: Find the median of the data set: 3, 4, 5, 5, 8, 9, 9, 9, 13, 15, 16
  Here, the number of observations is 11, so the underlined number will be the median.
  Median = 9

Mode

• Mode is the most common number in the dataset or the value that appears most frequently.
• Example: Find the mode of the data set: 3, 4, 5, 5, 8, 9, 9, 9, 13, 15, 16
  Here, 9 occurs more frequently than any other number, so it is the mode.
  Mode = 9

Variance

• Variance is the expected value of the squared variation of a random variable from its mean value or variance is the measure of how data points differ from the mean.
• How to find variance?
  1. Find the mean of the given data.
  2. Now subtract the average (mean) from each value and square them.
  3. Find the average of these squared values.
• 
  Where (mean) x = $\frac{\mathrm{x1\; +\; x2\; +\; x3\; +\; x4\; +\; x5\; +\; .......\; +\; xn}}{\mathrm{n}}$ and n = number of observatons.
• Example: Find variance for the following data:
  3, 4, 5, 5, 8, 9, 9, 9, 13, 15
  mean = 8

  +-----------------------------+
  |  x    |mean-x|   (mean-x)^2 |   
  |------------------------------
  |  3        5           25    |
  |  4        4           16    |
  |  5        3           9     |
  |  5        3           9     |
  |  8        0           0     |
  |  9        1           1     |
  |  9        1           1     |          
  |  9        1           1     |
  |  13       5           25    |
  |  15       7           49    |
  +-----------------------------+
                          136
                          

  sum of all (mean-x)² = 136
  n (number of observations) = 10
  variance = $\frac{136}{10}$ = 13.6

Standard deviation

• Standard deviation is the positive square root of the variance. It is abbreviated as SD and denoted by σ.
  It tells about the value that how much it has deviated from the mean value.
• Steps to find standard deviation:
  1. Find the mean of the observations.
  2. Find the squared differences from the mean.
     (the dae value - mean)²
  3. Find the average of the squared difference.
     Variance = $\frac{\mathrm{the\; sum\; of\; squared\; difference}}{\mathrm{the\; number\; of\; observations}}$
  4. Find the square root of the variances.
     standard deviation = √(Variance)

Why do we need a sample?

• As it is nearly impossible to collect data from each and every individual of the population, sampling helps us attain information about the entire population.
• While the results from a sample may not be completely accurate, they provide a close approximation of the population. It is crucial that the selected sample is representative of the population and not biased in any manner.
• For example, consider a study aiming to understand the preferences of coffee drinkers in a city with a large population. Instead of surveying every coffee drinker, which would be impractical, researchers can select a sample of coffee drinkers representing different demographics, such as age, gender, and location. By analyzing this representative sample, researchers can make informed conclusions about the preferences of the entire coffee-drinking population in the city.

Sampling techniques

1. Simple Random Sampling (SRS):
   • In Simple Random Sampling, everyone in the population has an equal chance of being picked. It's like picking names out of a hat without looking.
   • Example: If you want to know what students think about a new school rule, you put all their names in a box and randomly pick some names. Each student has the same chance of being chosen for the survey.
2. Stratified Sampling:
   • In Stratified Sampling, you divide the population into groups (like by age or gender), then randomly pick from each group. This ensures you get opinions from all types of people.
   • Example: If you're studying favorite colors among kids, you group them by age (e.g., 5-7, 8-10, 11-13) and randomly choose kids from each group to get a balanced view.
3. Cluster Sampling:
   • In Cluster Sampling, the population is divided into clusters or groups, and then clusters are randomly selected. All individuals within the selected clusters are included in the sample.
   • Cluster sampling can be done in two ways:
     1. Single-stage Cluster Sampling: In this method, all elements within the selected clusters are included in the sample.
     2. Two-stage Cluster Sampling: Here, a random sample of clusters is selected, and then a random sample of individuals is taken from each selected cluster.
4. Systematic Sampling:
   • In Systematic Sampling, you pick every nth person from a list after a random start. It's like selecting every 5th person from a line.
   • Example: If you want to survey shoppers in a mall, you start by randomly choosing a shopper and then survey every 10th shopper after that.
5. Convenience Sampling:
   • Convenience Sampling is picking whoever is easiest to reach. It's quick but may not represent everyone well.
   • Example: If you ask people in a shopping mall about their favorite TV show, you're using convenience sampling because you're only surveying those who happen to be there at that time.

• Note: The sampling techniques — simple, cluster, stratified, and systematic — are all probability sampling techniques and involve randomization. However, convenience sampling is a non-probability (or non-random) sampling technique because it relies on the researcher’s ability to select the sample. Non-probability sampling techniques can lead to biased samples and results.

Resampling Techniques

Resampling is a technique used in statistics and machine learning to manipulate the composition of a dataset by adjusting the distribution of its instances. The primary goal of resampling is to address specific issues within the dataset, such as imbalances, outliers, or to improve the generalization performance of a model. There are several common methods of resampling:

1. Oversampling:
   • Definition: Oversampling involves increasing the number of instances in the minority class to balance the class distribution.
   • Purpose: It is used to address imbalances in the dataset, especially when one class is underrepresented.
   • Methods: Techniques include random oversampling, Synthetic Minority Over-sampling Technique (SMOTE), and Adaptive Synthetic Sampling (ADASYN).
   • Example: Suppose you have a dataset with 90% positive cases and 10% negative cases. By oversampling the negative cases, you can create a more balanced dataset for training a classification model.
2. Undersampling:
   • Definition: Undersampling involves reducing the number of instances in the majority class to balance the class distribution.
   • Purpose: It is employed to mitigate imbalances where one class is overrepresented, which can lead to biased models.
   • Methods: Common approaches include random undersampling, Cluster Centroids, Tomek Links, and Edited Nearest Neighbors.
   • Example: In a dataset with 80% positive cases and 20% negative cases, undersampling the positive cases can help balance the class distribution for better model training.
3. Bootstrapping:
   • Definition: Bootstrapping entails creating multiple subsets of the dataset by sampling with replacement.
   • Purpose: It is often used in model training and validation to improve model robustness and assess variability in performance.
   • Methods: Techniques include random bootstrapping and Bagging (Bootstrap Aggregating) in ensemble methods.
   • Example: When training a machine learning model, bootstrapping can help create diverse subsets of the data to train different parts of the model, enhancing overall performance.
4. Cross-Validation:
   • Definition: Cross-validation involves dividing the dataset into multiple subsets for training and testing to assess model performance.
   • Purpose: It helps evaluate a model's ability to generalize by testing it on different subsets of the data.
   • Methods: Common types include k-fold cross-validation, stratified cross-validation, Leave-One-Out Cross-Validation (LOOCV), and Time Series Cross-Validation.
   • Example: In machine learning, cross-validation is used to estimate a model's performance on unseen data by training and testing it on different subsets of the dataset.

Statistical Inference

Statistical inference is a process in statistics that involves drawing conclusions or making predictions about a population based on a sample of data taken from that population. It encompasses the use of statistical methods to make inferences about the characteristics of a larger group using the information obtained from a subset of that group.

There are two main branches of statistical inference:

1. Descriptive Statistics:
   • Definition: Descriptive statistics involve summarizing and describing the main features of a dataset.
   • Inferential Statistics: Inferential statistics involve making inferences or predictions about a population based on a sample of data.
2. Key Concepts in Statistical Inference:
   • Population and Sample: Population refers to the entire group under consideration, while a sample is a subset used to make inferences.
   • Parameter and Statistic: A parameter is a numerical summary of a population characteristic, while a statistic is a numerical summary of a sample characteristic.
   • Hypothesis Testing: A formal procedure for comparing observed data with a hypothesis to test a population parameter.
   • Confidence Intervals: An interval estimate for a population parameter providing a range of likely values.
   • Regression Analysis: A method to examine the relationship between variables in a dataset.
   • Bayesian Inference: An approach that incorporates prior knowledge and beliefs about parameters.

SRSWR and SRSWOR

• If we have a population with a finite size (N), then the sample will also have a finite number of elements. Sampling from the population can be done in two ways:
1. SRSWR (Simple Random Sampling with Replacement): In this method, after an element is selected, it is placed back into the population before the next selection.
2. SRSWOR (Simple Random Sampling without Replacement): Here, once an element is selected, it is not placed back into the population for subsequent selections.
• Suppose the population size is denoted as N, the sample size as n, and we want to select K samples:
• If using SRSWR, the formula for determining K is K = Nⁿ. This means there are N raised to the power of n possible combinations, considering that elements can be selected more than once.
• For SRSWOR, the formula for K is K = ^NC_n, representing the combinations of selecting n elements from N without replacement.
• Example: Let's consider a scenario where N = 4 (population size) and the sample size (K) = 2:
  • For SRSWR: K = 4² = 16. This means we can select 16 samples when using SRSWR because elements can be chosen more than once.
  • For SRSWOR: K = $\frac{\mathrm{4!}}{\mathrm{(4-2)!\; *\; 2!}}$ = 6. Without replacement, we can select only 6 samples from the population.

Sampling Distribution

• Sampling distribution is essentially a theoretical distribution that represents the possible values of a sample statistic, such as the mean or standard deviation, that would be obtained from different samples of the same size taken from a population.
• We can analyze various types of sampling distributions, including the sampling distribution of the mean, sampling distribution of the standard deviation, and so on, depending on the specific statistic of interest.
• Let's consider a scenario where we have a population with a size of N, a sample size of n, and we take k samples from this population. In each sample, there will be its own set of statistics parameters like mean, median, standard deviation, and so on. For instance, the mean for sample 1 might be denoted as x1, sample 2 as x2, and so forth, similarly for standard deviation. Therefore, the sampling distribution of the mean will comprise a dataset containing each mean value and its associated probability.
  

Standard Error

• When we calculate the standard deviation of the sampling distribution of the mean, it is referred to as the standard error of the mean, denoted as S. E(x).
• Similarly, we can determine the standard error of the standard deviation and the standard error of variance, indicating that each statistical measure has its own specific standard error.

Mean of Sampling Distribution of Mean

• The mean of the sampling distribution of the mean, denoted as μx, is equal to the population mean (μ) irrespective of whether we use simple random sampling with replacement (SRSWR) or without replacement (SRSWOR).

Standard Deviation of Sampling Distribution of Mean

• The standard deviation of the sampling distribution of the mean, also known as the standard error of the mean, is calculated as σx = σ / √n, where σ represents the population standard deviation and n is the sample size.
• It's important to note that as we increase the sample size, the standard error of the mean decreases. This is because larger sample sizes lead to less variability in sample means and therefore a more precise estimate of the population mean.

Use of Standard Error:

1. Setting Confidence Intervals: Standard error plays a crucial role in establishing confidence intervals. Confidence intervals provide a range of values within which we are confident that the true population parameter lies. The width of the confidence interval is influenced by the standard error, with narrower intervals indicating greater precision in estimation.
2. Testing of Hypotheses: Standard error is integral to hypothesis testing, particularly in calculating test statistics such as t-tests and z-tests. These tests compare sample statistics to population parameters and help determine whether observed differences are statistically significant. The standard error is used in the computation of test statistics, aiding in the assessment of hypothesis validity.

Sampling Distribution and Standard Error of Mean Numerical

A population consists of the four members: 3, 7, 11, 15
Case 1: Consider all possible sample size two which can be drawn with replacement from population
Find the population mean, population standard deviation, the mean of the sampling distribution of mean and standard deviation of sampling distribution of mean.

Sol:
Members: 3, 7, 11, 15
size of population (N) = 4
According to case 1 (SRSWR)
Total sample (k) = Nⁿ
where n = sample size (2) and N = 4
So, k = 4² = 16

+---------------------------------------------+
|  S.No   |  Sample Variable  |  Sample mean  |
+---------------------------------------------+
|    1    |        3, 7       |    10/2 = 5   |
|    2    |        3, 11      |          7    |
|    3    |        3, 15      |          9    |
|    4    |        3, 3       |          3    |
|    5    |        7, 7       |          7    |
|    6    |        7, 3       |          5    |
|    7    |        7, 11      |          9    |
|    8    |        7, 15      |          11   |
|    9    |        11, 11     |          11   |
|    10   |        11, 3      |          7    |
|    11   |        11, 7      |          9    |
|    12   |        11, 15     |          13   |
|    13   |        15, 15     |          15   |
|    14   |        15, 3      |          9    |
|    15   |        15, 7      |          11   |
|    16   |        15, 11     |          13   |
+---------------------------------------------+
                     

Sampling distribution of mean with replacement will be

+---------------------------------------------------------------------------------------------+
|  Sample mean x |   3    |   5    |   7    |   9    |  11    |  13    |  15    | Total       |
+---------------------------------------------------------------------------------------------+
|  Probability   |  1/16  |  2/16  |  3/16  |  4/16  |  3/16  |  2/16  |  1/16  |  16/16 = 1  |
+---------------------------------------------------------------------------------------------+

We calculate probability by the number of occurrences in the sample mean. For example, if the occurrence of 9 is 4 times out of a total of 16 occurrences, then the probability is 4/16.
Next, we will find the mean of the sampling distribution of the mean and the standard error of the sampling distribution of the mean. Additionally, we know that the mean of the sample distribution of the mean is equal to the population mean (μ), so we will first find μ.
Population:

+-------------------------+
|  x  |  x-μ    |  (x-μ)2 |
+-------------------------+
|  3  |    6    |    36   |
|  7  |    2    |    4    |
|  11 |    2    |    4    |
|  15 |    6    |    36   |
+-------------------------+
                

μ = ( 3 + 7 + 11 + 15 ) / 4 = 9
∑(x-μ)² = 80
variance σ² = (∑(x-μ)²)N
σ² = 80/4 = 20
SD, σ = $\sqrt{20}$
Now we found statistic parameter of population, so we'll work on sample.
mean of sampling distribution of mean:
For this multiply each element of sample mean x to its probability and add it up.
E(x) = 3*1/16 + 5*2/16 + 7*3/16 + 9*4/16 + 11*3/16 + 13*2/16 + 15*1/16
144/16 = 9
E(x) = 9
and it is equal to μ
Now variance, Var(x) = E(x)² - [E(x)]²
To find E(x)² we just have to square the sample mean x element and multiply to its probability and add it up.
E(x)² = 9*1/16 + 25*2/16 + 49*3/16 + 81*4/16 + 121*3/16 + 169*2/16 + 255*1/16
E(x)² = 91
Now Var(x) = 91 - 81
Var(x) = 10
SD (x) =$\sqrt{10}$
Now find standard error of sampling distribution of mean S.E(x) = σx = = = = $\sqrt{10}$

Case 2: SRSWOR
here k = ^NC_n
k = ⁴C₂
k = $\frac{\mathrm{4!}}{\mathrm{2!\; *\; 2!}}$
k = $\frac{\mathrm{4\; *\; 3\; *\; 2}}{\mathrm{2\; *\; 2}}$
k = 6

+---------------------------------------------+
|  S.No   |  Sample Variable  |  Sample mean  |
+---------------------------------------------+
|    1    |        3, 7       |    10/2 = 5   |
|    2    |        3, 11      |          7    |
|    3    |        3, 15      |          9    |
|    4    |        7, 11      |          9    |
|    5    |        7, 15      |         11    |
|    6    |        11, 15     |         13    |
+---------------------------------------------+
                     

Sampling distribution of mean without replacement will be

+--------------------------------------------------------------------------+
|  Sample mean x |   5    |   7    |   9    |   11    |  13    | Total     |
+--------------------------------------------------------------------------+
|  Probability   |  1/6   |  1/6   |  2/6   |  1/6    |  1/6   |  6/6 = 1  |
+--------------------------------------------------------------------------+

E(x) = 5*1/6 + 7*1/6 + 9*2/6 + 11*1/6 + 13*1/6
54/6 = 9
E(x) = 9
Now variance, Var(x) = E(x)² - [E(x)]²
E(x)² = 25*1/6 + 49*1/6 + 81*2/6 + 121*1/6 + 169*1/6
E(x)² = $\frac{263}{6}$
Now Var(x) = $\frac{263}{6}$ - 91 = $\frac{20}{3}$
Var(x) = $\frac{20}{3}$
SD (x) =
Now find standard error of sampling distribution of mean S.E(x) = σx remember here, formula will not work because there is a thing called population correction factor, and we use population correction factor when $\frac{\mathrm{n}}{\mathrm{N}}$ > 0.05, then in this case we will we use finite correction factor whose formula is
In our case $\frac{\mathrm{n}}{\mathrm{N}}$ = 2 / 5 = 0.5 which is greater than 0.05 then the formula of S.E(x) = σx = * $\frac{\mathrm{N\; -\; n}}{\mathrm{N\; -\; 1}}$ = * $\frac{\mathrm{4\; -\; 2}}{\mathrm{4\; -\; 1}}$ = = SD (x)

Central Limit Theorem

• The Central Limit Theorem (CLT) states that if the population is normally distributed with a mean (μ) and standard deviation (σ), then regardless of the sample size, the sampling distribution of the sample mean (x̅) is also normally distributed. The mean of the sampling distribution of the sample mean is equal to the population mean (μx = μ), and the standard deviation of the sampling distribution of the sample mean is given by the formula: $\frac{\mathrm{\sigma }}{\sqrt{\mathrm{n}}}$ where σ is the population standard deviation and n is the sample size.
• When we say the population is normal, it means that the data follows a bell-shaped curve with a symmetrical distribution around the mean.
• When the sample size (n) is greater than or equal to 30, the sample is considered a large sample. Conversely, when n is less than 30, it is considered a small sample.
• However, what happens when the population distribution is non-normal, such as skewed or multimodal distributions? In such cases, the Central Limit Theorem still holds true. Even if the population is not normally distributed, as long as the sample size is large (n ≥ 30), the sampling distribution of the sample mean (x̅) will approximate a normal distribution. This is a key property of the Central Limit Theorem, making it a powerful tool in statistical inference.

Types of Statistics

Statistics can be classified into two main categories:

1. Descriptive Statistics: Descriptive statistics involve summarizing and describing the main features of a dataset. This includes measures of central tendency (such as mean, median, and mode), measures of variability (like range, variance, and standard deviation), and graphical representations (such as histograms and box plots). Descriptive statistics help us understand the characteristics of the data without making inferences about a larger population.
2. Inferential Statistics: Inferential statistics go beyond describing the data and involve making inferences or predictions about a population based on a sample of data. It allows researchers to draw conclusions about population parameters (such as means or proportions) using statistical methods. Inferential statistics are used in hypothesis testing, estimating population parameters, and making predictions.

For example, suppose we have collected data on the heights of students in a school. Descriptive statistics would help us summarize this data, such as calculating the average height of students or determining the range of heights. Inferential statistics, on the other hand, would allow us to make predictions about the average height of all students in the school based on our sample data.

Statistical Inference

• Statistical inference is the process of analyzing data and drawing conclusions from it, considering random variations in the data. It's a branch of statistics called inferential statistics.
• Hypothesis testing and confidence intervals are practical applications of statistical inference, helping us make decisions based on data.
• Statistical inference is a method used to make decisions about the characteristics of a population by analyzing a sample from that population. It helps assess relationships between variables and estimate uncertainty or variation between samples. The main purpose is to provide a likely range of values for population parameters. Key components for making statistical inference include:
  1. Sample Size: The number of observations in the sample, influencing the precision of estimates.
  2. Variability in the Sample: The degree of diversity or spread among data points in the sample, affecting the reliability of conclusions.
  3. Size of Observed Differences: The magnitude of differences between groups or variables, which indicates the strength of relationships or effects.

Types of Statistical Inference

Statistical inference encompasses various methods used to draw conclusions from data. Some common types of statistical inference include:

1. One Sample Hypothesis Testing: This involves testing a hypothesis about the population mean, proportion, or variance using data from a single sample.
2. Confidence Interval: Confidence intervals provide a range of values within which we are confident that the true population parameter lies, based on sample data.
3. Pearson Correlation: Pearson correlation measures the strength and direction of the linear relationship between two continuous variables.
4. Bivariate Regression: Bivariate regression analyzes the relationship between two variables, typically a predictor (independent) variable and an outcome (dependent) variable.
5. Multivariate Regression: Multivariate regression extends bivariate regression to analyze the relationship between multiple predictor variables and an outcome variable.
6. Chi-Square Statistics and Contingency Table: Chi-square tests assess the independence or association between categorical variables using contingency tables.
7. ANOVA or T-Test: Analysis of Variance (ANOVA) and T-tests are used to compare means of two or more groups to determine if there are significant differences between them.

Statistical Inference Procedure

The procedure involved in inferential statistics includes several steps:

1. Begin with a Theory: Start with a theoretical framework or concept that you want to investigate.
2. Create a Research Hypothesis: Formulate a specific statement or prediction about the relationship between variables based on the theory.
3. Operationalize the Variable: Define how you will measure or manipulate the variables in your study to test the hypothesis.
4. Recognize the Target Population: Identify the population to which you intend to generalize your study results.
5. Formulate a Null Hypothesis: Develop a null hypothesis that represents the absence of an effect or relationship in the population.
6. Collect a Sample: Gather a representative sample from the target population to conduct your study.
7. Conduct Statistical Tests: Use statistical tests to analyze the sample data and determine if the observed differences or effects are significant enough to reject the null hypothesis and support the research hypothesis.
   • Hypothesis Mean: This refers to the mean or average value predicted by the research hypothesis based on theory and previous research.
   • Null Hypothesis Mean: The null hypothesis predicts that there is no significant difference or effect, so the null hypothesis mean represents what would be expected if there were no effect.

For example, suppose we want to study the effect of a new teaching method on student performance. We start with the theory that the new method will improve learning outcomes (theory). Our research hypothesis states that students exposed to the new method will have higher test scores than those taught using traditional methods. We operationalize variables by defining test scores as the measure of performance. We recognize all high school students as our target population. The null hypothesis states that there is no difference in test scores between the two teaching methods. We collect a sample of students from different schools and conduct statistical tests (such as t-tests or ANOVA) to compare test scores between the two groups and determine if the observed differences are statistically significant.

Statistical Inference Solution

Statistical inference solutions involve the efficient utilization of statistical data pertaining to groups of individuals or trials. This process encompasses data collection, investigation, analysis, and organization. Through statistical inference solutions, individuals can gain valuable insights across various fields. Here are some key facts about statistical inference solutions:

• Assumption of Independence: A common approach is to assume that the observed sample consists of independent observations from a specific population type, such as Poisson or normal distributions.
• Evaluation of Parameters: Statistical inference solutions are used to assess the parameters of an expected model, such as the normal mean or binomial proportion. This involves estimating and interpreting key numerical characteristics of the data.

Importance of Statistical Inference

Statistical inference plays a crucial role in examining data and deriving meaningful conclusions. Proper data analysis is essential for making accurate interpretations of research results. It is particularly vital for predicting future observations across various fields, enabling us to draw inferences about the data. The significance of statistical inference extends to a wide range of applications in different sectors, including:

• Business Analysis: Utilized for analyzing market trends, customer behavior, and making data-driven decisions for business growth.
• Artificial Intelligence: Integral in developing and training AI models by extracting insights from data and improving model performance.
• Financial Analysis: Essential for assessing financial trends, risk analysis, portfolio management, and investment decision-making.
• Fraud Detection: Used to identify irregular patterns or anomalies in data that may indicate fraudulent activities.
• Machine Learning: Forms the foundation for building predictive models, classification algorithms, and regression analysis in ML applications.
• Share Market: Employed for analyzing stock trends, predicting market movements, and making informed investment strategies.
• Pharmaceutical Sector: Utilized in clinical trials, drug efficacy studies, and medical research for drawing conclusions about treatment outcomes and drug effects.

Statistical Inference Examples

Question: From the shuffled pack of cards, a card is drawn. This trail is repeated for 400 times, and the suits are given below:

+-----------------------------------------------------------+
|    Suit       |  Spade  |  Clubs  |  Hearts  |  Diamonds  |
+-----------------------------------------------------------+
| No. of times  |   90    |  100    |    120   |     90     |    
| drawn         |         |         |          |            |
+-----------------------------------------------------------+
                

While a card is tried at random, then what is the probability of getting a:

1. Diamond cards
2. Black cards
3. Except for spade

Solution:
By statistical inference solution,
Total number of events = 400
i.e., 90 + 100 + 120 + 90 = 400

1. The probability of getting diamond cards:
   Number of trials in which diamond card is drwan = 90
   Therefore, p(diamond card) = 90/100 = 0.225
2. The probability of getting black cards:
   Number of trials in which black card showed up = 90 (Spades are black cards) + 100 (clubs are also black cards) = 190
   Therefore, P(black card) = 190 / 400 = 0.475
3. Except for spade:
   Number of trials other than spade showed up = 90 + 100 + 120 = 310
   Therefore, P(except spade) = 310 / 400 = 0.775

Objectives of Multivariate Data Analysis:

• Identify Relationships: Explore and identify relationships, correlations, and dependencies among multiple variables in a dataset.
• Reduce Dimensionality: Reduce the dimensionality of data by extracting meaningful patterns and underlying factors to simplify analysis.
• Make Predictions: Build predictive models to forecast outcomes and make informed decisions based on multivariate data patterns.
• Segmentation: Segment data into meaningful groups or clusters based on similarities and differences among variables.

Advantages of Multivariate Data Analysis:

• Comprehensive Insights: Provides comprehensive insights into complex relationships among multiple variables.
• Improved Decision Making: Helps in making data-driven decisions by uncovering hidden patterns and trends.
• Predictive Capabilities: Enables predictive modeling to forecast future trends and outcomes.
• Effective Communication: Facilitates effective communication of results and findings to stakeholders.

Disadvantages of Multivariate Data Analysis:

• Data Complexity: Dealing with large datasets and complex relationships can be challenging and require specialized expertise.
• Assumption Violation: Some multivariate techniques may assume specific data distribution or relationships that may not always hold true.
• Interpretation Challenges: Interpreting results from multivariate analysis can be complex, requiring careful consideration of multiple variables and their interactions.
• Computational Resources: Certain multivariate techniques may require significant computational resources and processing time.

Multivariate Analysis Techniques: Dependence vs. Interdependence

When performing multivariate analysis, various techniques are employed to understand the relationships between variables. These techniques can be broadly categorized into two groups based on the nature of relationships:

• Dependence Methods: Dependence methods focus on identifying and analyzing the extent of dependency or correlation between variables within a dataset. These methods are useful for exploring how changes in one variable affect another and assessing the strength and direction of these relationships. Examples of dependence methods include correlation analysis, regression analysis, and covariance analysis.
• Interdependence Methods: Interdependence methods go beyond simple dependency and aim to uncover complex interrelationships among multiple variables. These methods explore how variables interact and influence each other in a more intricate manner. They are valuable for detecting patterns, clusters, and structural relationships within the data. Examples of interdependence methods include factor analysis, cluster analysis, and structural equation modeling (SEM).

Multiple techniques comes under dependence and interdependence like:

• Dependence methods:
  1. Multiple linear regression
  2. Multiple logistic regression
  3. Multivariate analysis of variance (MANOvA)
• Interdependence methods:
  1. Factor analysis
  2. Cluster analysis.

1- Multiple Linear Regression

• Definition: Multiple linear regression is a statistical method used to analyze the linear relationship between multiple independent variables and a single dependent variable. It estimates the impact of each independent variable on the dependent variable while holding other variables constant.
• Example: Suppose you want to predict a student's final exam score based on variables like study hours, previous exam scores, and attendance. Multiple linear regression can help model this relationship and predict the final exam score using the given predictors.

2- Multiple Logistic Regression

• Definition: Multiple logistic regression is a statistical technique used when the dependent variable is binary (e.g., yes/no, success/failure). It models the relationship between multiple independent variables and the probability of the binary outcome. It's commonly used in classification problems.
• Example: Predicting whether a customer will purchase a product (yes/no) based on factors like age, income, and previous purchase history using multiple logistic regression.

3- Multivariate Analysis of Variance (MANOVA)

• Definition: MANOVA is a statistical technique used to compare the means of multiple dependent variables across two or more groups to determine if there are significant differences between the groups. It's an extension of ANOVA to multiple dependent variables simultaneously.
• Example: Analyzing the effect of different teaching methods (grouped as independent variables) on students' performance scores in math, science, and literature (dependent variables) using MANOVA.

4- Factor Analysis

• Definition: Factor analysis is a statistical method used to identify underlying latent factors or dimensions that explain the correlations among a set of observed variables. It reduces the dimensionality of the data by grouping related variables into factors.
• Example: Identifying the underlying factors influencing customer satisfaction by analyzing survey responses related to product quality, customer service, pricing, and brand reputation using factor analysis.

5- Cluster Analysis

• Definition: Cluster analysis is a method used to group similar observations or variables into clusters based on their characteristics or patterns. It helps in identifying natural groupings within the data.
• Example: Segmenting customers into different market segments based on their purchasing behavior, demographics, and preferences using cluster analysis to tailor marketing strategies for each segment.

Linear Regression

• Definition: Linear regression is a statistical method used to model the relationship between a dependent variable (usually continuous) and one or more independent variables by fitting a linear equation to the observed data.
• Example: Predicting house prices based on features like square footage, number of bedrooms, and location using linear regression.

Logistic Regression

• Definition: Logistic regression is a statistical technique used when the dependent variable is binary (e.g., yes/no, 0/1). It models the probability of the binary outcome using one or more independent variables.
• Example: Predicting whether a patient has a disease (yes/no) based on factors like age, gender, and medical history using logistic regression.