Miskolci Egyetem Gazdaságtudományi Kar Üzleti Információgazdálkodási és Módszertani Intézet. Correlation & Regression

Miskolci Egyetem Gazdaságtudományi Kar Üzleti Információgazdálkodási és Módszertani Intézet

Correlation & Regression


Types of dependence • association – between nominal data • mixed – between a nominal and a ratio data • correlation – among ratio data


Correlation describes the strength of a relationship, the degree to which one variable is linearly related to another

Regression shows us how to determine the nature of a relationship between two or more variables

• X (or X1, X2, … , Xp): known variable(s) / independent variable(s) / predictor(s) • Y: unknown variable / dependent variable

• causal relationship: X “causes” Y to change


Correlation Measures 1. Covariance

2. Coefficient of correlation 3. Coefficient of determination

4. Coefficient of rank correlation


1. Covariance • A measure of the joint variation of the two variables; • An average value of the product of the deviations of observations on 2 random variables from their sample means.

x  x    y  y   Cx, y   n 1

– – – –

ranges from -  to +; C = 0, when X and Y are uncorrelated; its sign shows the direction of correlation it doesn’t measure the degree of relationship!!!


2. Linear correlation coefficient Σd x d y C r = s xs y d 2x d 2y

• • • • •

Pearson correlation A measure of how closely related two data series are. Its sign shows the direction of correlation It measures the strength of correlation 0 < r  < 1  statistical dependence r = 0  X and Y are uncorrelated  r = -1  negative ☻ r = 1  positive ☺ • You can use only in case of linear relationship!


3. Coefficient of determination • r2 • The square of the sample correlation coefficient between the dependent and independent variables. • Measures the degree of correlation in percentage (%)

• Shows how many percent of the variance of dependent variable is explained by the independent variable. • Varies from 0 to 1. 2

r 

S yˆ Sy

=1-

Se Sy


No relationship 4000 3000 Number of 2000 births

1000 0 0

10

20 30 Number of storks

40


Independence Y

=

-7 .4 E -0 2 + R -S q =

3

3 .4

0 .2 0 8 3 4 8 X %

2

1

0

-1

-2

-3

-2

-1

0

1

N i n c s k o r r e lá c i ó

2


Positive correlation Y = - 8 . 6 E -0 2 + 0 . 6 9 0 2 8 6 X 3

R -S q = 6 2 .5 %

2

1

0

-1

-2

-3

-3

-2

-1

0

1

2

P o z i t ív k o r r e l á c i ó

3


Negative correlation Y 3

=

5 .0 7 E -0 2 R -S q =

- 0 .6 4 7 8 7 2 X

7 0 .9

%

2

1

0

-1

-2

-3

-3

-2

-1

0

1

N e g a t ív k o r r e l á c i ó

2

3


Curvilinear relation Y

=

1 2 .0 9 5 8

+

6 .0 7 6 8 4 X + R -S q

4 0

=

8 8 .4

1 .1 6 6 8 6 X**2 %

3 0

2 0

1 0

0

-3

-2

-1

N e m

0

1

l i n e á r i s k o r r e lá c i ó

2

3


Scatter diagrams 1600

linear

5000

S a 1200 l e s 800 i n

400

$

0

S e l l i n g

0

10

20

30

4000 p r 3000 i c 2000 e 1000 0

40

0

Advertising in $ 50

curvilinear

w a s t a g e

2

4 6 8 Age of a house (year)

10

12

4000

40

S e l l i n g

30 20 10 0

3000 p r i c e

2000

1000

0

0

10

20

30

Production (number of products per day)

direct relationship positive slope

40

0

5 10 Age of a car (year)

15

inverse relationship negative slope


Example • A firm administers a test to sales trainees before they go into the field. The management of the firm is interested in determining the relationship between the test scores and the sales made by the trainees at the end of one year in the field. The following data were collected for 45 sales personnel who have been in the field one year. • Calculate different correlation measures!




X

Y

independent

dependent variable

xi  x  d x yi  y  d y xi  x   yi  y   d xd y

Salesperson

Test score

Number of units sold

K. A.

25

188

+9

+22

+198

L. Z.

16

157

0

-9

0

B. E.

30

165

+14

-1

-14

G. P.

5

124

-11

-42

+462

…

…

…

…

…

…

…

…

…

…

…

…

S. G.

10

158

-6

-8

+48

J. T.

24

224

+8

+58

+464

V. P.

17

169

+1

+3

+3

T. L.

6

114

-10

-52

+520

716

7 464

0

0

∑dxdy=8 894.5

Total


Number of observed pairs: n = 45 x  16

s x  8.26

y  166

s y  30.99

d  C

x

dy

n 1

8 894.5   202.15 45 - 1

Positive correlation


C 202.15 r    0.7897 sx  sy 8.26  30.99 

r 2  62.36 %

There is a strong & positive relation between test scores and number of units sold. The test scores explain 62.36 percent of the variation of number of units sold.


4. Coefficient of rank correlation ρ 1-

6   d i2

n  (n 2  1)

0   1

• Spearman correlation • Measure of the relationship between two ordinal data • n = number of paired observations, d = difference between the ranks for each pair of observations. • perfect correlation  ρ= 1  perfect inverse correlation  ρ = -1  in case of independence  ρ= 0


Example Ten students were ranked by mathematical and musical ability: Student A B C D E F G

H

I

their J

Total

Ability Mathematics

1

2

3

4

5

6

7

8

9 10

-

Music

3

4

1

2

5

7

10

6

8

9

-

-2 -2

2

2

0

-1 -3 2

1

1

0

4

4

4

0

1

1

1

32

di = xi - yi

d i2

6   d i2

4

6  32 ρ 11 0.806 2 2 n  (n  1) 10  (10 - 1)

9

4

strong relationship


Simple Linear Regression Model • We model the relationship between two variables, X and Y as a straight line. • The model contains two parameters:  an intercept parameter,  a slope parameter. y = β0 + β1x + ε where: y – dependent or response variable (the variable

E (y)

β1 = slope β0 = y-intercept x

we wish to explain or predict) x – independent or predictor variable ε – random error component β0 – y-intercept of the line, i.e. point at which the line intercept the y-axis β1 – slope of the line


Assumptions of the Linear Regression Model • Assumptions for Error term: – Normally distributed; – Expected value = 0 (E(ε)=0); – The variance is the same for all observations (Homoscadasticity); – Uncorrelated across observations (there isn’t any autocorrelation).

• Assumptions for the Independent Variables: – Not random, etc.


Deterministic component y

• y = deterministic component + random error

• We always assume that the mean value of the random error equals 0  the mean value of y equals the deterministic component.

ŷi = b0 + bixi Random error

x

• It is possible to find many lines for which the sum of the errors is equal to 0, but there is one (and only one) line for which the SSE (sum of squares of the errors) is a minimum:  Least squares line / regression line.


• The method of least squares gives us the best linear unbiased estimators of the regression parameters: β0, β1. • The least-squares estimators: b0 estimates β0 b1 estimates β1 • The regression line: y caret („hat”):

Ŷ = b0 + b1X

• The normal equations (with 1 x) Σy = nb0 + b1Σx Σxy = b0Σx + b1Σx2


Interpretation • b0: when x=0, y=b0 • b1: for every 1 unit increase in x we expect y to change by b1 units


Elasticity % change in x demanded % change in y

x E(y, x)  b1 b 0  b1x

x E(y, x) = b 1 y Elasticity at the mean


Estimation in Regression • Regression estimation is a technique used to replace missing values in data. • If we know:

1. The estimated parameter value; 2. The hypothesized value of the parameter; 3. Confidence interval around the estimated parameter. • The number of degrees of freedom equals the number of observations minus the number of parameters estimated.

•  = n-2


Estimation in Regression Parameter 0 1

Estimated value Standard error b0

yˆ 0

Y0

ˆ0 y

x i2 n(x i  x) 2

se

b1

0

b0  t  sb0

se

(x i  x) 2 se

se

(x 0  x) 2 1  n (x i  x) 2

(x 0  x) 2 1 1 + n (x i  x) 2

 = n-2

b1  t  sb1 ˆ  t  s yˆ y ˆ  t  s yˆ y

In case of average Y values In case of discrete Y values


Analysis of Variance in Regression Analysis Sum of Squares 2 Regression Syˆ = (yˆ i  y) Residual

Se = (yi  yˆ ) 2

Total

S y =  (y i  y)

2

1

Mean Sum of Squares S yˆ

n-2

s e2  S e /(n  2)

Df

n-1

S 2y  S 2yˆ  S e2 n

n

n

i =1

i =1

i =1

2 2 2 ˆ (y  y )  ( y  y ) + (y  y )  i  i  i

Sy n -1

F F=

Syˆ Se /(n - 2)


Model testing H0: β1 = 0 H :   H1: β1 ≠ 0 (linear model)

H1 :  1   2

Pr

Pr

1

Test statistic:

F=

Syˆ s

2 e

 0

1

2

Syˆ Se /(n - 2)

1 F1 ( 2 ; 1 )

H0 1

0 F

F

1

 ( 2 ; 1 ) 2

F

1

 ( 1 ; 2 ) 2

• F-statistic tests whether all the slope coefficients in a linear regression are equal to 0.

• Measures how well the regression equation explains the variation in the dependent variable.

F


Parameter testing H1 :   m0

H1 :   m0

Pr

H0: β1 = 0 H1: β1 ≠ 0 t1

Pr

H0 0

b1 Test statistic: t  s (b1 )

t1 / 2

0

t1 / 2

where: b1 is the least square estimate of the regression slope s(b1) is the standard error of b1


Seminar


Exercise 1 Book: p185 e44

In a bar waiters believe that there is a relationship between the amount of consumption of cola and the average daily temperature. To test it a sample of 20 days was drawn and they examined the amount of consumption and the temperature in these days:

Miskolci Egyetem Gazdaságtudományi Kar Üzleti Információgazdálkodási és Módszertani Intézet Day

The amount of consumption (l)

The maximum daily temperature (°C)

1.

520

25

2.

534

26

3.

610

28

4.

780

32

5.

708

27

6.

639

25

 y 2  7,505,555;  d x 2  179

7.

486

23

 d y  149,923;  d x d y  4495

8.

423

20

9.

452

22

10.

597

29

11.

640

30

12.

657

31

13.

678

30

14.

620

27

15.

635

28

16.

610

26

17.

585

25

18.

627

27

19.

608

26

20.

720

30

• Results:  y  12,129;  x  537;  xy  330,159;  x  14,597; 2

2

• Determine the relationship between the temperature and the consumption in case of linear and curvilinear relationship.


Exercise 2 (p188 e48) Country

Export (X)

Import (Y)

Austria

406

418

Belgium

87

93

Czech Republic

60

95

France

134

172

Holland

100

102

Poland

95

67

 d x d y  1,195,957;  x  2,948

Great-Britain

119

136

 y  3, 071;  x 2  2, 084, 046;

Germany

219

291

Italy

181

363

Russia

41

68

Switzerland

27

49

Sweden

49

75

Slovakia

54

21

Slovenia

47

53

Ukraine

1329

1068

• The export and import of Hungary with European countries are the following:

 y 2  1, 628,345

• Characterize the trade with European countries.


Exercise 3 p188 e48 • The table shows the inflation rate (x) and the unemployment rate (y) of Germany between 1972 and 1997. • Results:

Year

Inflation rate (%)

Unemployment rate (%)

1972

5.5

1.1

1973

6.9

1.2

1974

7.0

2.6

 x  92.4;  y  171.8

.

.

.

 d x 2  94.54;  d y 2  195.44

.

.

.

 xy  512.9

.

.

.

1996

1.5

11.5

1997

1.8

9.8

• Determine the relationship between unemployment and inflation rate.


Thanks for your attention!

Miskolci Egyetem Gazdaságtudományi Kar Üzleti Információgazdálkodási és Módszertani Intézet. Correlation & Regression

Recommend Documents