{
"cells": [
{
"cell_type": "markdown",
"id": "chicken-alliance",
"metadata": {
"tags": []
},
"source": [
"# Interpreting the coefficients of a linear regression\n",
"\n",
"A regression coefficient describes how much the response variable changes for a unit change of a covariate while all other covariates remain constant.\n",
"\n",
"In this notebook, we will deepen this intuition with a hands-on example."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "filled-relation",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"import statsmodels.formula.api as smf\n",
"\n",
"import seaborn as sns\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "developed-living",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"sns.set_context('poster')"
]
},
{
"cell_type": "markdown",
"id": "reported-estimate",
"metadata": {
"tags": []
},
"source": [
"## Generate data\n",
"\n",
"We generate synthetic data using a form of structural equation modeling.\n",
"This way, we can check whether we are able to recover the coefficients."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "grand-washer",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"N = 1000\n",
"\n",
"beta_g1 = 1.4\n",
"beta_g2 = -0.8\n",
"\n",
"mean_g1 = -2\n",
"mean_g2 = 10"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "southwest-impact",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"np.random.seed(42)\n",
"\n",
"X = np.random.normal(size=N * 2)\n",
"Y = np.r_[\n",
" beta_g1 * X[: int(len(X) / 2)] + np.random.normal(mean_g1, size=N),\n",
" beta_g2 * X[int(len(X) / 2) :] + np.random.normal(mean_g2, size=N),\n",
"]\n",
"group = ['$G_1$'] * N + ['$G_2$'] * N"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "democratic-framing",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" X | \n",
" Y | \n",
" group | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 0.496714 | \n",
" -1.979778 | \n",
" $G_1$ | \n",
"
\n",
" \n",
" | 1 | \n",
" -0.138264 | \n",
" -2.338089 | \n",
" $G_1$ | \n",
"
\n",
" \n",
" | 2 | \n",
" 0.647689 | \n",
" -1.885656 | \n",
" $G_1$ | \n",
"
\n",
" \n",
" | 3 | \n",
" 1.523030 | \n",
" -0.175720 | \n",
" $G_1$ | \n",
"
\n",
" \n",
" | 4 | \n",
" -0.234153 | \n",
" -4.221429 | \n",
" $G_1$ | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
" | coef | std err | t | P>|t| | [0.025 | 0.975] | \n",
"
\n",
"\n",
" | Intercept | -1.9946 | 0.032 | -62.740 | 0.000 | -2.057 | -1.932 | \n",
"
\n",
"\n",
" | group[T.$G_2$] | 11.9799 | 0.045 | 266.149 | 0.000 | 11.892 | 12.068 | \n",
"
\n",
"\n",
" | X | 1.4222 | 0.032 | 43.793 | 0.000 | 1.359 | 1.486 | \n",
"
\n",
"\n",
" | X:group[T.$G_2$] | -2.2785 | 0.046 | -50.067 | 0.000 | -2.368 | -2.189 | \n",
"
\n",
"
"
],
"text/plain": [
""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"res = fit.summary()\n",
"res.tables[1]"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "continental-place",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"Intercept -1.994595\n",
"group[T.$G_2$] 11.979866\n",
"X 1.422225\n",
"X:group[T.$G_2$] -2.278550\n",
"dtype: float64"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"coefs = fit.params\n",
"coefs"
]
},
{
"cell_type": "markdown",
"id": "greek-house",
"metadata": {
"tags": []
},
"source": [
"### Understand their meaning"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "appreciated-lease",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"fitted_beta_g1 = coefs['X']\n",
"fitted_beta_g2 = coefs['X'] + coefs['X:group[T.$G_2$]']\n",
"\n",
"fitted_mean_g1 = coefs['Intercept']\n",
"fitted_mean_g2 = coefs['Intercept'] + coefs['group[T.$G_2$]']"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "sustainable-aerospace",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" label | \n",
" true_value | \n",
" fitted_value | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" beta_g1 | \n",
" 1.4 | \n",
" 1.422225 | \n",
"
\n",
" \n",
" | 1 | \n",
" beta_g2 | \n",
" -0.8 | \n",
" -0.856325 | \n",
"
\n",
" \n",
" | 2 | \n",
" mean_g1 | \n",
" -2.0 | \n",
" -1.994595 | \n",
"
\n",
" \n",
" | 3 | \n",
" mean_g2 | \n",
" 10.0 | \n",
" 9.985271 | \n",
"
\n",
" \n",
"
\n",
"