{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Régression polynômiale et pipeline\n", "\n", "Le notebook compare plusieurs de modèles de régression polynômiale." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from teachpyx.datasets import load_wines_dataset\n", "\n", "data = load_wines_dataset()\n", "X = data.drop([\"quality\", \"color\"], axis=1)\n", "y = data[\"quality\"]" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(X, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On normalise les données. Pour ce cas particulier, c'est d'autant plus important que les polynômes prendront de très grandes valeurs si cela n'est pas fait et les librairies de calculs n'aiment pas les ordres de grandeurs trop différents." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import Normalizer\n", "\n", "norm = Normalizer()\n", "X_train_norm = norm.fit_transform(X_train)\n", "X_test_norm = norm.transform(X_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "La transformation [PolynomialFeatures](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html) créée de nouvelles features en multipliant les variables les unes avec les autres. Pour le degré deux et trois features $a, b, c$, on obtient les nouvelles features : $1, a, b, c, a^2, ab, ac, b^2, bc, c^2$." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 0.1909065078664849 0.16570749381482386 0.02195639999990817\n", "2 0.31686272332465504 0.2634484656108902 0.16658860000006825\n", "3 0.4117084105383497 -1.446755311176299 1.0382120000001578\n", "4 0.5940872457783092 -3926.677572477097 2.8583189999999377\n" ] } ], "source": [ "from time import perf_counter\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.metrics import r2_score\n", "\n", "r2ts = []\n", "r2es = []\n", "degs = []\n", "tts = []\n", "models = []\n", "\n", "for d in range(1, 5):\n", " begin = perf_counter()\n", " pipe = make_pipeline(PolynomialFeatures(degree=d), LinearRegression())\n", " pipe.fit(X_train_norm, y_train)\n", " duree = perf_counter() - begin\n", " r2t = r2_score(y_train, pipe.predict(X_train_norm))\n", " r2e = r2_score(y_test, pipe.predict(X_test_norm))\n", " degs.append(d)\n", " r2ts.append(r2t)\n", " r2es.append(r2e)\n", " tts.append(duree)\n", " models.append(pipe)\n", " print(d, r2t, r2e, duree)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
tempsr2_trainr2_test
degré
10.0219560.1909070.165707
20.1665890.3168630.263448
31.0382120.411708-1.446755
42.8583190.594087-3926.677572
\n", "
" ], "text/plain": [ " temps r2_train r2_test\n", "degré \n", "1 0.021956 0.190907 0.165707\n", "2 0.166589 0.316863 0.263448\n", "3 1.038212 0.411708 -1.446755\n", "4 2.858319 0.594087 -3926.677572" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas\n", "\n", "df = pandas.DataFrame(dict(temps=tts, r2_train=r2ts, r2_test=r2es, degré=degs))\n", "df.set_index(\"degré\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Le polynômes de degré 2 paraît le meilleur modèle. Le temps de calcul est multiplié par 10 à chaque fois, ce qui correspond au nombre de features. On voit néanmoins que l'ajout de features croisée fonctionne sur ce jeu de données. Mais au delà de 3, la régression produit des résultats très mauvais sur la base de test alors qu'ils continuent d'augmenter sur la base d'apprentissage. Voyons cela un peu plus en détail." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAAEMCAYAAAAmmGWrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XlcVNX/+PHXGfZ9R0UEXBIRV6C0RbO0LE2rj5XlJ03TssVspSxbrDTzq/0qbfm0W2aZWWmkpZYtmiu45YYbiyDKIsi+DJzfHzMYKpsKzIDv5+PBA5h77r3vO1ze855zz9yjtNYIIYQQQgjRUhksHYAQQgghhBCNSQpeIYQQQgjRoknBK4QQQgghWjQpeIUQQgghRIsmBa8QQgghhGjRpOAVQgghhBAtmhS8TUQpZVBK/aiUuq+WNiFKKa2UsjX//rNS6p7z2FeQUipfKWVzITG3dGc+382ZUmqAUirF0nEI0VJIzrY+krPFhZCC9xwppRKVUkXm5HRcKfWZUsq1HqvOAH7TWn9U331prW/UWn9ez5gGVVkvWWvtqrUur+++hBCiJZKcLYQAKXjP1zCttSsQAVwKPH9mA2Vy6vnVWj+rtX67CWNslqSHw7q0hJ4UIZCc3WgkZ1sXydk1k4L3AmitU4GfgW4ASqk/lFIzlFJ/A4VAB6WUh1LqE6VUmlIqVSk1vTJBKKVslFJzlFKZSqnDwNCq2zdvb0KV3+9TSu1VSuUppfYopSKUUguAICDG3IPxdDWX2QLMl+ZOKKUOVr1Ep5SappRarJT6wrzd3UqpqCrLnzHHnaeUildKDazuuVBKDTHHlGdu/5T58bFKqXVntNVKqU7mn+crpd5XSq1QShUA11Sz7fZKqT/N216tlHpHKfWledlZl4Wq9p6YL0tOUUodUkplmY/Vu4ZjaJT9KKUclVJfmh/PUUptUUq1qmb/U5RSS8547G2l1Fzzz+Oq/P0PK6UmVncc5rYBSqnvlFIZSqkEpdTkKsvmK6WmV/n9tGMzH9czSqmdQIFSyra+54EQ1kxy9mmxSs6WnH1RkYL3Aiil2gFDgG1VHh4N3A+4AUnA54AR6AT0Bq4HKhPifcBN5sejgNtq2dftwDRgDOAODAeytNajgWTMPRha6/+rZvWvgRQgwLyP1844+YcDiwBP4EfgHfM+Q4FJwKVaazdgMJBYQ4ifABPN7boBa2o6lmqMwnT50A1YV83yr4A4wBd4FTiXMXKTgVuAqzEdfzbwbg1tG2s/9wAeQDvAB3gAKKpmG18DQ5RS7nCq5+QOc1wA6ZjOF3dgHPCmUirizI0oUy9VDLADaAsMBB5TSg0+h+O5C9OLuSfQkfqfB0JYLcnZp5GcLTn74qK1lq9z+MJ00uQDOZiS43uAk3nZH8ArVdq2Akoql5sfuwv43fzzGuCBKsuuBzRgW2V7E8w/rwQerSWmQVV+D6ncDqZ/2HLArcrymcB888/TgF+rLOsKFJl/7oTpH3YQYFfH85IMTATcz3h8LLDujMc00Mn883zgi1q2G4TpxcelymNfAV+afx4ApNT0fAB7gYFVlrUByszPTdXnqTH3cy+wHuhRj/NrHTDG/PN1wKFa2i6tPCeqxgf0AZLPaPss8FmV53x6lWWnHZv5uO6t8nu9zwP5ki9r+0Jydk3Pi+RsydkX1Zf08J6fW7TWnlrrYK31Q1rrqu/8jlT5ORiwA9LMl0VygA8Af/PygDPaJ9Wyz3bAofOINQA4obXOO2M/bav8fqzKz4WAo1LKVmt9EHgMU4JNV0otUkoF1LCfEZh6TpLMl5guP4cYj9SyLADI1loXnBF/fQUDP1R5/vdiejE58/JUY+5nAaYXv0VKqaNKqf9TStnVsJ2vML3AgqkXpbKnAKXUjUqpjcp0mTMH0/PtW0MsAZWxmNs+V80x1+bU3+QczwMhrJHk7LNJzpacfVGRgrfh6So/H8HUW+BrTraeWmt3rXW4eXkapqRYKaiW7R7BdJmirn2e6SjgrZRyO2M/qbWs8++Gtf5Ka30Vpn9IDcyqod0WrfXNmF4YlgKLzYsKAOfKdkqp1ucYfxrgpZRyOSP+Smdu3wbwq7L8CHBjleffU2vtqE1j+ZpkP1rrMq31y1rrrsAVmC5xjanheL8FBiilAoFbMSdPpZQD8B0wB2iltfYEVgCqmm0cARLOiMVNaz2kumMB6vyb1Pc8EKIZkpwtOVty9kVACt5GpLVOA1YBbyil3M2D5Dsqpa42N1kMTFZKBSqlvIAptWzuY+AppVSkMumklAo2LzsOdKghhiOYLs3MVKaB+D2A8cDCuuJXSoUqpa41/+MWYxrDdNZtc5RS9kqp/yqlPLTWZUBulXY7gHClVC+llCOmd5z1prVOAmKBl837uQoYVqXJfky9G0PN78CfBxyqLP8fMKPyuVJK+Smlbm7K/SilrlFKdTcn3FxMl82qvf2Q1joD02XRzzAlwL3mRfbm/WUARqXUjZgup1ZnM5CrTB9acFKmD9p0U0pdal6+HdO4M2/zi9ljNWwHc/z1Og+EaO4kZwOSsyVnt1BS8Da+MZhO/D2YBsUvwTRWCOAjTJdNdgBbge9r2ojW+ltMHxL4CsjD9I688pOrM4HnzZdCnqpm9bswjX06CvwAvKS1Xl2P2B2A14FMTJfQ/DFdZqnOaCBRKZWLaYD/3ea49wOvAL8CB6j+Aw51GYVpjNMJ4CXgi8oFWuuTwEOYXlxSMb0TrvrJ3LcxfahjlVIqD9ho3lZT7qc1pr97LqbLZn8CX9ZyvF9hGnt16tKY+fLmZEwvuNnmWH+sbmVtupfnMKAXkIDp7/cxpg9hgOly3Q5M475WAd/UEguc23kgRHMnOVtytuTsFkhpXduVCSGsj1JqGqYPUNzdEvYjhBAtmeRsYQ2kh1cIIYQQQrRoUvAKIYQQQogWTYY0CCGEEEKIFk16eIUQQgghRIsmBa9o9sy311ll6TiEEEIIYZ2k4BXNilIqRCmllVK2lY9prRdqrWu6v6EQQohGoJRKVEoNusBtjFVKnc+tz4Q4J1LwiiZVtVAVQgghhGgKUvCKU5RSU5RSh5RSeUqpPUqpW82Pj1VK/a2UmqeUOqmU2qeUGlhlvT+UUjOVUpvNy5cppbzNyyp7ZMcrpZKBNebH+yql1ptvvL5DKTXgjO29at5nnlJqlVKqcv7xv8zfc5RS+Uqpy6v2EJhnNHpTKZVujmWnUqqbedkQ83HlKaVSK2/4rpTyUkr9pJTKUEplm38OrBJPe6XUX+b1flVKvauU+rLK8hqPRQghWiKl1AJMU/nGmHPx03Xk9bFKqcPmPJpgHooWhmnGs8vN28ix0OGIi4AUvKKqQ0A/TLO7vAx8qZSqnGGoD3AY8MU0o833lUWt2RjgXiAAMAJzz9j21UAYMFgp1RZYDkzHNPPQU8B3Sqmq85yPAsZhmiHG3twGoL/5u6fW2lVrveGM/VxvbtMZ8ARGAlnmZZ8AE7XWbkA3zMU3pv+DzzDNOR6EaRrGd6ps8ytMUz/6YJpmc3TlgnoeixBCtCha69FAMjBMa+2KaerjanOhUsoF02vCjeb8ewWw3TwN7wPABnM+97TEsYiLgxS84hSt9bda66Na6wqt9TeYppW8zLw4HXhLa11mXhYPDK2y+gKt9S6tdQHwAnCHeR7yStO01gVa6yJMU1iu0FqvMO9rNaY50YdUaf+Z1nq/uf1iTFMu1kcZ4AZ0wXTbvb1a67Qqy7oqpdy11tla663m487SWn+ntS40Twc5A1OBjlIqCLgUeFFrXaq1Xsfp00PW51iEEKKlqysXVgDdlFJOWus0rfVui0UqLkpS8IpTlFJjlFLbzZejcjD1glYOJUjVp9+0OQlTb26lI2css6uy7pnLg4HbK/dj3tdV/DtfPZjm/65UCLjW5xi01msw9c6+CxxXSn2olHI3Lx6BKfkmKaX+VEpdbj5uZ6XUB0qpJGWaV/4vwNNcsAcAJ7TWhRdwLEII0dLVmAvNHSEjMfXmpimlliululgyWHHxkYJXAKCUCgY+AiYBPuZLS7sAZW7SVimlqqwSBByt8nu7M5aVAZlVHqtaLB/B1CPsWeXLRWv9ej1CrXOmFK31XK11JBCOaWhDtPnxLVrrmzENk1iKqecY4EkgFOijtXbn32ETCkgDvJVSzjUc64UcixBCNGf1zuta65Va6+swdQbsw/R6c+Y2hGg0UvCKSi6YEk8GgFJqHKYe3kr+wGSllJ1S6nZM43FXVFl+t1Kqq7kwfAVYorUur2FfXwLDlFKDlVI2SilHpdSAqh8Uq0UGpktjHapbqJS6VCnVRyllBxQAxUC5Usre/CEJD611GZALVMbnhmncbo55XPJLldvTWidhuiw3zbyNy4FhDXQsQgjRnB3n31xcYy5USrVSSg03j+UtAfL5N/8eBwKVUvZNH764mEjBKwDQWu8B3gA2YEpA3YG/qzTZBFyCqdd2BnCb1jqryvIFwHxMQxEcgcm17OsIcDPwHKYC9gimXtg6z0fz0IIZwN/my2Z9z2jijqnnIBvT0IosYI552Wgg0Txs4QFMY84A3gKczMe2EfjljG3+F7jcvK3pwDeYkvYFHYsQQjRzM4HnzcMXRlJzLjRgupJ2FDiB6TMSD5m3sQbYDRxTSmUiRCNRpw/LFOJsSqmxwASt9VU1LP8D+FJr/XFTxmUpSqlvgH1a65fqbCyEEEIIi5NeKCHqYB4m0VEpZVBK3YCpF2OppeMSQgghRP3Uq+BVSj2qlNqllNqtlHqssYMSwsq0Bv7ANO5sLvCg1nqbRSMSohaSs4UQ4nR1DmlQplmqFmG6H2sppvGND2qtDzR+eEIIIc6F5GwhhDhbfXp4w4CN5pvyG4E/gVsbNywhhBDnSXK2EEKcwbYebXYBM5RSPphu3TQE022aTqOUuh+4H8DFxSWySxe5p7QQovmJi4vL1Fo356mhJWcLIS4a9c3Z9bpLg1JqPPAwpjGMe4AirfXjNbWPiorSsbFn5VchhLB6Sqk4rXWUpeO4EJKzhRAXi/rm7Hp9aE1r/YnWOkJr3R/TPfRkLJgQQlgpydlCCHG6+gxpQCnlr7VOV0oFAf/BdBN+IYQQVkhythBCnK5eBS/wnXk8WBnwsNY6uxFjEkIIcWEkZwshRBX1Kni11v0aOxAhhHUpKysjJSWF4uJiS4fSKBwdHQkMDMTOzs7SoTQ4ydlCXHwkZ9euvj28QoiLTEpKCm5uboSEhKCUsnQ4DUprTVZWFikpKbRv397S4QghxAWTnF07mVpYCFGt4uJifHx8WlziBFBK4ePj02J7QoQQFx/J2bWTglcIUaOWmDgrteRjE0JcnFpyXrvQY5OCVwghhBBCtGhS8AohmoVp06YxZ86cOttlZWVxzTXX4OrqyqRJk5ogMiGEENWxprwtH1oTQrQojo6OvPrqq+zatYtdu3ZZOhwhhBB1aIq8LT28QgirNWPGDEJDQxk0aBDx8fH1WsfFxYWrrroKR0fHRo5OCCHEmaw1b0sPrxCiTrM2z2LfiX0Nus0u3l145rJnalweFxfHokWL2LZtG0ajkYiICCIjI5k9ezYLFy48q33//v2ZO3dug8YohBDNkSVyNlh33paCVwhhldauXcutt96Ks7MzAMOHDwcgOjqa6OhoS4YmhBCiGtact6XgFULUqa539Y2lutvQWENPgRBCWDNL5Wyw3rwtBa8Qwir179+fsWPHMmXKFIxGIzExMUycONEqegqEEEKczZrzthS8QgirFBERwciRI+nVqxfBwcH069ev3uuGhISQm5tLaWkpS5cuZdWqVXTt2rURoxVCCGHNeVsKXiGE1Zo6dSpTp0495/USExMbPhghhBB1sta8LbclE0IIIYQQLZoUvEIIIYQQokWTglcIIYQQQrRoUvAKIYQQQogWTQpeIYQQQgjRoknBK4QQQgghWjQpeIUQzcK0adOYM2dOne1Wr15NZGQk3bt3JzIykjVr1jRBdEIIIc5kTXlb7sMrhGhRfH19iYmJISAggF27djF48GBSU1MtHZYQQogaNEXelh5eIYTVmjFjBqGhoQwaNIj4+Ph6rdO7d28CAgIACA8Pp7i4mJKSksYMUwghhJm15m3p4RVC1OnYa69Rsndfg27TIawLrZ97rsblcXFxLFq0iG3btmE0GomIiCAyMpLZs2ezcOHCs9r379+fuXPnnvbYd999R+/evXFwcGjQ2IUQwppZImeDdedtKXiFEFZp7dq13HrrrTg7OwMwfPhwAKKjo4mOjq5z/d27d/PMM8+watWqRo1TCCGEiTXnbSl4hRB1qutdfWNRSp31WH16ClJSUrj11lv54osv6NixY6PHKYQQ1sRSORusN29LwSuEsEr9+/dn7NixTJkyBaPRSExMDBMnTqyzpyAnJ4ehQ4cyc+ZMrrzyyiaMWAghLm7WnLflQ2tCCKsUERHByJEj6dWrFyNGjKBfv371Wu+dd97h4MGDvPrqq/Tq1YtevXqRnp7eyNEKIYSw5ryttNZ1N1LqcWACoIF/gHFa6+Ka2kdFRenY2NgGC1II0fT27t1LWFiYpcNoVNUdo1IqTmsdZaGQGoTkbCEuPpKza1dnD69Sqi0wGYjSWncDbIA7zzNWIYQQjUhythBCnK2+QxpsASellC3gDBxtvJCEEEJcIMnZQghRRZ0Fr9Y6FZgDJANpwEmttdznRwghrJDkbCGEOFt9hjR4ATcD7YEAwEUpdXc17e5XSsUqpWIzMjIaPlIhhBB1kpwthBBnq8+QhkFAgtY6Q2tdBnwPXHFmI631h1rrKK11lJ+fX0PHKYQQon4kZwshxBnqU/AmA32VUs7KdDfhgcDexg1LCCHEeZKcLYQQZ6jPGN5NwBJgK6bb2xiADxs5LiGEOM20adOYM2dOne02b9586j6OPXv25IcffmiC6KyH5GwhhLWwprxdr5nWtNYvAS81+N6FEKKBdevWjdjYWGxtbUlLS6Nnz54MGzYMW9uLZ2JJydlCiOakKfK2zLQmhLBaM2bMIDQ0lEGDBhEfH1+vdZydnU8lyeLi4mrndRdCCNE4rDVvXzxdHkKI87Z28X4yj+Q36DZ927nS747ONS6Pi4tj0aJFbNu2DaPRSEREBJGRkcyePZuFCxee1b5///7MnTsXgE2bNnHvvfeSlJTEggULLqreXSGEsETOBuvO2/IqIISwSmvXruXWW2/F2dkZgOHDhwMQHR1NdHR0rev26dOH3bt3s3fvXu655x5uvPFGHB0dGz1mIYS4mFlz3paCVwhRp7re1TeW6i5r1aenoFJYWBguLi7s2rWLqKg6p1oXQogWwVI5G6w3b8sYXiGEVerfvz8//PADRUVF5OXlERMTA5h6CrZv337WV2XSTEhIwGg0ApCUlER8fDwhISGWOgwhhLhoWHPelh5eIYRVioiIYOTIkfTq1Yvg4GD69etXr/XWrVvH66+/jp2dHQaDgffeew9fX99GjlYIIYQ1522ltW7QDQJERUXp2NjYBt+uEKLp7N27l7CwMEuH0aiqO0alVJzW+qIa/yA5W4jmT3J27WRIgxBCCCGEaNGk4G3mdFkZub+sRJvHvgghhBBCiNNJwdvM5a5cRepjj5H5ocwcKoQQQghRHSl4m7nCLVsAyHzvfYr37rVwNEIIIYQQ1kcK3mauMDYWp969sfHy5OiUZ9GlpZYOSQghhBDCqkjB24wZs7IoPXQI12uvoc3Lr1ASH0/G++9bOiwhhBBCCKsiBW8zVhgXB4BzVBRu116Dxy23kPXhRxT984+FIxOi4U2bNo05c+bUu31ycjKurq7ntI4QQoiGY015WwreZqwwNhbl6IhTeDgArZ57FltfX44++ywVJSUWjk4Iy3r88ce58cYbLR2GEEKIemrMvC0FbzNWGBuLU69eKHt7AGzc3Wkz/VVKDx4ic948C0cnxIWbMWMGoaGhDBo0iPj4+Hqvt3TpUjp06EC4+c2gEEKIpmGteVumFm6myvPyKNm7D9+HHz7tcdd+/fC8/XayPv0M14EDce7d20IRipbk9/kfkp50uEG36R/cgWvG3l/j8ri4OBYtWsS2bdswGo1EREQQGRnJ7NmzWbhw4Vnt+/fvz9y5cykoKGDWrFmsXr1ahjMIIS5KlsjZYN15WwreZqpo61bQGueos2fT83/mafL/XkfalGdpv/QHDE5OFohQiAuzdu1abr31VpydnQEYPnw4ANHR0URHR9e43ksvvcTjjz+Oq6trk8QphBDCxJrzthS8zVRhbCzY2eHUs8dZy2xcXQl47TWSx44j4623aPXssxaIULQkdb2rbyxKqbMeq6unYNOmTSxZsoSnn36anJwcDAYDjo6OTJo0qSlCFkIIi7NUzgbrzdtS8DZThVticerWrcbeW5e+ffEaNYoTXyzAbdAgnC+9tIkjFOLC9O/fn7FjxzJlyhSMRiMxMTFMnDixzp6CtWvXnvp52rRpuLq6SrErhBBNwJrztnxorRmqKCqiaNcunKMia23n/+QT2AUGcvS5qVQUFDRRdEI0jIiICEaOHEmvXr0YMWIE/fr1s3RIQgghamHNeVtprRt8o1FRUTo2NrbBtytMCjZuJHnsONp98D9cr7661raFsbEkjR6D11130vrFF5soQtES7N27l7CwMEuH0aiqO0alVJzW+uzB8S2Y5Gwhmj/J2bWTHt5mqHBLLCiFU0REnW2do6LwHjOG7K++pmDDhiaITgghhBDCukjB2wwVxsbiENYFGzc3juYf5ZHfHuFYwbEa2/s9/hj2ISEcnTqV8vz8JoxUCCGEEMLypOBtZnRpKUXbt5+6Hdn3B77nj5Q/mLlpZo3rGBwdCXh9JsZjx0mfNaupQhUtQGMMebIWLfnYhBAXp5ac1y702KTgbWaKdu1Gl5TgHBWF1pqViStxtHFkzZE1/J78e43rOfXqhc/4e8n5dgn5f/3VhBGL5srR0ZGsrKwWmUC11mRlZeHo6GjpUIQQokFIzq6d3JasmSmMM32wxDkqigM5B0jMTWTKZVNYsn8Jr21+jT5t+uBs51ztur6TJpH3+++kPf8CHWJ+xMbDoylDF81MYGAgKSkpZGRkWDqURuHo6EhgYKClwxBCiAYhObt2UvA2M4Wxsdh37Iittzertn2FQRkYHDKYcJ9wRv88mvd3vM+TUU9Wu67BwYGA12eROHIkx1+bScCs15s4etGc2NnZ0b59e0uHIYQQoh4kZ9euziENSqlQpdT2Kl+5SqnHmiI4cTpdXk5R3NbThjNEtYrC18mXXv69GHHJCBbsWUD8ifgat+HULRzfifdzctky8tasacLoG8DOxfBmN5jmafq+c7GlI/qXNccGDR+ftR/vRazZ5Ww5l0R9ybnSMjXR37XOgldrHa+17qW17gVEAoXAD40SjahVSXw8Ffn5pw1nuD74+lPLH498HA8HD17Z+AoVuqLG7fg+8AAOXbqQ9uJLGLOzmyL0C7dzMcRMhpNHAG36HjPZOhKeNccGDR+ftR/vRa5Z5Ww5l0R9ybnSMjXh3/VcP7Q2EDiktU5q8EhEnQpjK8fvRrIqcRUGZWBg8MBTyz0cPHgq6il2Zuxkyf4lNW5H2dsT8PpMyk+e5Pir0xs97gbx2ytQVnT6Y2VFpsctzZpjg4aPz9qPV1Rl3TlbziVRX3KutExN+Hc914L3TuDr6hYope5XSsUqpWJb6oBpSyvcEotdYCC2rVufNpyhqps63ESf1n14K+4tMosya9yWY5cu+D30ILkrVpD7y8rGDv3CnUw5t8ebkjXHBg0fn7Ufr6jKunO2nEuivuRcaZma8O9a74JXKWUPDAe+rW651vpDrXWU1jrKz8+voeITZlprCmNjaxzOUEkpxdS+UykuL2b2ltm1btNnwgQcw8M59vLLGLOyGiv0huFRwycza3q8KVlzbNDw8Vn78QqgmeRsOZdEfcm50jI14d/1XHp4bwS2aq2PN3gUok6lhw9Tnp2N86VR1Q5nqKq9R3smdJ/AioQVrD+6vsZtKjs7Al6fSUV+PsemvWzd9+4b+CLYOZ3+mJ2T6XFLs+bYoOHjs/bjFZWsP2fLuSTqS84Vq5KVksx7E0axYMqj/P75RxzYvJ7C3JPnvqEm/Luey23J7qKGS2Oi8RVuMY3fdYqMZOW2R6odzlDV+O7jWZGwghkbZ/D9zd/jYONQbTuHSy7B79HJpM95g9yfluMx7KZGif+C9bjD9P23V0yXOjwCTf8QlY9bkjXHBg0fn7Ufr6hk/TlbziVRX3KuWA2tNb9//hEV5eU4OLuwc/XPbF2xDACfwCACw7oR2LUbgV3CcfX2qX1jTfh3VfXp1VNKOQNHgA5a6zpL+KioKB1r/oCVaBipT0VTuGkTFcs+4raY23i+z/OM7DKy1nU2HN3A/avvZ2KPiUzqPanGdrq8nKRR/6UkMZEOMT9i5+/f0OEL0WwopeK01lGWjuNCSM4WQjSWw1u38MOslxkw5j4ih96MsayM44cOkLJ3Fyn7dpO6bw9lxaYPonm2bmMqgM1f7n7+KKUaNJ765ux69fBqrQuBOsp00Vi01hRu2YLzpVH8kLS61uEMVV0ecDlDOwzlk12fMKTDEDp4dKi2nbKxoc3MmSTceivHXnyJwPffa/ATUgjRdCRnCyEaQ7nRyB9ffIxXm7b0GjwEAFs7O9p26UrbLl3pA1SUl5OeeNhUAO/dzcHNG9j1+2oA3Hz8CAwLJ7BrN9p2Ccfdrw0VRk25UWMsK8fVyxGDoXHqD5lprRkoS03FePw4ThGRrExcVOdwhqqeinqKv1L+YvrG6Xxy/Sc1FrIOHdrj/8TjHJ/5Oid/WIrnf25tyEMQQgghRDO3Y9VystNSGf7kC+z47SgFOSWUGysoL6ug3FiBsayCcqOm3FhOeVkQ5cZAvIIHUVJwnJL8JIqLkti3fjN71/1h2qByxmDbFoNtIAbbQO6dczMuno6NErsUvM1A5fjdjFA/EvckcnfY3fVe19fJl8cjH+eVDa8QcziG4R2H19jWa/Roclev5vjMmbhdew02np4XHLuwHlpr6bkXQghxXgpzT7J+yVcE9+jNsSQvdv52CHsnW2zsDNjYKmxsDdjaGbCxNWBjZ8DW3oCDi63pd9uCpIISAAAgAElEQVRO2Nh1xtbWgMFGUVacSX7WYXIzD5Fz7CAlBQcAKM7vh4tnQKPELwVvM1AYuwWDhwerDfH1Hs5Q1YhLRrDs4DLmbJlD/7b98XSsvpBVBgOtX3iRhFtuIevTz/B/4vGGCF9YgaMHclj96W6Cwn0Y8N9QKXyFEEKckw1LvqK0sIguV97On4tS6HZ1W66+K/Q8t9YZuOLUb7kZ6Rw9sA+fwMYpduHcJ54QFlAYG4tzZASrjqw+p+EMlQzKwAt9XyC3NJc3t75Za1vH0M64DxnCiQULrP/evKJOWmu2/5rM0je3UVZazp51R9m47LClwxJCCNGMZB5JYsfqnwkfcD1bfs7Fq7UzV4zo1GDbd/fzp8sV/Rtse9WRgtfKlaWnU5aUTEF4CAknE6qdbKI+Qr1DGdN1DN8f+J6tx7fW2tZ30sPokhKyPvzovPYlrENpkZGVH+7i7yUH6dDTlzHTr6BrvwC2/pLEzt9ldiIhhBB101rzxxcfY+/kRGlpFMX5ZVx3bzh29jaWDu2cSMFr5Yri4gDY1CrvvIYzVPVAzwdo49KGVze+Sll5WY3tHNq3x+OWW8j++mvKjlvvPetFzbJS81k8cwuHd2Ry5W2duH5CVxJ3bKTnAFfa9/Rl7eL9HIxLt3SYQgghrFzCtliSdm6jfcRQkvcU0vfmjvgFuVk6rHMmBa+VK9wSi3J24ge1/byGM1TlbOfMc32e42DOQT7f83mtbX0fegitNZn/+995709YRvzGNJa8HktZSTm3PN6bkO72LJn+PDFvvs6iac8QeYMbrdt7sPqz3aTGZ1s6XCGEEFaq8jZkHv5tOBLflrahXvQa1K5h95FfQO7q1RybPgNdUdGg266qxRa8Vj1N7jkojI1Fdw/lUH7ieQ9nqGpAuwEMDBrIBzs+ICWv5sva9oFt8bxtBDlLvqM0RS5/NwflZRX88VU8v87fi3+IO7dPieTYwb/4PPphjiccot+osRgMBr6f+QJX/McHD18nVry/k8yUfEuHLoQQwgptX2m6DZm9ywBs7ewYNDYM1QD3yS1NSeHEgi9JHj+BA5dfTuojkzm5bBllqakNEHX1WmTBG7sigW+mb6bgZImlQ7kg5Tk5lOzfz6FghwsezlDVlMumYFAGZmyaUesbA98HHkApReZ77zfIfkXjyc0q4vs5cez+K5WIwUH0H+lHzJsv8fv8D2kX1o175rzLZTffxu0vvAbAsjkv0v/OVtg52vLTvO3knSi28BEIIYSwJoW5J9nw3Vd4tu5CbnZrBvy3C65e53ePXG00UhgbS/qcORy66SYODbqO4zNmUJaWhtfo0QR98Tmd1/+NfbuG7T2uqsUVvPnZxcSuSCIrtYCYeTsoKTJaOqTzVrjV9OGyVV6pFzycoarWLq2Z1HsS61LXsTppdY3t7Fq1wuuuuzi5dCklhxMaZN+i4SXtzmLxa1vIOV7IDfeHY2u7ky+nPEpWShI3PPQ4t06ZhruvHwA+ge24/YUZVBiNLJ/7MgNGtaastIKYudspzq95XLcQQoiLi+k2ZIUUFl5G2JUBdIr0P6f1y0+e5OTy5aRGP82BK68i6e7RZM3/HFs/P1o9O4WOK3+h44rltHo6GpfLLkPZ2TXSkZi0uII3dkUiWmsG/DeU7KMF/Pz+TsrLGm9MSGMq3BIL9nasc09rkOEMVd3V5S66eHdh1uZZ5JfWfEnb5/77UI6OZL7zToPuX1y4igrN5pjD/PTODlw9HRk0rg2bvvs//vryU4J79mbsnPcIv3rgWffc9W0XzG3PT8dYUsKqD6bT/87W5GYWs/y9HZSVllvoaIQQQliLytuQObj1xrN1O/rdcUmd62itKTl8mKxPPiVp9Bj2X3ElR598ioJ163C95hravvUWnTesJ/izz/C+5x7sg4Ob4Ej+1aIK3pz0Qvb+nUb4VQGE92vLtfeEkbo/h9Wf7aGiovmN6S2MjSW7gx/ldjYNNpyhkq3Blhf7vkhGUQbzts2ruZ2PD96jR5O7YgXF8fENGoM4f0X5pfz0zg62LE8k9DI/gsISWPZ/T5N97ChDHnmKm596HldvnxrX9w/pwG3PT6e0qJC1X87iyttacSwhl1Uf76aivHm+QRRCCHHhKm9DpgwOYNOH6+7tir1j9fOU6dJSCjZs4Nhrr3Fo8A0cHjKU9NmzKc/Lw+e+CYQs+ppL1q0l4PWZuN8wGBs3y93doUUVvFt+SsBgo4gcEgJAaJ/WXDGiE4e2prNu8YFm9UG28vwCivfsYWtAcYMOZ6iqu193RoaO5Ot9X7M7c3eN7XzuHYfBzY2MeTUXxqLpHE/IZfGMLRzdn0PUDa4c2/8JG75dQIfIyxj7xnuEXTWgXjOptWrfkdumTqc4P5/N3/8/LhvqR+LOTP78en+z+l8RQgjRcCpvQ2aw68Nlw7rSur3HWW2MJ06Q+uRT7L/8CpLH3UvON4txaN+e1tNeotOa3+iw9Af8H3sMp169UDbWcb/eFlPwZqXms3/LcXpcG4iLh8Opx3tfF0SvQe34548U4n5JsmCE56Zo+3YoL2e9f06DD2eoanLEZHydfHl5w8uUV1R/OdvGwwPvcWPJ//U3iv7Z1WixiNpprfnnjxS+nxMHqoJOvRJZv/g1cjMzGPb4FIY/8Rwunl5nrVdRWMiJLxdyaMhQjs98/bTbvrTueAkjnnuFwtwc/vl1Ht0HeLJn3VG2/CRjtoUQ4mJTbjSy5rMPMdh4ERjWj8gbzh52UHY8naTRY8j77Tfchw4l8L336LxxA+0++B9ed96JXUDjTQ98IVpMwbvpx8PYO9jQ+/pgytLTKYyNPbXsiv90ovNlrdi07DB7/j5qwSjrrzB2CxUGxcHAhh/OUJWbvRtPX/Y0e0/sZVH8ohrbeY8Zg42nJxlz5zZaLKJmZSXl/PrZHv5atB+/oGJ06TfsWL2Ezn2vZOwb79G571VnrWPMziZj3jscvHYgx6dPR5cbOfH55xx76SV0+b9vbtpcEsp/nn2F/OxsDm36kI6RrmxZnsjutY13exghhBDWZ9svMZxMT8PB/RquG98Dg83pZWJZaipJo0djTEsj6KMPafPKy7hdew0GJycLRVx/LaLgPZZwkoQdmfS+Pgh7m3I23z+emClPcPCFF9ClpSiD4toxYbTr6s0fC+NJ3Jlp6ZDrVLgllpS29nRvd2mjDGeoanDwYK4MuJJ52+ZxvKD6mdVsXF3xuW8CBWvXUmie/U00jexjBSyZFcv+zan4t9vDkR3/ozjvJDc/9TxDJ0fj7H765abSlBSOvTqdg9dcS+a77+IUEUHwVwvp+Msv+D70IDnfLiHtuefQxn/vYNI2NIz/THmJ3KwM0vZ+RtvOjvz5VTyHt2c09eEKIYSwgMLck/z9zVcYbIO5duyNuPueXsSWJiWRePdoynNyCPrsU5wvvdRCkZ6fFlHwblp2GCc3O3pc247UWbPYaigjxcedmH1xrBw9kqIjydjYGrjh/m74tXNl5Ue7OHb4pKXDrlFFSQlFO3ewPaC0UYczVFJKMbXvVIwVRmZtmVVjO69Ro7Dx8yXjrbdljGcTORiXzrczY8nNSMLe9juSd/5Clyuv5p433qPTpX1Pa1u8Zw+pTzzJoesHk714Me5DhtBh+U+0e+9dnCMiUErhN3kyfo89xsllP5L6VDS67N9bkQWGdeM/z7xEbkY6OUcX4t3WhlWf7CbtYE5TH7YQQogm9tun8zGWFtPx0v8Q2qf1actKDh4k6e7R6OJigj+fj1PPnhaK8vw1+4I3Zd8JUvZlE3lDCCUb1xH3+ypK7WwZ9sSzBIV0YjdlfP7oRPZ99SX2jrYMfbgnLp4O/PTuDk6kFVg6/GoV79wJZUbi2zXucIaq2rm1Y2KPiaxOWs1fKX9V28bg5ITvxAco3LKFwg0bmiSui9nmnxL45cPtKL2egowvqSgv5tZnXuLGh5/AydX0SVetNQXr15N873gS/jOC/D//xHvcWDr9upqA12bg0LHjWdv1fWAi/s88Q94vv5Dy2ONUlJaeWtYuvAe3RL/AyWNplOQswcVDs/y9nZw4ap3/K0IIIS5c2sHD7N/wK47uvRl8f//TPvhcvHcvSaPHoNEEL/gCx7AwC0Z6/pp1wau1ZuOyw7h6ORAaZs/hF54nwd+LLpf3o3OfKxkxey7Dxj4ANjYsX7aIJQ9PwFiSw7DJvTDYGIiZu538bOubYapgi2n8sVNE70YfzlDV2PCxdPDowKsbX+VYwbFq23jecTu2bdqQ/rb08jamPX8fZdPSDVC2iNzjfxM+YBD3zHmXDhGmS0jaaCR3xQoSR9xG8r3jKT6wH78nn6DT72toFR2NXatWtW7fZ9xYWr34Avm//UbKw5OoKP73/yC4Ry9ufmoq2WlHqCj5AWUoJWbedvKzm/fMhUIIIc6mtSbmzXmg7Bny8AQcnP+dAKJoxw6S7hmLcnQkZMECHDp1smCkF6ZZF7yJ/2RxPCGXqBuDSX/pBfa52KHsbLlq1NhTbTrfeBP3fvwl3Tz8OJJ+lE8njWffmu8Y8kA4JUVGYubtoLjAumaYytq0liQ/6N91aJPu187Gjtf6vUZ+aT7jV46vdjyvwd4e34cepHjHTvL/+KNJ4wM4GLuJ3Mz0Jt9vU0rdn82a+X9QVrAEe8cKRjz3CoMfmIyjiysVRUWcWLiQQzfcSOoTT1JRVESb6a/S6bff8L3vPmzc3eu9H+9Ro2gzYzoF69ZxZOIDVBQWnloW0iuS4U9N5URqMrbEUFxQQMy87ZQUWtf/ihBCiAuzfsmv5GUeIKTXENr3Cjr1eOGWLSSPuxcbDw9CvlyAfUiI5YJsAM224NUVmk3LDuHh50Srw2s4GruJFA8Xet84HA//03u37D08uf6DTxlxw3/wyStg3feLWPHGk0QOtiEnvZAV7+/EaCUzTGmjkbLt/7AvyNBkwxmqCvcJ5/1B75NVnMX4VeNJLzy7uPS85RbsgoLImDvvtFtcNbakndtZNvtVFr30DHknrP+Dh+cjJ72Qn+b9Smn+97j7ejNq+huE9Iww3XHh3XdNd1x4dTq2Pj4EvjOPDst/wvO22zDY25/X/jxHjCDg/2ZRGBtL8oT7KM//d9a9Dr0vZdgTz3LiaCIO9ivITsthxfv/YCyzjv8VIS5WukKTfayA+E3H+H3RHubPXcX6+C21zpopRHVOZuSxeekX2Dr4MOyx0acez//7b5Lvux/b1q0J/nIBdm3bWjDKhqEa47J0VFSUjq1yW7DGsH/LMVZ/socBN3pheP4eYntcQq6jPePnfoSji2uN6xX98w9x0U+w08mGIntb2ob1ITOtJx16hXDD/d3OugVHUyvcuZOkO0YSM7YzT09ZZrE4tqdvZ+Lqifg7+/Pp4E/xc/Y7bfnJH3/k6NPP0Patt3C/YXCjx2MsK+OL6EkYy0opKcjHzcePkS/POjWWtSUoLihj0au/cCLpC5w9nBn16mwcS8o4MX8+Od99hy4qwnXAAHwmjMcpMrJek0vUV+4vK0l96ikcu3Yl6KMPsfH4984PBzatJ+at1/EK6EhB/g10igjg+vu6YTA03P6tiVIqTmsdZek4mlJT5GxxfrTW5GYWkZ6UR3pSHhlJuaQn51FWbHrjWW4oowJNnsMJloW/jb+3D128u5z25efk16D5QrQMukLzxbPvkZn4M9dNnEKPa023t8xb8zupjz6KfceOBH3yMbY+Nc/aaQ3qm7OrnyvOypWXV7D5xwS82zjj8NELpLXyId1YwjW33VNrsQvg1L07fb9eTNunn2bn/t0cZjPKbjsHNvXF0dWGa+7uatHEcGTtLwB07D+sXu2NZeUc3p5BqxB3PPycGyyOXv69eH/Q+zzw6wOMXzWeTwd/etp4YvehQ8n84EMy5s3D7bpBjT6TSuyP35GdlsqI517BYGPD9zNf4odZL3P789Oxc3Bs1H03hfLyCmLm/klW4pc4uTpwx9RXKPzgI1K//hqUwuOmm/AZfy8Ol9Q9n/n5cL9hMMrentRHHyVp7DhTkvP2BuCSPlcwdHI0y9+ejWfrlRzcOhjnb+zpd2dneREVogFprck7UUyGubhNT8olIzmPkkLTLQQNtgrfQDcuucyfvTZb+fbEl9j7VPBowLMkft2ae1Ne5FDI7+w9sYfVSatPbdfb0fusIjjYPRiDal4XectKS4j/+y8CQrviHdD8exwtbfPyvWQmrcGnXVe6X3MlALk//0xq9NM4hoWZOj88PZssnmJjMY62jfd63iwL3vgNxziZUcTl3nsoOXiIA9f0wdPejp7X3Viv9W29vAj54APcPviAtv97n32XBHHc5k+2/7yb8tL/ct346xr5CGp2fP3vGL3h6t631Kv9usUH2L3WNJmGd4ALIT18ad/Dl1Yh7qgL7IGLaBXBewPf46HfHmLCygl8PPjjU0WvsrHB75FHSH3sMXJ/+gmPm2++oH3VJuf4MTb9sJjOl/cjpGcEAEMmR/PTm7OIefN1bn7qeWxsm+WpDJhe5H79dCPJOz7B3lEx4tFocqdMpSguDs+77sT3/vuxa9Om0eNwu/YaAt97j5RJk0i+5x6CPv0UWz9Tz37o5f2oKC/n53f+H+5+sPOPwbh4ORB5Q0ijxyVavqLdu8n+4gv8p0zB1uvs2QJbIq01BTmlp4ra9KRc0pPyKM43jZM3GBQ+ga50jPTHP8gN/2B3vANcOFqUynNrn2N7xnaGhA9hat+puNu7c9ApnZUf72LggTHMntidAmM++7P3s+/EvlNfX+z5AmOFqXh2snWis1fnUwVwmHcYnbw64WDjUFvYFlFWWsLO1b+w5cclFORkY+foxA0PPlrthDuifjKO5LFxyULQZQyd/BBKKXKWLiXtuak49e5Nuw/+h41r7R2IF6K0vJT92fvZlbmLXZm72J21m8STifx151+42TfOldtmN6TBWFbOwhc34mgooft3D3PipsFsOnKQ4U88xyV9rjjn7eX//TepT0WTZgs7g4MpK8sjILQvw594uNppWhtTRXk526J6cLCnLyPn/1ln+4Nx6az8aBfdrm6Lp78zCTszOXogB12hcXK3J6S7D+17+BIY5o2d/fn3wG45toWHf3uYtq5t+fj6j/FxMl3e0BUVJNx2GxV5+XRcsRxlZ1fHls6d1pofZr1Myt7djHvzfdy8/+1l3vnrL6z+6B3C+l3DjQ89jjI0r96KSpt+/Ie/v34dg00pI8bdT/H0mZSfOEGbGTPwuKlpP7gIULBpM0cefBA7f3+C5n+GXet/78e45681/Pzem7j6XEKZ8Qb63RFGz4HtmjzGxiRDGppWaUoKiSPvpDwrC7fBg2n71pst+spBWWk5fyzcR8rebApzTbcEVAaFd4CLubB1wy/YHZ+2Ltja/Zu3tdbEHI7htU2voVA83/d5hnY4PT/sWHOEdYsP0K1/W/rfdfYVmLLyMg6dPMTerL3EZ8ef+l5QZrrtoI2yob1He3r792Zk6EhCvUMb+dmoXVlpCf/8+gubl5kK3XbhPeh94zC2LFtC2oF4IofeQr9RY62iw6OstIR1X31OwvZY2ve+lPCrB+If0sHSYVWrrLScr15cTmbCR3S79gYGT3yY7EXfcGzaNFyuuJzAd97B4NxwV4zLK8pJOJnArqxdpwrc/dn7KaswvbnzdvQm3Cecbr7dGNVlFJ6O59ar3GKHNOz+6yj52SWEHvwE27Au/JObSUBoVzpddvl5bc/1yivp8P13ODz2OF5b/2HL5cM5Gr+ZTybvoN+oMfS8bgiGRr5cX+ng1jU4F1XgWY/CPTeziN8X7KVVe3euuuMSbGwM9BzYjuKCMpL3ZJGwI5NDcens/TsNWzsDgWHetO/pS0h3X5zdz+0DTpe2vpR3rn2Hh397mAmrJvDJ4E/wdvRGGQz4TZ5MygMPkvPDD3jdccf5HnqNDm7ZQMK2WAaMue+0Yhegx6AbzDPDLMDJzZ0BYyY0uxfKfRsO8ffX/4dSxQwdNJSCJ5/GxtOT4IULceoWbpGYXPpcRtDHH3Pk/vtJuns0QfPnYx9ounzYtf+1VJSXs/J/b+Pqo1i7GIrySulzc4dm99wLyys/eZIj909EG414jhxJzjffkLt8hUXe6DUFrTW/zd/LoW3phF7WGv8Qd/yD3fANdMW2lk6JkyUneXXjq6xMXEmEfwQz+80kwDXgrHY9r21HQU4J21Yl4+LpQNSQkNOW29nYnerRrVShK0jNS2Vf9r5TBfBPh3/i2/3f0rdNX8Z0HcOVba9s0uEPZxa6vkFd6BB1N2Wlrdm4rIDWHe7Bq82fxC1fyrFDB7jpsWdw9fJusvjOdPzwQVbMm8OJoykEdO7K9pXL2bpiGX5BIXS9eiBhVw1o8g602qxfcoATR1Zi5+hEv7vuJmv+fNJfn4XrgAG0ffstDA7n38uvtSYlP4XdmbtNxW3WLvZk7aHIWASAi50L4T7h3N31brr5dKObbzfauLRpktePZtXDW1ps5MvnN+CSm0zPzXNIf3A8W9b8wqjpb9Dmkgt7J6pLSzk+ew6ZCxcR1/dhTpTuoKIsGb+QDgwa/yABnRv/RstLZ00k9LO/8F6+mFYdu9fYrtxYwfdztpJzvJCRUy89a/q/qu2OHsghYWcmiTsyyTtRDApahbjTvqcv7Xv44dXGud4n2sa0jUz6bRLB7sF8fP3HeDl6obUm6c67KDt+nI4rf7mgf5QzlRYX8dkTD+Lk4srdr7+NwcaG8vIKbKp8sFBrze+ff8i2n2O46q576HPL7Q22/8aWuj+NxS8/iy4/ycDOPbFf/A1OkZEEvv0Wtr5Nd//lmhT98w/JE+7D4OxM8PzPsA8OPrVs52+/sPrDd/AJuoz83CsJvyqAq0eFWvxDnw1BenibRkVpKUfGT6Bo+3aCPv0Ep969Sfrv3ZQkJtLhxx+xa+XfpPE0hS3LE9gck8AV/+lE7+uD6l4B2Jy2mefWPUdWURYP9XqIe7vdi42h5uJYV2h+/XwP+zcd59oxXQi74uzC+EwVpaUUbd1K/tq1FPy1ltK0o+QEe7PJK5sd/oWUdgnh5r5jGdZhWKONsTSWlZORlM22lT9zcPPPlBXnYmPfDoN9X2zs2qEMCi8/B1wdy0lNNaI1tA45RvLOb3FwduGmx54hMKxbo8RWk4qKcrYs+4713y7E3tkNzzbDycnwxcamBMVBSgt3UVqYCsqAZ+sutA27nKDwSFy8XHB2s8fJzQ4nV3ts7JoubybuzOTHt5ZSVrCUa+65j6DkNDLeett0dWX2/6HO8Y4/GYUZpwrb3Zm72Z21m5wS0+yc9gZ7uvh0OVXYhvuGE+Ie0uBvnuqbs+tV8CqlPIGPgW6ABu7VWtc41VZjJc/YFQls+jGByLjZBD50B9+t+YmOkZdx02PP/Nto52I2//kKm8tzcbd3xzPsZjwvGYyngyeeDp54OHjgZu9W4xOeu2IFSdNmEhv2MIX2ORjK11BYUMiVfon07QgMfBF6NHxPptaab0ZeRvvkUi7bsL3WInT9dwfZtjqZwYNz6ZT4PJxMAY/AWmPTWpOVWkDizgwSdmSSnpQHgLufE+3N437bFK3G8PsrtW5vw9ENPLLmEULcQ/j4+o/xdPSkYMMGksfdS6upU/Eeffe/jXcuht9q315t/vzyU2JjvufOV2bTNjSMgyt3sOaHNFr5K258diD2TqYLFLqighXvvMG+v//kuvsfocfAxr9rxIXKPpbNF09HYyzJoI9LK3zWr8Xzjjto/fzUc044ZyqvKCc1P5VDOYc4dPKQ6XvOIZJyk+jp15Mno56s96XK4n37SB53L8rWlqD5n502c9u6RQvY9MM3dLz0dlIPtqNDLz+uG9/1tMuw9XKB50lDb68lFLzWkrNroisqOBr9NLnLlxMwZw77o/xZfng5E72GcfKuCThfeintPvygRV01qByC1iW0mCuzolEFx7EPbIW67qVqz8+y8jLmbZvH/N3zCXIPYla/WYT7nnHVZ+diCn6eTk5WNgGtPVCDTOd6ubGC5e/uICU+hyEPdiek+9lvoEtTUilY+xf5a9dRsHEjurAQ7OxwjozEPiiI4t27KY6PB6NpzG+mGyQF2uPeO5Koa0fROuKK877sXZRXSmZKPplH8slMySMjOYfMpI2UFW0GXYCNXSC+bfvR2tMH14KjOB/bh/3+WHR6GgAVHcJJGxrN/t2FQBa6dDklBVn0/+84Iofe0iTnzcn0Y6x45w2Oxu/F2TOcct0fV9sigvPXUuHgSqlvOCXObcnLP0Fu9m5K8nahK/JBOWBjF4qNQ1eUjaln097RBic3e/OXnel70UGcjvyMQ0kqti6u2PYagU3nq7G1s8HWzoCtvQEbO4Ppd3sDtnaG2jscdi6mcOWbfH34SfJPfoebh2JIpyiyP/oIj5uH02bGDFQ9h4acLDnJ4j9fYEnKGo7amJ5rGxSdvDqfKmy7+XSjk1cn7AwNP9TxTA1d8H4OrNVaf6yUsgectdY5NbVvjORZXFDGF8+uwyNtJ1f4H2B3t0vYt+53xr35Pzz8TWMMj8V+zJwts1jpXPs7UBtlg4eDBx4OHng5eOHh4GEqiB1NRbF/ehn+c1awo9U9YFuKe8X7pOa7MqbDVnxdNQyb2+BF7/4T+8m87maI6s4VHyyusV3Srix+emcH4eFFDMgdD2VF/y60c6p3bPnZJST+k0nizkxS9mVTbqzAwZBPsH0clziuJcQxrsbtrU9dzyNrHqGjZ0c+uv4j3O3dSb5nLCWHD9Np9SoMTk6moiNm8nnHl5mcyIIpjxJ+9UCuHTWOja99y44TQTgVZ1Hs6I2bo5Fhz/XHs5ULAOXGMpbOnk7Sjm0Me2IKl1x27uO5m0phXj6fPhpNSUEq3UtdaXfgH1o/PxWvO+88p+0YK4yk5KWcVdgmnEygtOLf6YJbObeik2cnAlwDWJW0itySXG7udDOTek2ilUvtM7KBee4U9GcAACAASURBVA71ceOgvIKgzz7FMdRULFdUlLNs9nQSd2yl99BH2b0WAi7xZMhDPXBwqudoqQs8T2raXl5hOYVGO1o5FZzz9lpIwWvxnF2b9P/3JlkffojHo5P4tHc238R/A0CASwDvZg+h7I3/0fqVlxtlmJQlZCTn8f3sOHy9S+if8BCZm01vapVtBY5eFThGXYlj/1twDO+KQ4cOJOQnM2XtFPae2MttnW8jOioaZztTcfn/2TvvuKbOtw9fCQkJeyN7CSqigAP33tU6qnVW6+xu3aN1VKu2tdZRZ7XWqrV1b+vee6IogiDKHrI32TnvH1T6s+LG1TcXHz4J5OTkfpKc53zP/dyjKCebpJs3SD61k6TIa+SqSlf4OrvepIpdcdl3Xa3Usn1u6Upgt1G1cXCRUXLxEsWnT1F08hTquDgApK6umDVrinnTppjWq4+RuVmZ3XqlEmXkTRTXrpF68Tgl169hmV3alVEvFiGu7Il1rXqYBAViEhiIceXKD+RSKArVpNzKIyu5sEzkFueVdm0UBC1GokhUhefQaYuxNLaiSpEOh5hIRNrS+E6RsTHGvpWRV6mKrGpVJLY2pP84B31JCeYTvyUi142Yy0nolYfQKG/hV68xHT4dgbFJxcWg/i+CIHDj2GGO/LYMvQ4k8pZY21eheu7vmF64gkhTvqbSi0RkOzmTbGtJukSLHgG5xAxbq8pY2gSA3AGV3hilRoyySIlSKULg6ZwHYrEIo7/F7z0hbCQVI9XkYpQbRbHWhuziVNSK07TQJGAaqS11tEyb+kQ5MMmFyfxx8w+2RW9CodfQqERBU4WSGioVVQUjTN6ueG30JFSY4BWJRJbANcBHeML4hxcxeZ7dFMXVIyk0jF2G87zJrPt2MnU6dqXF+8PQ6DSsvbmWZZfnoRcEhuUXMCi/ELUI8sVG5Fo6k9d9KfmqfHKVueSp8krvq3LJV+WTp8ojT5lHniqvTCjI1AIfHHQCq5Ho9BkUKbfgKi+gn3s4Imt3GHWjQse3cu9MGo3+E/PJ43DvP6TcbYrzVGyYeREzK2PeNf0YSWEsJVop57I8UOn+9nZKTaFap3/a/goCwt+3//wtgEDZNnqdHkVMKMUaU0p01ghiB2rYamhu+StSG8dyx3o65TTDjw7H19qXFe1WIL1xm4T3+uM4bix2Q4fC/BqQn/TgIKwe/94JgsDGaRPITkninfbvcnFLLCk2dXCWZfHW+ObcmreG83nVEBnLaP9pLTxrli59apRKNs+YREZCLD2++gb3gMDHv/EvGbVKyW8jJlCcG0vlQguq58bjtnABpiEhD32ORq8hqSCpTNTG5sVyO/828fnxZUH/UCoWfKx9qGxVmcrWpb8+Vj6YG/+TaVugLmDF9RX8efNPJGIJAwMGMjhgcNnJ9KF2x8eTMGgwgkKB+8qVZfHFyuIi1k0ajaqkhEa9J3J2azq2rmZ0/iL4yWLFn+N78rD9KbLT2JAQhFpvxNDKl5CIhafa35sueF+XOfth5G7cxN2pU1G93ZwJ9WNJK7lL/2oDqG/cnK+jx4Fez9K/nJBExeK9ayfGbm4vxa4XRXG+ii2zSt/btpmjyT9dgLmLEgs3Bcpcaelvngyh1JGK3lhCrIOeJBdjajXpgV9QazIURSTHRJEcGU5uWmlVHmMjPa7yPNzM8onIq4QIgfd9riD++/wkCAL5EXfYsSIWjVJLnbB5mOSnIDI2xrRePcybNcWsSVOMvb2eyiOaEH+dI/t/JvPyWbySNVS9K0b2d01gsZkZ8po1MQkMRB4YSArunN5zF7VCi0gM1tZGWBkXY1acQkHqBRKVqagkYmyKFFS5m4OjlQ0mfwtbedUqyKpWxdjT8wHPoyY9nZThI1Bcu4bdsKEI3Ydyfnc88VcPolWcxsymEj0mTsHBw7O8ITwzxbl57Jgzl7u3ryKSuGHv2Y2Qt6pjObcjBVE6TB1UuDTIQyQW0KrE6CRO6Fr9iDYnF11ODtrcHHQ5uSizs0ksyiVR0JAtL/WA2hYpcMspxCm/CIleQECEzkQOZjJEpkaITCSILC0QGg1BZGmDYGENZpboxFJ0GgGtRodWrS+91ejR/c997Z2z6DQCap2OzNwD2KsLCY7IwK6mCMdNEY/9/G9k3WB1xGoOJRxCLBLTUaFhYOZdqmj+1XnzWeft56QiBW8w8AsQCQQBocAIQRCK/7Xdh8CHAB4eHnUSEhKe0fQHKc5XsXbCSezTQ3lrfHP2Hd1L+p0YhixcwbXCCL678B1x+XG0LC5hfE4ubtp/d4ISwbSHOjfKEAQBhVZRKoBVeeT90pL8mEDiJGMRik+h1IZR2+MWLc0ynmh/T4ogCMyc3IJ3t2bgs3cPMp8HMzv1eoFdC8JIj8un51ch2C53BQR2J1fjdqEdFlLVvZGC3b0Eor+/xCJR6b2/v9Si/729dz89Aig9yLJUZhgZB+Bg4U8Hm3nYfXu9XLtPJp9k5LGR+Nn4saLdCvI+G4syPJzKhw9hNMeN0pXUf/P4zyLixBH2L51PsNSGu9rG5FlXIaiOCY2HNkAkFiEIAvGzl3Ii3JJicxcadvGk1luVEYlEKAoL2DB1AkU5WfSaOotK3pUf+VovE61Gw+/jJ5KbehPX4krUk+bivnjRAx1sCtWFnEs9x6mUU9zIukF8QXxZKSEAV3PXMkF7T9x6W3ljJjX790s+lOTCZBZcWcD++P3Ym9jzWfBnvOP7ziNjA9XJySQOHIQuPx/3Fb9gWqsWANnJifw5aQx2rm407D2eQyujMbWS0WV4MFYO5ceXlzHNmmf9npRr49e2bEmoQYbKnB7uN3A3y3/q/f0HBO8rn7MfRtHJkyR98impAY6MfisDD2tvptadTuo2SLqZS/W37fhBNQHSs5i7Uo+5fwAev695YyuwaDU6dsy7SnZKEW1rZqGaOwVzFyVujXMQ/c+hJuhFpPc+zMZd36OOuIVnkS2CyogcmZQSWakgkgrgYG6Fq48vniENcT3VB4mk9NiJKrBnT4o/HZ2icCsppNj9c4pOnUKTlESJiSOhdcdhbCymUxcz7JrVK12Fe07yVflsubWF9ZF/IknOoGGuPa2LPHBOKKQoNpXoyj3JdKyNtSKJqrnHkd8OBUFDkq0lsZVsUUqNcDAxp3ateng3boa8SpX7WqMLgkB6STpx+XHEF8QTnx9PfEE8yYXJ1HKsxYf+g5EuWkveho2YNWqIy9y53M2A42sPk357MyI01O08lKb9Oj13iINOq+f0xkNc2fMbep0Cy0otad6/D+62CtJGj0J16xZ21QtxqFHI/dGSj5938u6mEnn4AJFnT5CfnYVEIsFTl463NhcHVTF6hREahRHaEiO0CjFl5/V7r2BqipGTE+JKDuDgiMjOFsHGBqytECwswMIc7dahaAURtwvtiC20o0l0El6Vc3CoWYTom/Lt0wt6TiWfYlXEKkLTQ7GQWtCzak/6VetHpR+rUpHz9vNSkYK3LnAeaCwIwgWRSLQAKBAEYcrDnlPR3oLDPx7hVoye9l43MerYmK3fT6VOn97ssb3O/vj9uJm78VX9r2i2+dMK9xaRn0RUagOOCCNRF/1BkTSHRnWSaT0i/PkH9jcxuTEc+aArjZNMqHH+crkH57345ZYDqlG9sQvMr8Ht5GJ2JgfQxCGe+vZ/j/s5xwpwJtOT81keyE1qYWTSmGb9A/FvVH4W5YmkE4w8PpJqNtVY7DaKjD4Dsf/icxzUPz/TZ1GSkcFvoz5CVqRCbDMYtYktrQb6U7XBg4kX6WvXc2JHKpkOtfALtqHVkEAkxkYUZmexfso4dFoNfabPxsbp8UkbLxqdVsuGr6dy98417DS+tPYQ4frdt4hNTREEgVu5tzidcppTKacIywhDJ+iwMLagtmNtfK19S7211j54W3o/1hv7NFzLvMacS3MIywzD19qXMXXH0MT14bUtNWlpJA4ajDYzE/fly8o80zGXzrFrzrcENG9DUIdB7FlyHSMjMZ2HB2Hv9oiaihXo4dVpNWz/tAOJ+XI6u93EzyL7mfb3HxC8r3zOLg9lZCSx/fqRbKNn8nvQJ3gQgyt/wOFlUWQmFeHgbk5GYiGNB3kyM30Criej+fgvLY4TJmA3eNALte1FcK8iQ/SFuzQNyEe6ZCLmnuBaL5X/va7MV8s4YuTNCaU1tlkizEtKvZkyM3NcPL1xNLXArkSJSWwiqps30RcUlD5RDDJLDXIbNRIzHTv1/uh0YppFJyE2McGsfv3SUIUmTcjRWbFz/lVsnMzoNroWxvKKK9Ck0Ws4GH+QNRFruJlzk+qF9Wl2uycitZRAl1w8s06j1+tJtJARkZZIcXERLlX9adTzPTxqBKHQKsoEbVxBXJmwTShIKMvsh9Lsfi9LLxxNHTmbehatXktX364MinNH9cMiJPb2uC5aiLx6dSJPx3Bk5Tw0imQs7OvT/pOP8Kzx9EmQWo2OG8cTOLNpNcqCK0jkDjTr9znBbWtTsGcPaVOnIpbJcGmQj7lV8oM7eIp5RxAEUm9FEXniCNHH96LSiTGXqHAxKUAriNHqjdCI5KgtK6NRKNCqlGg0GnQ6LfqnKD7gk5FLU/t47AOKyrVPpVPx152/WBO5hrj8OJzMnBjgP4AeVXr841Sp6JW556QiBa8TcF4QBK+//24KfCkIwkPrxlTk5JkTlcSGeTdxU0XRafkHrJ08htyibDY2ikMr1jO05lCG1BhSWiz7BcUDolFwI7YhxyW9UBVvI8K7mGb9BzIwYGCFBMcvCVtCjY8W41a3OZWXLHvg8dSYPHbMu4Jv3Uq0HVLaCU596U9WLViDXKyhv/dVjERChY1VEGB/WhUi8yvh6NaaguIgqtSrRPN+VcudKI8nHWfU8VFUt63Ot3stUV24jO/CzzA69uUTfxaCIFB46BAHF88lQS7FxLIfJtZuvPVZEGauYs6knuF40nEupF2gpn1NRtQeQWXryuQfPMTZnw4Q69EReycZnUbUwdxGTnZKEhumTkBmYkKf6T++0pI1er2Ord98Q2LUFSxENena3Buzof24kHaBUymnOJVyioySDACq2VajqWtTmrg2IdAhEIn4xVcOFASBw4mHmR86n6TCJBq5NGJ0ndEPTWzTZGSQOGQImuQUbN9/H+tevTB2c+XMpj85v3U9rQZ/hGdQK3YvDEOt0NLps0Bc/B5SkqeCjllBr2fPojlEnz1JO7c4alr8z8nn/1kM76ues8sjJ+EWcb16UyIoWf65D+Pf+h4vkR+7F12jMEdJ+w9q4F7Nhh3zr5KdXES74VWZcWciLRdfoHaCGN/tO5D5+r4w+14EVw4kcG77HQK9irFfPb605NMHTRHvH82tbDPuFNmSWGJFkaY050QjA3f/mvgHN8TNvwYOHl4PeLYFQUCTnIwyIhLl8S0oLxxDmS1GpzYiy1nORUdXWtXyI2j0D4j/lfwaH57F3p/Dca9mQ8fPAu+rdlMRqBQadq49T+YVDdmmqZyssp4GNWrRRh1M9ObdFOVkY+7thqSJL2l2CuILEogriCub+wDEIjEuZi54WXnhZemFt5V32a29iX3Z+TazJJOVN1ayOXozekHPYKOmtPvlGuTl4zRtGtbvdEOj1rBr7mLiw44gMnLBM/g9mvSqRSUvy4cNoQyNWkfEyRQu/XWRwvSdCPpc/Oq3563PPsRIEEj/9jvyNm/GpE4dXOfOQZpxEnYP565eTbjMGKkAxkZS5I1GIKv6FnIjOTIjGXJJ6a3MSIZULH2oftBeWc+d9dOJzLEmV22CVKxDKgaJUxWkdp5IZXIkMhlSY9n9tzI5EokEsVIJRSWIiwohLx9iwyDmEpTocXQvxMav5IF5MU+Zx8bojayLWkeOMgd/W38GBQyirVfbBxPPKlprPScVnbR2ChgmCEK0SCSaBpgJgjDuYdtX1OQp6PXs+uhXUkWe9P7Mh1OxR7m9YTfHamXiXa8eE0Im4Gbxr/iuF5jxHRbVmuMSL3SaGP5qnEWTOq35usHXSI2ePQtREATeX9ORibPiqTRxIrbvD7jvcWWRho3fXkQsEdN7YkhZZYKjq5dzdd9u+gak4aK/U+Fj1Vm4sS2jAclJmdRo9TExoTKsHE1pNywAB/cHPXZHE48y5vgYWqh9+HDuTew++ADHNq5P9FmoExK4O/NbkkMvcs7PDSNZHez92iG0T+RU3jEupV9Cq9diLbMmxCmEc6nnKNGW0M23G58GfYpFdApXJizkhlcvjC1MeevTYJx9rUm7Hc3m6ZOwruREr2mzHtt2+kUg6PXs/OZr7kSFIZfUwamFmH0eSYRmhKLVazGTmtHIpRFNXZvS2LUxjqavrhSTRqdhQ/QGll1bRqG6kG6+3fi81ufl2qTNyeHu1KkUHjkKgoB5s2ZY9e7FkUuniL92hXcnz8TayY/dC8MoyFLSblgAPsEO5b/wcx6zgiBwdNUywg7soWm/QdTz1huqNLyiObs8TkXtR/3hOKzytYTN6MWAThMpuqth98IwtBo9nT4NxNm3tNB8SYGarbMvo1Hp6DIukPnnJtJl2lH0zvbU33HkARH3uhJ3PYu9P1/H00GFz6bRWDRvjuuihYiNjbn8y1ROHAnFWKIhxU7NHTsVISFtGd72S+TSpww1uL4J4fA36LNTEdu5sjauFlojUwbNXVpu/fjIM6kcWxtF1QZOtB7oX2HVDFJj8jiyJpLCbCW12nng1FzK+pg/ORC+i7eO2lJkpuNitRzS7EpLY1oYW+Bt6f2AsHW3dH+qTm93i+/ya/ivbI3ZimWxwPQD1jjeTMemXz8qfTkBkbExkSePc3D5IvR6CVLTjvjWq039Lj7YOj8YAqZWagk/nkzY4QSKMs+iVZ3DxMKKTl+MwTMwGFVcHCkjR6GKjsbugw9wGDEcFVqOJh5lZ9hyzhXcQXjC91QsEpeJ33+LYZmRDFlJLvLs21gri3GXmOER0BP3Gr1xt3B/qvC1Mh4yzyYVJrE2ci07bu9AoVXQxLUJgwIGUc+p3qO/HxWttZ6Diha8wZSWuDEGYoHBgiDkPmz7ipo8Yxf/zr5wF3xcCznW6DQWv0egNjei9ZfjaOHR4rn3/7QIWi3nPpzEueJbGMmd+bPpVQLcqjK/xXxs5M9WVDomN4a533djxC493tu2Iq9e/Z/XEwT2/hxOYkQ2PcbXwdGz9Mo07XY06yaPJbhdR1oP+aRCxlYeqpJiNkydQEFmOq2GTObSnkKUxVqa9PQloJnrAwfDkYQjjD0xlkn7TKkRVYLv4cNI7Oweun+9Ukn2LyvI/vVXdMZyjvpXRa1RU+DXjF3eK9EaqfG28qaFewtauLUgyCEII7ERucpcfrn+CxuiN2AkMqK/f38GmLUkceS3XHXqgcrUgWZ9qxLQ1JX461fZPusbnP2q0mPSdKTGL69tZrG6mO3jh5Oelo7EuC6n/E8T7ZqOr7UvTd2a0tS1KcGOwS+lbMvTkK/K55frv7Auah1SsZRBAYMYFDCo3FAKTVoaeZs3k7d5C9rMTARXZ854OKARi+n/wwKkcmv+WnydzIQCWvT/Oxyngjm7eR3ntqyjbufuNH9IwufT8B8RvK9kzv5f8lX5zDk3i5rf78Q/WUA8byoB7fuQejuPvUuvI5GK6Tw8GDvX+y9Ec9KK2fZjKKZWMrqNCWbjqi9osOQU17r60+P7ja/d8fJvslOK2Do7FAtjJTX3jMOqaaMysXvz9HH2LpqDtLorq70vYi23YWaTmTRyqZiqMjEXz7Jr7nd0+HQUAc1bl7vNvVrAtTt40rDb8+U4aDU6LuyKI+xwIpZ2ctoMql528QKwc+Esbp8/C0Pq4eXhXyZwbeW2FVo6LK0ojRXhK9gZvY2+x3R0uqBFElwTr4WLkTo6kpWUwM6535F3Nw2ZeVNEktpUa+RCvbe9sbCVoyzWEH48mWtHklAUZiEWDqMsTKRqo2a0GfopcnNz8vfs4e6UrxEZG+P8wyziqtuw4/YO9sftp1BTiLOZM10qd6Gle0sAlDolKp0KlVaFSqcq/VurKvu/Uvv34/d+H/JYjjKHHGXOfeO1ldviYeGBu4U77pbuuFu4l/1tLbN+ovc2PDOc1RGrOZx4GLFITCfvTgwMGIifjV+FfS4viwoVvE9LRUyeihsR7J62n2y7ANY3+AHvOD2B0Rb0mPYtXv5BFWTp06MrLGTH4I+JN8rHxLEzfwavw9LalMWtF1PZ+uknjyVhS9D9sJR2MSZUvXAB0f9cld9rE9mkp19Z+1adVsufX41EUVTIoLk/I6vA9n/lUZidxbrJYwDo/uX3nN+VQWJEDpVrO9JyQLUHyk8dSjjE3G1jmPuLBqv+7+E2aXL5+z1+nLszZ6JNTuFOg2qEy6pjnHeNJA838lopaenRkuZuzfGy8nqobcmFySwJW8Ke2D1YGFvwmVtfas89yVVZc3Js/KnR3JUmvfy4ffEMfy2YTeU69egyeuIL7ZwXnx/PqZRTnIk/TtXfb6EXrDCS1SGnuYiaTQNo6toUJzOnx+/oNSCpIImfrvzEwYSDOJg48Hmtz+lauWu5iW2CRkPh0WPkbdxAeuhlzvq5YSE3ocfHIzGp04D9KyJIisyh4TuVqdXOo8JOdmEH9nDkt58JaN6G9p+MqJD9/hcE79NS0YL3WOIxpp/7hr6bM2l6Q4/jdzOx696DuGuZHPg1AgtbOZ2HB2FpV75HMyU6l10Lw3Dxs6bTZ4Gc/rgHduei2TK2LhMGLK/QOPaKRFGoZvOsy2gLi6l9fAq2DYNwW7QIsbEx587u4cyiZWTaqthfJ42W3q2Z1nDaU7dRfRSCIPDHlyNRK0oYPH9ZuXOdIAgcXxdN5KlUmvauQmDLZ6uCkZlYyOHVkeSkFhPQ1IVGPXzvC3lLvRXF+iljqdetJ037DnzmMT0NKUUprLi+gvSdW/lorxbBzASn+XNxbtgSVUkJB5ct4NaFM9i4BqJUNkMkluEdaE9iZA5qhRbbSglkxu3FSGJE66Gf4N+kBXqVivTvvydvw0YkQTW48EljNucdIzY/FrmRnDaebejm240Qp5AX1o2uSF1EUmESiYWJJBUmlf0mFiSSXpJ+37YWUgvcLNzwsPQoE8JuFm54WHhgb2LPyeSTrI5YzZWMK1hILehVtRf9/Pu90tXF5+WNFrz6khIu9BzKFfcPuFZpH6Z1i/DenIFPcF06j/6qAi19NhRxcfw2bgQqqQVW3oPYVO0nCiQ5/Nj8x0cm/PwbQRDourMr4+Yn41ktBI/ly8sey0wsZMvsy3j429Lx08CyE/nFnVs4tW41XcZOwi/k2dopPy0Z8bFsnDYBK4dK9Jr6A5Fnsji/MxYLWxntP6hR5nm+x4H4A8SOH02TSAHP/XuxdPMqeyz9TjgJ06diceEmKfYi1rV0oWruQCRZ25E5V+K9777DxuTpvOVROVH8dOUnzqScwcvIkam7TEkrqkGiR1tc/Kxo/0FNbp07VCqMWrSh/ccVI4zuEZcfx/74/RyIO8Cd/DtYlAh8eMCObJkFRrJgmg4YTJ22r0+1iKclLCOMHy//yPXM6/jZ+DG2zlgauT7cI6WKi+PGL8s4HhuBa04hdY1MsezVm6uqIG5fyyW4jTuNuvsiEj/fZxB15gR7Fs2p8AsZg+B9dvKUeXx/8Xv2xu3lk0s2tDycif3wL3D49FMiz6Ry/I8oHDwsePuLIEzMHx2eEHUujSNrbuLfyJlmXZyJ7NiOu+Ii/hgXxML2P1eoUKwIdFo9uxaEcfdOLrUv/YhzbU+cFszj+N0z7DizFo892RSZ6hD6BNMrqB+B9oEVOg/d407oBXbMnkH7j0dQo2XbcrfR6/TsW36D+PAsOnxQg8q1n1zs6HV6rhxM5NJfccjNpbQa4I9njftX8gS9nvVTxlGQlcGQn5a/sJq4DyOpIImNe2dTZ8Fh7PMhakAjWo2cja3cltA9Ozj55yos7Z1wrtaXpCgRblXllOTuJ+nGZdwDAunw6Ugs7R1Rx8eTNGIk6uhoQtt4MK9OGhqxQLBDMN18u9HOqx0Wxo9Iyn0JqHQqUgpTysRwYkEiSUVJJBUkkVqUilb4p8KPkcgInaDD2cyZAdUH0N2v+7OFR7xmvLGCN704nXMj3keR/zYZDt7UHGWO9lg4EcePMGje0tci4x4gastG9mxei1TeECuPZhyouYII5TXGh4ynX7V+TzSRxeTGMGj9O/y6UIfDmNHYf/ABUBpHtOnbS2g1enpPDik7MeTdTWPNuM/xCqpN17GTXuj4/k1paMA03KrXpPuX08hIKObgrzcoKVDTqIcvgS3d7hvzofPrcBoyg/CGlQiavYTzCadQ/bGZRgdTEURwoJUVRg0HYH7WH23RfrSqGAbNXfJcn++FtAvMC51HdEYEXx40o1JqVaKqD8DUxpSOnwQSc2EX57asJ6RLD5q9N/i53o+kwiQOxB9gf9x+onOjESGidqXadNUHIl28j2gLGUbGNaj11kCa93vxbalfNIIgcDDhIPND55NSlEJj18aMrjMaH6vSEnpltZ3//gG4sHEdl3ZupSbGuF+7CTI5sc2+IF7jhU+IHQ37ed4neu/FsT0J8deusP2H6bhUqUb3id9UaKiKQfA+G4cTDjPj/AwKVAV8ndOEassOY9WjO04zZnD1YCLnd8TiXt2WDh/WeOIqARd2x3J5Tzz1u/pQzSyRpA8/4q8GEk5182Z5m+U4mzs/l80VhSAIHP8jisgzaVSPXIW7nxFHPqrDpvgdKLJyePu8C3K5Gb2++R435xd78SsIAn9OHI2yqIDB85dj9JDuWRq1jl0/XSUzsYguI4Jx8Xv8BUReegmHV0eSHleAbx1Hmvetitz8wRCTyJNH2bdk3iNDK14G8ck3iBn1OW7h6ZwKkqAcOZD3aw2jKDaZvxb8gEpRQt23u3P98D5UxUU06fM+dTp1A5GIyE2/ovt+ESqRlsWdRCQHOdGlche6Vu76NgwN+wAAIABJREFUyJXH1wmtXktacVqpR7ggiZSiFPzt/Gnr2falJES/LN44wavRa/gz8k8ubVhI/0MuXKk1mpBunvhUN+L38cOp1eFtWg76sMJtfR62jPmMpMQE5LaDMXd05mr9HRzK3kevKr34sv6Xj401WxK2hKubljFmmxbP9eswrVWrNGN+VSQxl9LpNrpWWXa7IAhs/e5r0mKiGDTvZyxsH2wV+aK5cfwwB37+qWz5WFWi5ciam8Rfz8I7yJ5W7/sjN/tnzBfGDMV031mWvyXmnXN6XHIgs74vDhPGU5LozvmdsVjaZJJxZy0N3+1Ho579nttGvaDnYMJBFl1eQLPdCbSIcCO8zhfojC1oOaAaieHbuXZwD836DyGkc/en2vfd4rtlIvdGdmnplSCHIDp4tqeVwgvJiYuE7t5OZCVrjIyr4V23H50/D350u8c3DLVOzfqo9Sy/vpxCdeGjNxag5RUH3DNMuOaXTsjtEppECKQ7dyDOuzPF4htsDVpFibzUAyEVS+nu151hNYc9MuwjLSaaTTMmYuPkQu9ps5CZVqyHwiB4n44cZQ7fXfiOA/EH8Lf1Z7q0B8LYmZjVq4fbzz9zdmcC144m4RdSidYD/TGSPPnxcG8+vHUxnbZDq2Ox+2dyN29m1kBzUipbsLzNcnxtXn31hnvhZ54J+zE2Pc/kTgUojHQ0s22A/34FIqWOPtNnY+fq/lLsib16ie2zvnlsq3VlkYZtc0IpKVDzztja2LmUn9grCAI3TqRwdtttjCRimvetil9I+V0a1UoFq0Z+hLmtHf1mzn3lNZQFvZ6YuTPRrVzPHScRS3qZ8VaDAfR07cKJpUtJiYrE3sOLjp+PQeRowd6onegW/kqDc3ncchUT9nlL2tTrSwPnBo+sU27g1fFGCd5rmdeYemYquUm3mb9KxM3g8ajtvek/syG7504n9VYUQxeuwMTi8eVEXib5GXf5bfiH2BbqUHqOwszOjMw2ofwWv5z6zvWZ23wuVjKrcp97L5yh575CQi7kUfXiBUTGxtw8m8bR329Sr7M3IZ28y7aPPHWMfYvn0nrIJwS3f2h1oRfOvQShhu/2pVHP9xAEgWtHkji3/Q5mVjLaDQvAyad0zJr0DGLatkGk1iB2d8X162nIGzTk+B/RRF+4S+XatqRELAUEBv64BEkFZmBr9Bq23tpK9LK5dDki4mrdT1DIPKnV1p3sxK3cOn/6ibwPmSWZHEw4yIH4A1xNv4pcLaaG3I8GFrWplmOO5sYt8mPvUKJSojSWUGAiQyL3w96rJ+9OCEFm+non2Dwreco8dt7ZSYm2BNHfP1DazESE6B9vv0pL4aoTCCUqLIe2wlgmxf7UTTgrJs6uI5ZFCbjanyGnQw1uWBWw885OAN7xfYdhNYfhYn6/xz87OZENUycgNzOnz/TZmFk/W7LoozAI3idDEAQOJBzgu/PfUagp5JOgT+gnbUxK/4FI3dxwW/M7x7cmEXMpncBWbjR51++Zwlh0Gj07F1wlPb6Azh/5oxw1AI1ew+jBkC9Rs6T1Emo51nrq/VYU0ddSOPxzFPZZ19GW/Movfa3oXK073T27cnb+UrKTE+k5ZSYuVV7eSo8gCKyfPJaivByGLvgFI8nD56GCbAVbZ4ciFovoPq4OFrb3r7IU5So5ujaKpMgcPKrb0nKAP+Y2D19ROb1hLRe2b6TvjB9f6pgfR+HRYySPG4tCpGV2Zy0Jfpa8V7UfTbUBZNhp2JXwF9E3TjJimxqfdMjs1ojAybOxNn944rWB14M3SvCGZYQx8cSXfL9FRlGGGWFVh9G8bxXMbTLZ+u2UZ/LGvSzObPyD89s2UCNNT0LQWEys5Jh2z2JmxFTczN1Y1GpRucsfMbkxdN/VnTUb7bFz9MJz9Spy7xaz6btLVPKypMvIWoj/PjmUFOSzevQnWDu70Peb2a/0ilkQBA78vICIE4fvixFLjyvgwK83KM5V0aBbZYLbuCMSi8jftQttZhY2/d9DoYR9y8JJjyugXmdvtMoLnN30Bz0mTscrqPYLsbdEU8LelZPwXXyIG/69yLdrQqUqcvSFe0iOCqfr2Mn41KqLoqiQopxsinNzyMhM5nrcZeJToynKzcZEaYSlRoZMKQL9g8eLTCRGbmGNib0zhXlWSEwa0fOr+o/vMvb/hJzUFNZNGo2VoxN9pv+AVCZHEARubrvIiUOFmBTfJfjaYqwDfTH6YihrhDNsi9mGgEDXyl0ZVnMYbhZuFGRlsP7r8Qg6HX2m/4h1pReT/GcQvI8nR5nDzPMzOZRwiAC7AGY0noGXyoL43n0AcP19HUd2ZpAYmUODbj7Ubu/5XPGqyiINW2ZfRlWspVNHGbmfD0LavROj6kaRVpzGnOZzaOHe4pn3/yzE5Maw+eJOzLf7Y1mUjXXRL/DtcDpU7YIUCTtmTychPIyuYydTuU69l2obQHxYKFu/n0qbYZ8S1LbjI7fNSi5k+5wrmNvK6T62NjJTKYIgEHMpnZMbbqHT6mn8rh8BTV0e+TnmZ6SzavTH+NVrRKfhD62C98pQxcWR/MUXqGPjON/Nj3lVbpd1G217x5xBu4uRSmW4zZqNRauWr9haA0/KGyV4ATKWLyNr/kLCOv+E1ticvlNDWD9pFCqFgsHzfq5Q719FolEpWT3qE8jKomaamPBaI5FZyPAdIOXLsDFoBS3zWsyjgXOD+563JGwJay8uZ9V8LfaffYbNRx+zZVYoxfkq+kyuh5n1P1fQ+5fO5+bp4wyYtQB7D6+XPMIH0Wm1bP/hG5IirvPOhKllYlVVouHo2ihir2biWdOONgOrl8V3ZSYVsnfpdZTFGtoMqo6dq541Yz7Fp049Oo/68oXbnHbqMBkjRhNfqT6Jnr3AogRz4TC5KQmIRGL0Ou0Dz9EYg7GlObZyW+RFOoSsQlCKESTWiOy80dt4oRbboSjSc+8wEktEdB1R64ni4f4/EXvlEttnT6dao2Z0/GJs2UkzKSqHfUuvI0VNcMRS5Km3sO7ZE+GjfqxK2sLWmK0IgkAX5w64/pWBuqCI3tNm4eDp/ZhXfHYMgvfxZCmy6L27N/38+5U24ClRkvBefzTJyVT6dQ2HDigqvBRdXkYJW38IRWYmoZn8LMW//4r10nmMLvmdmzk3mdpwKu/4vVMhr/UwNHoNRxKPsCFqA+EpkfS/PBozlQl1dLup9csSxDIZgiCwf+l8Ik8epd1Hw6nZqt0LtelhCILAhq/HU5CdydAFK5BIH73alByVw+5F13DysaLtkABOb47hzpUMnHwsaT2oOtaOj0882z1/FrFXLzFk/nIs7F5+2N2ToCsqJm3SJAoPHIDWjTnSqzKN9iZisvM48qBA3ObNe6Ddu4HXmzdK8CrCw4nv24/CVgO4pK5L60H+aJURHPj5JzoNH0e1xs0r3MaKJObCWXbN+44aBSrsBEeuVvsAY1MpDT5y4cuw0cTlxzGx/kR6VS0tynwvnCEk1oh3f4nCY/VqLsdZc+NECp0+C8Sr5j8TRUJ4GFtmTqb+O71p0mfAw0x46ahKStg4dTz5men0nvYDjl5/Jy/9Het1eksMphbGtB0agKJQzeFVkcjNpHT8JBB7d3O2//ANyTcjGDz/5cUjK2/dIn7YUDIEO0JrDkYQtIgcTpMvzuOusgBEZlhKXHAzqYqtwgltroBCLUH/r+B+I4kIc1s5FrZyzG1kmNuU3lrYyrF1MX/kct//Z85v28iZjWtpPmAodd/+R5hkJBTw1+JrqBVa5CgQ56ZjLKgw9/NA4utMRP4VSkL3IleoUDetzdvN++Pp6IbcTIrMRPLc1R7+jUHwPhlKrRK5RI6g0ZD08ScUnz+PzdxlHDknoTBHSbuhj2g2Ug4apRKp/NFJi2m389j5UxiOnubUPD4DivJx2rqBsVencjb1LCNqj2BojaEVWv1AoVWQUpjCgYQDbLm1hSxFFm6mbrx7fhCqQhsa6o8RtGQaYlnpcX9q3Wou7txC4179adCjT4XZ8SwkXA9jy7eTaTXkY2q1f/ux28dcSufgyojS1UUR1OvsTa22Hk+Uh5AUGc6mb76iUc/3aPhu34ow/4UhCAI5K1eSMW8+IokEQa3GdtAgHEePQvSaOtcMPJwnnbNfizQ9QaNFHhjEFasm2BiJ8Q6yZvWo33HyrULVRs1etXmPxbdeQzxqBhMTE41T5C1C7m7isksfzi9LZcnnK5h582tmnJ9BXH4cY+qOIS4/rvR+Vn2QSkkTuXLjxC2C27jfJ3Y1ahWHVyzBxtmFBt17v8IRPojM1JR3vprGuslj2T5rGn1nzsXS3gGRSETNFm44+Vixf8UNdsy7iqAXqORtyVsf18TMSkbMxbPEXb1Mi/eHvdTkO3mVKlTeuAnjDz/E/MJczjT+BHFGZxyAe6dlEQIyTQGUJGOmzsPJzhSbqp7Y1auBlbsd5rYy5GYPbwlp4OHUf6cXGXF3OPnHKhw8vPEMDAbA0dOS7uPqcON4CooiNSUZVhTdSSYjvhBNqg5xSSimWjVSs66YhlfmdHgyp/m7dbAIZKYS5KZS5OZSZKZS5Oalf1vYyQlu4/EKR/zfRi4pDU1JmzaN4jNnMPnqO/YfF6FRqeky/Mmy/gW9ntirl7m0ayspURHYOLvgHhCIR40g3KvXxNTq/n04+1rTeqA/B1dGYNJqPJ6/fUTB7Hks/n4xU85OYcGVBWQpshgfMv6Ja6IWqgtJLUot/S0uvU0rTiv7X66qtF+HCBFNXJvQp1ofVGuKuFUsJ0h//j6xe2XfLi7u3EJQ247Ufw3mbI+aQbhWC+Di9k3UbNnusSulfiGVUJVoiLmcQZNefuV21SwPvV7HsTUrsLB3oG7nF+tlrwhEIhF2w4Yhr16dzEWLsRs2FIvWr66ahIGXw2vh4QW4eTaVo79H0eGjGmTGHePMxrX0njYLN/8aFW7fiyA7OYnfx39OFT9/fDbtQt+uJ+clrZEYG9F5RBC/Ji1lbeRaGrs2xtvSm3VR69jyVxU0EhvOOvXHytGU7mNr35fBfGr9Gi7u2EzPKd/hUSPwFY7u4WQmxrPh6/FY2jvQZ/rs+zLm1Qotp7fEIDYS06SnLxKpEWqlgtWjP0VuZkb/WQteaBOIh6HLzyf5s88pvHKN3HeHI0dAFHoao+RbyHTFmDdqgGX7Dli0boWRtSE0oSJRK0pYN3ksxXm59P9+PlaO5cfhCoJA3s6d7Fu5lDRTY+o6VabG51+RqVWxJ3I/FxNCkahlBJgHUtMiGJnGBGWJFmWRBlWJBmWRBjMbOf2m1n9qGw0e3icn6+efyVywEAaN4UxGFYykYrqU0z3t3+i0Gm6ePsHl3dvITk7Ewt6Bao2bk52UQPLNG6gVCgDs3T1xDwjEvUYg7v41kZuX7vfyvngu7IylukMmTpun4bpwAeZt2zDn8hzWRq7lLa+3mNlkJlKxlFxVLmlFaaQUpdwnZFOLU0krSqNQc3+1EZmRDGczZ1zNXXE2d8bFzAUXcxeCHIJws3Dj0sI9XIw0wUd3k/aLhiL+2ysdfe4Ufy2YjV9IQ94eNQHxa5LRn3jjOptnTKTloA+p/VaXF/Ia14/s59Avi+k0YjzV3gAnlYH/Fm9USINOo+fPqeeRm0vp+Ellfhv5EZ41g196rdnn5fjalYTu2cHb9Zqj/2UlkiGjOJnhj5GRiG6ja3M4fy/fnv8WraCloW1tRkwMI7z1TAqxpNekevclOWUmxPHHVyPxb9qSDp+MfIWjejwJ4WFs+34qbv416P7VtEdmBJ/44zcu795Gn+k/4lr11WXw6lUqUsdPKI3jkkgwa9TQIHJfErl3U/lz4igs7R3pO/3HcpexBUHgyMqlXDu0j9rOnjgdPI6RlRWOY0Zj1b072aoc1kSsYWP0RpRaJR28O/Bx4Mf4WPuU7UOvF8oSP58Gg+B9MvJ37SJ1/ARKOg7lkqYu5jYyugwPxtL+4cmaqpISwo/sJ3TvTopysnHw9CakSw+qNGhSVi9Wr9ORHnubxIjrJEVcJyUqEq1aBSIRjl4+eNQIwq16TWLDJNy6mEtgwWEqxR3HZ/cujOzsWB2xmnmh87CT21GiLUGhVdxng5nUDBdzF1zMXMoVto9qexu9cjdHLsixJ53uP/VAYlo61sQb19n2/dc4+Vbl3UkzXruck03ffEVOajJDF/1a4e3VVSXFrBzxIbYurvSe9oNh9cvAS+eNErwRp1I4/mc0nYcHcevsBsKPHmTgnKXYurxZgeOqkhJWjfoICzsHWiCnYNt2TL/+kSNXrRCJRXQbVYsYIYIpZ6YwUdKZwuVJJHi2p92wAPzq/lPTUK/XsWHKePIy7jJ43s+vXTm28og4cYT9S+dTvVkrOnw6qtxJLysxnrVfjqB6s9a0/3j4K7DyfgSdDsW168h8vA0i9yUTFxbKtlnTqNqgCZ1GjH/g+3Jm0x+c37qBkK7v0qzfIJRRUdydPgPFlSuYBAVR6espmAQEkK3IZk3kGjZEbUCpVdLeqz0fBX70XLVZDYL38Shv3SKux7tkh/TkurwxDu4WvP15ECYW5Qu9otwcru7bxbVD+1CVFONRI5CQzj3wDKr9WIGk1Wi4ezuapIhwEiOukXYrCp1Wi0gsxtjUFZ3OGb+kq/hXc8Zr8RJEIhH74/ZzKOEQlcwqlQpb87+FrZkzlsaWzyTKkv7cyb6jImQSHb1mtcXEptTbXNqJ8kss7Ozp883sMi/060Ry5A02fvMlLd4fVtpYoQK55+jp/918Kvm8+prIBv7/8UYJXq1GR9y1LGwqKfl9/BcEte1I6yEfV7hdL4N7HWbaffAZlmvWowgLw2L+bxw4oABRqei1dTYj/PtVnEzwxL++A60G17xvH1f37+boquV0/GIs/k1avJqBPAPntq7n7KY/adCjD4179b/vMUEQ2DjtS7JTkhgyf9kbIeINvFgu7NjM6fVraPbeYEK69Cj7/5V9uzi2+hdqtGxHu4++KBMngiCQv3MnGT/OQZebi02fPjiMGI6RlRW5ylx+j/yddTfXUaItoa1nWz4O+pgqNlWe2i6D4H08er2esz/s5FqCFW7VbHjr45rldk/LSU3m8u5tRJ48il6nx69BY0I6d8epst8z26pRq0iNvklSRDgJ18O4eycG0CPSC1Sq5Ix381Z4BATi5Ff1sZUJHocgCKjv3CHn4DEOnDNBbWrHu5MbYOteWv85PyOd9V+PQyQW02/GnNe2MgHA5hmTyEpKYNjCXx+bHPik5KSmsGbsZ1Rv1uq1cGIY+P/JGyV473Evc3/owhWYWpbfsOF1514pmNy7qQyc9gNpQ4aiLyzCaula9q5LRRAE2g0NYN/8cxjrSui3tDtS439ivQqyMlk95lNcq/rT/atv3qjlIUEQOLh8ETeOHXygHM89D/CrLNNj4PVCEAT++ukHYi6cpftX0/AKqs3N08fZu2gOviEN6Dzqq3JjvHUFBWQuXETuunUYWVvjOHYsVt26IhKLyVPmlQrfqHWYSc040OPAU7fQNAjex5OdUsTGmRepXMeRNoOqP9A9LfXWTS7t2srtyxeQSKQEtGxL3U7dsHaq+FbAWck5bJ21C03+LWSFVymQS0AQMDYxoXLdBlRt2ATPwNpPJH4FtRplZCQloVfIuXyDu7H55EoqkW0bgMK0Em9/WgOPoNK485KCfDZMnUBJfi59vpmNvbtnhY+tIkmJimTD1PEPXGA+D6Xn7BsM+emXF9IExoCBJ+GNE7yJN66xecYkmvYbRL2u71a4TS+T9Njb/DFxFLU7dKZRs7bE9+mDtJIjVgt+Y9fyaEry1Yh1alq7RVFl6j/xuYIgsHPOTBKuhzFo7pKHJvS8zui02rKC690nTMUruA6KokJWjfoYGycX+nzzwytvNWng9UGtVLB+yjiKsrNo0vd9jq5ajmvV6nT/6pvHxkEqb97k7jfTUYSFYVKrFk5fT0HuXxoXnq/KJ74gniCHoKe2ySB4n4y023k4+ViVlYUrrbhw6e+KC5HIzS0Ibv82tTq8/cIdGOlxBWyfG4pZXjz1pKcRfzKUuKuXibl4DmVRIcYmpviGNKBqw6Z4BgaX5RnoiopRhIVRHBpKxpVY0u+qyTPxIN/KB6VJqbdWLBZwcJET1MG3LPRMo1SyecYkMhPi6DF5Bm7VAl7o+CqKLd9OISPuDsMWr8RY/nyNce41tqhIAW3AwLPwRgleQa/nj69GoSgqYPD8ZRUeVP8qOPzrEq4fOcD7PyzEJCWNxA8+xKxRQyymzeXgz1dwPLGSOlMGYNm2bdlz7tXzfZ07yz0JakUJG6Z9Sd7dNHpPm8X1w/sIP3qQAbMWvNCGAQbeTPLupvHnxFEoi4tw9KpMr6nfIzN9fJF7KJ078nfsJGPOHHR5edj064fD8C8wsnz2kBmD4H06yqu4UPftd6jRsu1zi6qnIfZqJvuWX8chM4zW3Z2xe68fOq2WpBvXiD5/mpiLZ1EVF2NsLMPFzBqzXDHaAksKLHwosPRCJym11UQm4FTZCpfqjjj5WOHgboGR9J+LdL1Ox845M4m7GkrnMV/hF9LwpY3xeUm9FcX6KWNp0ncg9bv1fOb96LRafh//BXqtloFzlz536IgBA8/DG1WH9+aZE2TE36Hj52P+E2IXoHHvAUSfO83RVcvp+fV3OH09hbtfT8V41SJaeziQmXEJ0zo/lW2vKinmyKplOHj5UKdj11do+fNjbGJK9wlTWTd5LFu/nYKiqJA6HbsYxK6BcrF2cqbLmImEHdhDqyEfP7HYBRCJxVh3fweL1q3IXLCQ3HXrKNi3D8dxY7Hq2vWNCgl601CVlHD9yH6u/E/FhY5fjL2v4sLLxKeWA416+HJ2q4hzG4/RrnE8AFa34/GOK0SW70KKDnJFGuJzUwE1WOVgai7Fxd+ZKg2DcK1ih4Wd/KHfG0EQOLRiCbFXLtFm2KdvlNgFcKlSDe/gOlzevY3gdp2e6lj7X64d2kdOShJdx00xiF0DbwyvhYdXUVhA+NGDhHTu/p9a7g47uJcjK5fy9sgJVG3YlPRZP5CzejVGNjYY2dpSec9fZdse/nUp1w/vp9+3c58roeN1IispgQ1fj0cqkzF4/jKMTZ5tcjVg4ElRRESQPn0GimvXMK1bF4/fVj515ySDh/fxZKcksW7SGNSKEjxqBBHSpQeegbVe+QWGIAgc/y2MyEu5uKSdQS0xI9/KB41xqcdfaqTH0UWGk78DInEKmQlXibt6EbWiBLm5BX71GlKlYVM8AgLLjR+/Vz2kQY++NO713sseXoVw9/Yt/pw0msa9BzxTQyNFYQG/jfgQRx9f3p0045V/5gYMvFEeXhMLyzc+brc8Atu05/qR/RxfuxKfWiE4jhuLOj6eouPHsWj3T+JWSlQk1w7tpU6nrv8ZsQulReP7z1qASIRB7Bp4KZgEBOC5fh3527ejjo83tAl9Qdg6u1KjZVuqN235WpWiEolENB8UREHKCZJpjLmJDi8fS1wCXXH2tcbG2ex/ajNXA1qjVauJv36VW+dOEXX2FOFHD2JiYYlfvUZUadgE9+o1ERsZEXZwL+e3bqBmq3Y06tnvVQ7zuXDyrYJPnXpc/msbtTq8fV+zoCfh7OY/USlKaPn+MIPYNfBG8Vp4eP/L3MuMrf9OL5r0eR9dUTHpM2di814/TGrWRKfVsHbCCNRKBYPmLn2pMW8GDBh4EIOH981Hr9OjVuqQmz3dcrtGrSL+2hVunTvNncsX0KiUmFha4VEjiOhzp/CpHULXMZNeSYfIiiQ99jZ/fDWSRj3fo+G7fZ/4eVmJ8fw+fjhB7d6i9ZBPXqCFBgw8OW+Uh/e/jGu16lRv2pLLu7cR0KINNk4uuMz6vuzxSzu3kp2cyDsTphrErgEDBgxUAGIjMXKzpw+PkxrL8AtpiF9Iw1LxezWU6HOnuBN6AZcq/rw9YvwbL3YBKvn44hvSgNA9O6jVofMTNcsQBIFjv/+KzNSURj3fzHAOA/+/+e8EzL7GNH1vMEZSKcfXrLjv/zmpyZzftoEqDZviUzvkFVlnwIABAwb+jdRYhl/9Rrw9cgKfrdxA76nfI5VVTMOG14GG7/ZDVVJM6N4dT7T9ndCLJIaH0bDne4bGQQbeSAyC9yVgbmNLgx59ib1yidgrl4B72b6LkRjLaDXow1dsoQEDBgwYeBgSqfQ/4dn9Xxy9fPCr34gre3eiKCp85LZajYYTv/+KnZsHQW3fekkWGjBQsRgE70ui9ludsXVx49jqX9Cq1dw4fojkyBs06z/Y0KHGgAEDBgy8dBq92w+1UknoX9sfud3VfbvIS0+jxfvDXknJOQMGKgKD4H1JGEmktBz8EXnpaZxav4aTa3/DtVoANVsa2uwaMGDAgIGXj72HF1UaNPm/9u4nxq6yDgPw+5WWtJIyJXVAZKpICCEGS8SR8MdUBDEqwkZDMEE3YmOi6M7EDYlE90hiYogb1BjSBYSUiNEgVVmgnYJtMEJssIZiLGNCC0hbpvVzMUOxLTL3Ts+dnvnmeZJmek/P3Pt+95788ubk9Nw89ejWvP7Kgbfd59/7X86TDz6Qiz5yZS68/IpFTgjdUXgX0YUbP5yLP3p1nvrFw5k5fCg3fvUbTd13GICl5ZovfDEzhw9l6v+c5X3igZ/myBszue5LX1nkZNAtbWuRXfflO7J67dm55tbbs35iw+mOA8Aytn7ifbn0mk15+pdb8/qB/cf9277nd+eZbb/OFZ+9Jeecf8FpSgjdUHgX2di55+VrP/pJk1+0AcDSc9Xnb8vRN2ayfeuDx7bVWvP4/fdlzdqzF/SNbNA3A119XkrZk+TVJEeTHOn0puy7tiSP3Z0c2JuMTSQ33JVsvLWzp++dXVtyxnJab5/1+djrczZ6b6QzO+n38dl1tj6vtSPrL9iQSz/28fzp0Ycz+eIPctbBv+e5mUvy4u5zc+PmO4f+NrZj+v7e9T0fnRrmv1u2eNsaAAAFSklEQVR+otb6r05ffdeWZOs3k5mDs48PvDD7OGnzoFtu6+2zPn8Wfc7GUtL9zE76fXx2na3Pa+3Y1ZeN5dnfH8n2PSXXjpf8bs/ZGV/9ei5b//LCnrDv713f89G503tJw2N3v3WwvWnm4Oz2Fi239fZZnz+LPmeDPh+fXWfr81o7ds7T9+SDY/uy8+Xz89t9H8irR1bn+vP+mhWPf29hT9j3967v+ejcoIW3JvlVKWVHKeVtvyWhlLK5lDJVSpmanp4e7FkP7B1u+1K33NbbZ33+LPqcjaViNDM76ffx2XW2Pq+1awf25qp3v5CjdUV27n9vLlk7nYl3vdLue9f3fHRu0MJ7ba31iiSfSfL1UsqmE3eotd5Xa52stU6Oj48P9qxjE8NtX+qW23r7rM+fRZ+zsVSMZmYn/T4+u87W57V2bWwi6848lA+t+2dWlqPZdO7fjm1f6PMNtX2x9T0fnRuo8NZa/zH386UkDyW5spNXv+GuZNWa47etWjO7vUXLbb191ufPos/ZWBJGNrOTfh+fXWfr81q7NrfWG96zO3dcvD1jZx5u+73rez46N2/hLaWcVUpZ++bfk3wqyTOdvPrGW5Ob703GNiQpsz9vvrfdC8aX23r7rM+fRZ+z0XsjndlJv4/PrrP1ea1dm1vrinUbctbKI+2/d33PR+dKrfWddyjlosyeIUhm7+rw81rr99/pdyYnJ+vU1FQ3CQEWUSllR+e38VpEZjawnAw6s+e9LVmt9fkkl3eSCoCRMrMBTuab1gAAaJrCCwBA0xReAACapvACANA0hRcAgKYpvAAANE3hBQCgaQovAABNU3gBAGiawgsAQNMUXgAAmqbwAgDQNIUXAICmKbwAADRN4QUAoGkKLwAATVN4AQBomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBpCi8AAE1TeAEAaJrCCwBA0xReAACaNnDhLaWcUUp5upTyyCgDAXDqzGyAtwxzhvdbSf4yqiAAdMrMBpgzUOEtpUwkuSnJj0cbB4BTZWYDHG/QM7z3JPl2kv+MMAsA3TCzAf7HvIW3lPK5JC/VWnfMs9/mUspUKWVqenq6s4AADM7MBjjZIGd4r01ySyllT5IHklxfSvnZiTvVWu+rtU7WWifHx8c7jgnAgMxsgBPMW3hrrd+ptU7UWi9McluS39Rabx95MgCGZmYDnMx9eAEAaNrKYXautW5Lsm0kSQDolJkNMMsZXgAAmqbwAgDQNIUXAICmKbwAADRN4QUAoGkKLwAATVN4AQBomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBpCi8AAE1TeAEAaJrCCwBA0xReAACapvACANA0hRcAgKYpvAAANE3hBQCgaQovAABNU3gBAGiawgsAQNMUXgAAmqbwAgDQNIUXAICmKbwAADRt3sJbSlldSvljKWVnKeXPpZTvLkYwAIZnZgOcbOUA+xxOcn2t9bVSyqokT5RSHq21PjnibAAMz8wGOMG8hbfWWpO8Nvdw1dyfOspQACyMmQ1wskHO8KaUckaSHUkuTvLDWusf3mafzUk2zz18rZTyXGcpARbP+093gFNlZgPLyEAzu8yeDBhMKWVdkoeS3FlrfWaBwQBYBGY2wKyh7tJQa92fZFuST48kDQCdMbMBZg1yl4bxubMEKaWsSfLJJM+OOhgAwzOzAU42yDW85ye5f+6asBVJttRaHxltLAAWyMwGOMFQ1/ACAMBS45vWAABomsILAEDTFF4AAJqm8AIA0DSFFwCApim8AAA0TeEFAKBp/wXIg39D/1/4LwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "fig, ax = plt.subplots(1, 2, figsize=(12, 4))\n", "\n", "n = 15\n", "ax[0].plot(y_train[:n].reset_index(), \"o\")\n", "ax[1].plot(y_test[:n].reset_index(), \"o\")\n", "ax[0].set_title(\"Prédictions sur quelques valeurs\\napprentissage\")\n", "ax[1].set_title(\"Prédictions sur quelques valeurs\\ntest\")\n", "for x in ax:\n", " x.set_ylim([3, 9])\n", " x.get_xaxis().set_visible(False)\n", "\n", "for model in models:\n", " d = model.get_params()[\"polynomialfeatures__degree\"]\n", " tr = model.predict(X_train_norm[:n])\n", " te = model.predict(X_test_norm[:n])\n", " ax[0].plot(tr, label=\"d=%d\" % d)\n", " ax[1].plot(te, label=\"d=%d\" % d)\n", "ax[0].legend()\n", "ax[1].legend();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Le modèle de degré 4 a l'air performant sur la base d'apprentissage mais s'égare complètement sur la base de test comme s'il était surpris des valeurs rencontrées sur la base de test. On dit que le modèle fait du [sur-apprentissage](https://fr.wikipedia.org/wiki/Surapprentissage) ou [overfitting](https://en.wikipedia.org/wiki/Overfitting) en anglais. Le polynôme de degré fonctionne mieux que la régression linéaire simple. On peut se demander quelles sont les variables croisées qui ont un impact sur la performance. On utilise le modèle [statsmodels](http://www.statsmodels.org/stable/index.html)." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "poly = PolynomialFeatures(degree=2)\n", "poly_feat_train = poly.fit_transform(X_train_norm)\n", "poly_feat_test = poly.fit_transform(X_test_norm)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Model: OLS Adj. R-squared: 0.306
Dependent Variable: quality AIC: 10768.5223
Date: 2024-01-23 00:08 BIC: 11268.3493
No. Observations: 4872 Log-Likelihood: -5307.3
Df Model: 76 F-statistic: 29.30
Df Residuals: 4795 Prob (F-statistic): 0.00
R-squared: 0.317 Scale: 0.52557
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Coef. Std.Err. t P>|t| [0.025 0.975]
const 874.2126 1866.6100 0.4683 0.6396 -2785.1996 4533.6248
x1 17.2438 25.1175 0.6865 0.4924 -31.9980 66.4856
x2 -735.9147 164.2593 -4.4802 0.0000 -1057.9383 -413.8911
x3 -375.2205 200.9788 -1.8670 0.0620 -769.2311 18.7900
x4 2.1457 13.7859 0.1556 0.8763 -24.8809 29.1723
x5 -1219.9140 760.0849 -1.6050 0.1086 -2710.0291 270.2011
x6 33.0684 8.6300 3.8318 0.0001 16.1496 49.9873
x7 45.6122 23.6785 1.9263 0.0541 -0.8085 92.0328
x8 -1621.7821 721.4602 -2.2479 0.0246 -3036.1752 -207.3890
x9 -123.6719 196.5043 -0.6294 0.5291 -508.9104 261.5667
x10 -213.6188 172.6441 -1.2373 0.2160 -552.0806 124.8429
x11 274.6811 25.3731 10.8257 0.0000 224.9381 324.4241
x12 -888.0924 1860.0506 -0.4775 0.6331 -4534.6449 2758.4602
x13 213.0448 149.3410 1.4266 0.1538 -79.7320 505.8216
x14 -169.2454 191.2389 -0.8850 0.3762 -544.1614 205.6706
x15 -2.3959 21.0911 -0.1136 0.9096 -43.7441 38.9523
x16 151.4367 661.2643 0.2290 0.8189 -1144.9447 1447.8180
x17 -13.3943 9.8122 -1.3651 0.1723 -32.6306 5.8421
x18 -12.3144 22.1599 -0.5557 0.5784 -55.7580 31.1291
x19 -228.1023 1055.2972 -0.2161 0.8289 -2296.9691 1840.7644
x20 263.5729 260.8085 1.0106 0.3123 -247.7314 774.8773
x21 210.1261 147.0152 1.4293 0.1530 -78.0912 498.3434
x22 -102.4573 26.1357 -3.9202 0.0001 -153.6952 -51.2193
x23 -1256.2263 1979.4050 -0.6346 0.5257 -5136.7684 2624.3158
x24 2503.6940 1629.0043 1.5369 0.1244 -689.9019 5697.2899
x25 -304.5840 139.4970 -2.1834 0.0291 -578.0621 -31.1060
x26 6503.3193 4276.6479 1.5207 0.1284 -1880.8729 14887.5116
x27 176.1465 65.4348 2.6919 0.0071 47.8643 304.4288
x28 541.9165 145.6814 3.7199 0.0002 256.3141 827.5190
x29 -5408.0462 5170.0682 -1.0460 0.2956 -15543.7521 4727.6598
x30 1591.6613 1576.5785 1.0096 0.3128 -1499.1559 4682.4786
x31 -3066.2318 1347.2069 -2.2760 0.0229 -5707.3755 -425.0882
x32 611.6934 183.2814 3.3375 0.0009 252.3778 971.0089
x33 861.9664 2070.4337 0.4163 0.6772 -3197.0336 4920.9664
x34 -307.3877 171.8498 -1.7887 0.0737 -644.2921 29.5168
x35 -8483.4913 6547.3948 -1.2957 0.1951 -21319.3893 4352.4067
x36 150.5489 83.1030 1.8116 0.0701 -12.3711 313.4689
x37 300.7497 178.4947 1.6849 0.0921 -49.1819 650.6813
x38 14067.7740 7800.2113 1.8035 0.0714 -1224.2191 29359.7672
x39 -5133.5558 2077.6861 -2.4708 0.0135 -9206.7738 -1060.3378
x40 -2372.2746 1576.8448 -1.5044 0.1325 -5463.6139 719.0647
x41 708.8006 236.3385 2.9991 0.0027 245.4687 1172.1325
x42 -910.1293 1867.0943 -0.4875 0.6260 -4570.4908 2750.2323
x43 1971.4865 757.4887 2.6027 0.0093 486.4611 3456.5118
x44 -7.6328 5.0273 -1.5183 0.1290 -17.4886 2.2230
x45 2.8665 12.6000 0.2275 0.8200 -21.8354 27.5684
x46 1429.2194 705.7754 2.0250 0.0429 45.5757 2812.8631
x47 -287.7160 203.0709 -1.4168 0.1566 -685.8281 110.3961
x48 -189.7045 168.1916 -1.1279 0.2594 -519.4371 140.0282
x49 -18.0129 19.5540 -0.9212 0.3570 -56.3477 20.3219
x50 10142.2002 7074.9790 1.4335 0.1518 -3728.0049 24012.4054
x51 -201.9536 331.1123 -0.6099 0.5419 -851.0857 447.1785
x52 1197.2260 682.4747 1.7542 0.0795 -140.7375 2535.1896
x53 20265.9335 23064.5613 0.8787 0.3796 -24951.1898 65483.0568
x54 -12226.8108 6517.5063 -1.8760 0.0607 -25004.1137 550.4920
x55 -3613.4768 4978.7999 -0.7258 0.4680 -13374.2092 6147.2555
x56 2196.0436 843.4185 2.6037 0.0092 542.5563 3849.5310
x57 -909.6884 1867.1743 -0.4872 0.6261 -4570.2067 2750.8299
x58 -24.9437 7.6926 -3.2426 0.0012 -40.0247 -9.8627
x59 549.2329 298.2374 1.8416 0.0656 -35.4492 1133.9150
x60 -13.8185 79.9507 -0.1728 0.8628 -170.5585 142.9216
x61 59.9612 69.6414 0.8610 0.3893 -76.5679 196.4903
x62 -60.9958 9.6344 -6.3310 0.0000 -79.8838 -42.1079
x63 -915.2771 1867.4927 -0.4901 0.6241 -4576.4197 2745.8655
x64 856.3795 643.6237 1.3306 0.1834 -405.4183 2118.1773
x65 172.8750 176.1471 0.9814 0.3264 -172.4541 518.2041
x66 233.8040 154.5694 1.5126 0.1304 -69.2230 536.8310
x67 -208.7613 22.8714 -9.1276 0.0000 -253.5998 -163.9229
x68 2377.5655 18554.7387 0.1281 0.8980 -33998.2360 38753.3671
x69 11008.3563 10291.6307 1.0696 0.2848 -9167.9622 31184.6748
x70 -6367.0944 6065.2219 -1.0498 0.2939 -18257.7123 5523.5234
x71 -2160.1700 944.6336 -2.2868 0.0223 -4012.0853 -308.2546
x72 -4031.2392 1137.0986 -3.5452 0.0004 -6260.4741 -1802.0043
x73 1604.8538 1639.6859 0.9788 0.3277 -1609.6830 4819.3905
x74 766.9861 249.3365 3.0761 0.0021 278.1721 1255.8001
x75 -2563.1016 2045.6050 -1.2530 0.2103 -6573.4261 1447.2228
x76 596.8511 210.7614 2.8319 0.0046 183.6620 1010.0402
x77 -1033.7653 1865.5885 -0.5541 0.5795 -4691.1748 2623.6442
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Omnibus: 74.655 Durbin-Watson: 2.012
Prob(Omnibus): 0.000 Jarque-Bera (JB): 119.025
Skew: 0.141 Prob(JB): 0.000
Kurtosis: 3.712 Condition No.: 4390541685726112

\n", "Notes:
\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
\n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that there are strong multicollinearity problems or that the design matrix is singular." ], "text/latex": [ "\\begin{table}\n", "\\caption{Results: Ordinary least squares}\n", "\\label{}\n", "\\begin{center}\n", "\\begin{tabular}{llll}\n", "\\hline\n", "Model: & OLS & Adj. R-squared: & 0.306 \\\\\n", "Dependent Variable: & quality & AIC: & 10768.5223 \\\\\n", "Date: & 2024-01-23 00:08 & BIC: & 11268.3493 \\\\\n", "No. Observations: & 4872 & Log-Likelihood: & -5307.3 \\\\\n", "Df Model: & 76 & F-statistic: & 29.30 \\\\\n", "Df Residuals: & 4795 & Prob (F-statistic): & 0.00 \\\\\n", "R-squared: & 0.317 & Scale: & 0.52557 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\n", "\\begin{center}\n", "\\begin{tabular}{lrrrrrr}\n", "\\hline\n", " & Coef. & Std.Err. & t & P$> |$t$|$ & [0.025 & 0.975] \\\\\n", "\\hline\n", "const & 874.2126 & 1866.6100 & 0.4683 & 0.6396 & -2785.1996 & 4533.6248 \\\\\n", "x1 & 17.2438 & 25.1175 & 0.6865 & 0.4924 & -31.9980 & 66.4856 \\\\\n", "x2 & -735.9147 & 164.2593 & -4.4802 & 0.0000 & -1057.9383 & -413.8911 \\\\\n", "x3 & -375.2205 & 200.9788 & -1.8670 & 0.0620 & -769.2311 & 18.7900 \\\\\n", "x4 & 2.1457 & 13.7859 & 0.1556 & 0.8763 & -24.8809 & 29.1723 \\\\\n", "x5 & -1219.9140 & 760.0849 & -1.6050 & 0.1086 & -2710.0291 & 270.2011 \\\\\n", "x6 & 33.0684 & 8.6300 & 3.8318 & 0.0001 & 16.1496 & 49.9873 \\\\\n", "x7 & 45.6122 & 23.6785 & 1.9263 & 0.0541 & -0.8085 & 92.0328 \\\\\n", "x8 & -1621.7821 & 721.4602 & -2.2479 & 0.0246 & -3036.1752 & -207.3890 \\\\\n", "x9 & -123.6719 & 196.5043 & -0.6294 & 0.5291 & -508.9104 & 261.5667 \\\\\n", "x10 & -213.6188 & 172.6441 & -1.2373 & 0.2160 & -552.0806 & 124.8429 \\\\\n", "x11 & 274.6811 & 25.3731 & 10.8257 & 0.0000 & 224.9381 & 324.4241 \\\\\n", "x12 & -888.0924 & 1860.0506 & -0.4775 & 0.6331 & -4534.6449 & 2758.4602 \\\\\n", "x13 & 213.0448 & 149.3410 & 1.4266 & 0.1538 & -79.7320 & 505.8216 \\\\\n", "x14 & -169.2454 & 191.2389 & -0.8850 & 0.3762 & -544.1614 & 205.6706 \\\\\n", "x15 & -2.3959 & 21.0911 & -0.1136 & 0.9096 & -43.7441 & 38.9523 \\\\\n", "x16 & 151.4367 & 661.2643 & 0.2290 & 0.8189 & -1144.9447 & 1447.8180 \\\\\n", "x17 & -13.3943 & 9.8122 & -1.3651 & 0.1723 & -32.6306 & 5.8421 \\\\\n", "x18 & -12.3144 & 22.1599 & -0.5557 & 0.5784 & -55.7580 & 31.1291 \\\\\n", "x19 & -228.1023 & 1055.2972 & -0.2161 & 0.8289 & -2296.9691 & 1840.7644 \\\\\n", "x20 & 263.5729 & 260.8085 & 1.0106 & 0.3123 & -247.7314 & 774.8773 \\\\\n", "x21 & 210.1261 & 147.0152 & 1.4293 & 0.1530 & -78.0912 & 498.3434 \\\\\n", "x22 & -102.4573 & 26.1357 & -3.9202 & 0.0001 & -153.6952 & -51.2193 \\\\\n", "x23 & -1256.2263 & 1979.4050 & -0.6346 & 0.5257 & -5136.7684 & 2624.3158 \\\\\n", "x24 & 2503.6940 & 1629.0043 & 1.5369 & 0.1244 & -689.9019 & 5697.2899 \\\\\n", "x25 & -304.5840 & 139.4970 & -2.1834 & 0.0291 & -578.0621 & -31.1060 \\\\\n", "x26 & 6503.3193 & 4276.6479 & 1.5207 & 0.1284 & -1880.8729 & 14887.5116 \\\\\n", "x27 & 176.1465 & 65.4348 & 2.6919 & 0.0071 & 47.8643 & 304.4288 \\\\\n", "x28 & 541.9165 & 145.6814 & 3.7199 & 0.0002 & 256.3141 & 827.5190 \\\\\n", "x29 & -5408.0462 & 5170.0682 & -1.0460 & 0.2956 & -15543.7521 & 4727.6598 \\\\\n", "x30 & 1591.6613 & 1576.5785 & 1.0096 & 0.3128 & -1499.1559 & 4682.4786 \\\\\n", "x31 & -3066.2318 & 1347.2069 & -2.2760 & 0.0229 & -5707.3755 & -425.0882 \\\\\n", "x32 & 611.6934 & 183.2814 & 3.3375 & 0.0009 & 252.3778 & 971.0089 \\\\\n", "x33 & 861.9664 & 2070.4337 & 0.4163 & 0.6772 & -3197.0336 & 4920.9664 \\\\\n", "x34 & -307.3877 & 171.8498 & -1.7887 & 0.0737 & -644.2921 & 29.5168 \\\\\n", "x35 & -8483.4913 & 6547.3948 & -1.2957 & 0.1951 & -21319.3893 & 4352.4067 \\\\\n", "x36 & 150.5489 & 83.1030 & 1.8116 & 0.0701 & -12.3711 & 313.4689 \\\\\n", "x37 & 300.7497 & 178.4947 & 1.6849 & 0.0921 & -49.1819 & 650.6813 \\\\\n", "x38 & 14067.7740 & 7800.2113 & 1.8035 & 0.0714 & -1224.2191 & 29359.7672 \\\\\n", "x39 & -5133.5558 & 2077.6861 & -2.4708 & 0.0135 & -9206.7738 & -1060.3378 \\\\\n", "x40 & -2372.2746 & 1576.8448 & -1.5044 & 0.1325 & -5463.6139 & 719.0647 \\\\\n", "x41 & 708.8006 & 236.3385 & 2.9991 & 0.0027 & 245.4687 & 1172.1325 \\\\\n", "x42 & -910.1293 & 1867.0943 & -0.4875 & 0.6260 & -4570.4908 & 2750.2323 \\\\\n", "x43 & 1971.4865 & 757.4887 & 2.6027 & 0.0093 & 486.4611 & 3456.5118 \\\\\n", "x44 & -7.6328 & 5.0273 & -1.5183 & 0.1290 & -17.4886 & 2.2230 \\\\\n", "x45 & 2.8665 & 12.6000 & 0.2275 & 0.8200 & -21.8354 & 27.5684 \\\\\n", "x46 & 1429.2194 & 705.7754 & 2.0250 & 0.0429 & 45.5757 & 2812.8631 \\\\\n", "x47 & -287.7160 & 203.0709 & -1.4168 & 0.1566 & -685.8281 & 110.3961 \\\\\n", "x48 & -189.7045 & 168.1916 & -1.1279 & 0.2594 & -519.4371 & 140.0282 \\\\\n", "x49 & -18.0129 & 19.5540 & -0.9212 & 0.3570 & -56.3477 & 20.3219 \\\\\n", "x50 & 10142.2002 & 7074.9790 & 1.4335 & 0.1518 & -3728.0049 & 24012.4054 \\\\\n", "x51 & -201.9536 & 331.1123 & -0.6099 & 0.5419 & -851.0857 & 447.1785 \\\\\n", "x52 & 1197.2260 & 682.4747 & 1.7542 & 0.0795 & -140.7375 & 2535.1896 \\\\\n", "x53 & 20265.9335 & 23064.5613 & 0.8787 & 0.3796 & -24951.1898 & 65483.0568 \\\\\n", "x54 & -12226.8108 & 6517.5063 & -1.8760 & 0.0607 & -25004.1137 & 550.4920 \\\\\n", "x55 & -3613.4768 & 4978.7999 & -0.7258 & 0.4680 & -13374.2092 & 6147.2555 \\\\\n", "x56 & 2196.0436 & 843.4185 & 2.6037 & 0.0092 & 542.5563 & 3849.5310 \\\\\n", "x57 & -909.6884 & 1867.1743 & -0.4872 & 0.6261 & -4570.2067 & 2750.8299 \\\\\n", "x58 & -24.9437 & 7.6926 & -3.2426 & 0.0012 & -40.0247 & -9.8627 \\\\\n", "x59 & 549.2329 & 298.2374 & 1.8416 & 0.0656 & -35.4492 & 1133.9150 \\\\\n", "x60 & -13.8185 & 79.9507 & -0.1728 & 0.8628 & -170.5585 & 142.9216 \\\\\n", "x61 & 59.9612 & 69.6414 & 0.8610 & 0.3893 & -76.5679 & 196.4903 \\\\\n", "x62 & -60.9958 & 9.6344 & -6.3310 & 0.0000 & -79.8838 & -42.1079 \\\\\n", "x63 & -915.2771 & 1867.4927 & -0.4901 & 0.6241 & -4576.4197 & 2745.8655 \\\\\n", "x64 & 856.3795 & 643.6237 & 1.3306 & 0.1834 & -405.4183 & 2118.1773 \\\\\n", "x65 & 172.8750 & 176.1471 & 0.9814 & 0.3264 & -172.4541 & 518.2041 \\\\\n", "x66 & 233.8040 & 154.5694 & 1.5126 & 0.1304 & -69.2230 & 536.8310 \\\\\n", "x67 & -208.7613 & 22.8714 & -9.1276 & 0.0000 & -253.5998 & -163.9229 \\\\\n", "x68 & 2377.5655 & 18554.7387 & 0.1281 & 0.8980 & -33998.2360 & 38753.3671 \\\\\n", "x69 & 11008.3563 & 10291.6307 & 1.0696 & 0.2848 & -9167.9622 & 31184.6748 \\\\\n", "x70 & -6367.0944 & 6065.2219 & -1.0498 & 0.2939 & -18257.7123 & 5523.5234 \\\\\n", "x71 & -2160.1700 & 944.6336 & -2.2868 & 0.0223 & -4012.0853 & -308.2546 \\\\\n", "x72 & -4031.2392 & 1137.0986 & -3.5452 & 0.0004 & -6260.4741 & -1802.0043 \\\\\n", "x73 & 1604.8538 & 1639.6859 & 0.9788 & 0.3277 & -1609.6830 & 4819.3905 \\\\\n", "x74 & 766.9861 & 249.3365 & 3.0761 & 0.0021 & 278.1721 & 1255.8001 \\\\\n", "x75 & -2563.1016 & 2045.6050 & -1.2530 & 0.2103 & -6573.4261 & 1447.2228 \\\\\n", "x76 & 596.8511 & 210.7614 & 2.8319 & 0.0046 & 183.6620 & 1010.0402 \\\\\n", "x77 & -1033.7653 & 1865.5885 & -0.5541 & 0.5795 & -4691.1748 & 2623.6442 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\n", "\\begin{center}\n", "\\begin{tabular}{llll}\n", "\\hline\n", "Omnibus: & 74.655 & Durbin-Watson: & 2.012 \\\\\n", "Prob(Omnibus): & 0.000 & Jarque-Bera (JB): & 119.025 \\\\\n", "Skew: & 0.141 & Prob(JB): & 0.000 \\\\\n", "Kurtosis: & 3.712 & Condition No.: & 4390541685726112 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\\end{table}\n", "\\bigskip\n", "Notes: \\newline \n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline \n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that there are strong multicollinearity problems or that the design matrix is singular." ], "text/plain": [ "\n", "\"\"\"\n", " Results: Ordinary least squares\n", "===================================================================\n", "Model: OLS Adj. R-squared: 0.306 \n", "Dependent Variable: quality AIC: 10768.5223\n", "Date: 2024-01-23 00:08 BIC: 11268.3493\n", "No. Observations: 4872 Log-Likelihood: -5307.3 \n", "Df Model: 76 F-statistic: 29.30 \n", "Df Residuals: 4795 Prob (F-statistic): 0.00 \n", "R-squared: 0.317 Scale: 0.52557 \n", "-------------------------------------------------------------------\n", " Coef. Std.Err. t P>|t| [0.025 0.975] \n", "-------------------------------------------------------------------\n", "const 874.2126 1866.6100 0.4683 0.6396 -2785.1996 4533.6248\n", "x1 17.2438 25.1175 0.6865 0.4924 -31.9980 66.4856\n", "x2 -735.9147 164.2593 -4.4802 0.0000 -1057.9383 -413.8911\n", "x3 -375.2205 200.9788 -1.8670 0.0620 -769.2311 18.7900\n", "x4 2.1457 13.7859 0.1556 0.8763 -24.8809 29.1723\n", "x5 -1219.9140 760.0849 -1.6050 0.1086 -2710.0291 270.2011\n", "x6 33.0684 8.6300 3.8318 0.0001 16.1496 49.9873\n", "x7 45.6122 23.6785 1.9263 0.0541 -0.8085 92.0328\n", "x8 -1621.7821 721.4602 -2.2479 0.0246 -3036.1752 -207.3890\n", "x9 -123.6719 196.5043 -0.6294 0.5291 -508.9104 261.5667\n", "x10 -213.6188 172.6441 -1.2373 0.2160 -552.0806 124.8429\n", "x11 274.6811 25.3731 10.8257 0.0000 224.9381 324.4241\n", "x12 -888.0924 1860.0506 -0.4775 0.6331 -4534.6449 2758.4602\n", "x13 213.0448 149.3410 1.4266 0.1538 -79.7320 505.8216\n", "x14 -169.2454 191.2389 -0.8850 0.3762 -544.1614 205.6706\n", "x15 -2.3959 21.0911 -0.1136 0.9096 -43.7441 38.9523\n", "x16 151.4367 661.2643 0.2290 0.8189 -1144.9447 1447.8180\n", "x17 -13.3943 9.8122 -1.3651 0.1723 -32.6306 5.8421\n", "x18 -12.3144 22.1599 -0.5557 0.5784 -55.7580 31.1291\n", "x19 -228.1023 1055.2972 -0.2161 0.8289 -2296.9691 1840.7644\n", "x20 263.5729 260.8085 1.0106 0.3123 -247.7314 774.8773\n", "x21 210.1261 147.0152 1.4293 0.1530 -78.0912 498.3434\n", "x22 -102.4573 26.1357 -3.9202 0.0001 -153.6952 -51.2193\n", "x23 -1256.2263 1979.4050 -0.6346 0.5257 -5136.7684 2624.3158\n", "x24 2503.6940 1629.0043 1.5369 0.1244 -689.9019 5697.2899\n", "x25 -304.5840 139.4970 -2.1834 0.0291 -578.0621 -31.1060\n", "x26 6503.3193 4276.6479 1.5207 0.1284 -1880.8729 14887.5116\n", "x27 176.1465 65.4348 2.6919 0.0071 47.8643 304.4288\n", "x28 541.9165 145.6814 3.7199 0.0002 256.3141 827.5190\n", "x29 -5408.0462 5170.0682 -1.0460 0.2956 -15543.7521 4727.6598\n", "x30 1591.6613 1576.5785 1.0096 0.3128 -1499.1559 4682.4786\n", "x31 -3066.2318 1347.2069 -2.2760 0.0229 -5707.3755 -425.0882\n", "x32 611.6934 183.2814 3.3375 0.0009 252.3778 971.0089\n", "x33 861.9664 2070.4337 0.4163 0.6772 -3197.0336 4920.9664\n", "x34 -307.3877 171.8498 -1.7887 0.0737 -644.2921 29.5168\n", "x35 -8483.4913 6547.3948 -1.2957 0.1951 -21319.3893 4352.4067\n", "x36 150.5489 83.1030 1.8116 0.0701 -12.3711 313.4689\n", "x37 300.7497 178.4947 1.6849 0.0921 -49.1819 650.6813\n", "x38 14067.7740 7800.2113 1.8035 0.0714 -1224.2191 29359.7672\n", "x39 -5133.5558 2077.6861 -2.4708 0.0135 -9206.7738 -1060.3378\n", "x40 -2372.2746 1576.8448 -1.5044 0.1325 -5463.6139 719.0647\n", "x41 708.8006 236.3385 2.9991 0.0027 245.4687 1172.1325\n", "x42 -910.1293 1867.0943 -0.4875 0.6260 -4570.4908 2750.2323\n", "x43 1971.4865 757.4887 2.6027 0.0093 486.4611 3456.5118\n", "x44 -7.6328 5.0273 -1.5183 0.1290 -17.4886 2.2230\n", "x45 2.8665 12.6000 0.2275 0.8200 -21.8354 27.5684\n", "x46 1429.2194 705.7754 2.0250 0.0429 45.5757 2812.8631\n", "x47 -287.7160 203.0709 -1.4168 0.1566 -685.8281 110.3961\n", "x48 -189.7045 168.1916 -1.1279 0.2594 -519.4371 140.0282\n", "x49 -18.0129 19.5540 -0.9212 0.3570 -56.3477 20.3219\n", "x50 10142.2002 7074.9790 1.4335 0.1518 -3728.0049 24012.4054\n", "x51 -201.9536 331.1123 -0.6099 0.5419 -851.0857 447.1785\n", "x52 1197.2260 682.4747 1.7542 0.0795 -140.7375 2535.1896\n", "x53 20265.9335 23064.5613 0.8787 0.3796 -24951.1898 65483.0568\n", "x54 -12226.8108 6517.5063 -1.8760 0.0607 -25004.1137 550.4920\n", "x55 -3613.4768 4978.7999 -0.7258 0.4680 -13374.2092 6147.2555\n", "x56 2196.0436 843.4185 2.6037 0.0092 542.5563 3849.5310\n", "x57 -909.6884 1867.1743 -0.4872 0.6261 -4570.2067 2750.8299\n", "x58 -24.9437 7.6926 -3.2426 0.0012 -40.0247 -9.8627\n", "x59 549.2329 298.2374 1.8416 0.0656 -35.4492 1133.9150\n", "x60 -13.8185 79.9507 -0.1728 0.8628 -170.5585 142.9216\n", "x61 59.9612 69.6414 0.8610 0.3893 -76.5679 196.4903\n", "x62 -60.9958 9.6344 -6.3310 0.0000 -79.8838 -42.1079\n", "x63 -915.2771 1867.4927 -0.4901 0.6241 -4576.4197 2745.8655\n", "x64 856.3795 643.6237 1.3306 0.1834 -405.4183 2118.1773\n", "x65 172.8750 176.1471 0.9814 0.3264 -172.4541 518.2041\n", "x66 233.8040 154.5694 1.5126 0.1304 -69.2230 536.8310\n", "x67 -208.7613 22.8714 -9.1276 0.0000 -253.5998 -163.9229\n", "x68 2377.5655 18554.7387 0.1281 0.8980 -33998.2360 38753.3671\n", "x69 11008.3563 10291.6307 1.0696 0.2848 -9167.9622 31184.6748\n", "x70 -6367.0944 6065.2219 -1.0498 0.2939 -18257.7123 5523.5234\n", "x71 -2160.1700 944.6336 -2.2868 0.0223 -4012.0853 -308.2546\n", "x72 -4031.2392 1137.0986 -3.5452 0.0004 -6260.4741 -1802.0043\n", "x73 1604.8538 1639.6859 0.9788 0.3277 -1609.6830 4819.3905\n", "x74 766.9861 249.3365 3.0761 0.0021 278.1721 1255.8001\n", "x75 -2563.1016 2045.6050 -1.2530 0.2103 -6573.4261 1447.2228\n", "x76 596.8511 210.7614 2.8319 0.0046 183.6620 1010.0402\n", "x77 -1033.7653 1865.5885 -0.5541 0.5795 -4691.1748 2623.6442\n", "-------------------------------------------------------------------\n", "Omnibus: 74.655 Durbin-Watson: 2.012 \n", "Prob(Omnibus): 0.000 Jarque-Bera (JB): 119.025 \n", "Skew: 0.141 Prob(JB): 0.000 \n", "Kurtosis: 3.712 Condition No.: 4390541685726112\n", "===================================================================\n", "Notes:\n", "[1] Standard Errors assume that the covariance matrix of the errors\n", "is correctly specified.\n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that\n", "there are strong multicollinearity problems or that the design\n", "matrix is singular.\n", "\"\"\"" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from statsmodels.regression.linear_model import OLS\n", "\n", "model = OLS(y_train, poly_feat_train)\n", "results = model.fit()\n", "results.summary2()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ce n'est pas très lisible. Il faut ajouter le nom de chaque variable et recommencer." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
1fixed_acidityvolatile_aciditycitric_acidresidual_sugarchloridesfree_sulfur_dioxidetotal_sulfur_dioxidedensitypH...density^2density * pHdensity * sulphatesdensity * alcoholpH^2pH * sulphatespH * alcoholsulphates^2sulphates * alcoholalcohol^2
01.00.0613160.0035110.0034610.0197790.0004550.3065820.9395260.0097730.030263...0.0000960.0002960.0000440.0013150.0009160.0001380.0040700.0000210.0006120.018090
11.00.0352360.0013730.0019220.0654380.0002060.2059240.9747060.0045720.014552...0.0000210.0000670.0000130.0001920.0002120.0000420.0006130.0000080.0001210.001772
21.00.0425790.0013190.0019190.1013490.0003360.2938520.9475220.0059960.020210...0.0000360.0001210.0000140.0003450.0004080.0000460.0011630.0000050.0001310.003314
31.00.0536380.0009200.0014560.0375470.0004210.2068900.9731470.0076270.025210...0.0000580.0001920.0000240.0005490.0006360.0000790.0018160.0000100.0002260.005188
41.00.0714980.0024130.0029490.0107250.0004470.3664280.9205400.0088480.026812...0.0000780.0002370.0000360.0009810.0007190.0001080.0029710.0000160.0004460.012282
\n", "

5 rows × 78 columns

\n", "
" ], "text/plain": [ " 1 fixed_acidity volatile_acidity citric_acid residual_sugar \\\n", "0 1.0 0.061316 0.003511 0.003461 0.019779 \n", "1 1.0 0.035236 0.001373 0.001922 0.065438 \n", "2 1.0 0.042579 0.001319 0.001919 0.101349 \n", "3 1.0 0.053638 0.000920 0.001456 0.037547 \n", "4 1.0 0.071498 0.002413 0.002949 0.010725 \n", "\n", " chlorides free_sulfur_dioxide total_sulfur_dioxide density pH \\\n", "0 0.000455 0.306582 0.939526 0.009773 0.030263 \n", "1 0.000206 0.205924 0.974706 0.004572 0.014552 \n", "2 0.000336 0.293852 0.947522 0.005996 0.020210 \n", "3 0.000421 0.206890 0.973147 0.007627 0.025210 \n", "4 0.000447 0.366428 0.920540 0.008848 0.026812 \n", "\n", " ... density^2 density * pH density * sulphates density * alcohol \\\n", "0 ... 0.000096 0.000296 0.000044 0.001315 \n", "1 ... 0.000021 0.000067 0.000013 0.000192 \n", "2 ... 0.000036 0.000121 0.000014 0.000345 \n", "3 ... 0.000058 0.000192 0.000024 0.000549 \n", "4 ... 0.000078 0.000237 0.000036 0.000981 \n", "\n", " pH^2 pH * sulphates pH * alcohol sulphates^2 sulphates * alcohol \\\n", "0 0.000916 0.000138 0.004070 0.000021 0.000612 \n", "1 0.000212 0.000042 0.000613 0.000008 0.000121 \n", "2 0.000408 0.000046 0.001163 0.000005 0.000131 \n", "3 0.000636 0.000079 0.001816 0.000010 0.000226 \n", "4 0.000719 0.000108 0.002971 0.000016 0.000446 \n", "\n", " alcohol^2 \n", "0 0.018090 \n", "1 0.001772 \n", "2 0.003314 \n", "3 0.005188 \n", "4 0.012282 \n", "\n", "[5 rows x 78 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "names = poly.get_feature_names_out(input_features=data.columns[:-2])\n", "names = [n.replace(\" \", \" * \") for n in names]\n", "pft = pandas.DataFrame(poly_feat_train, columns=names)\n", "pft.head()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Model: OLS Adj. R-squared: 0.306
Dependent Variable: quality AIC: 10768.5223
Date: 2024-01-23 00:09 BIC: 11268.3493
No. Observations: 4872 Log-Likelihood: -5307.3
Df Model: 76 F-statistic: 29.30
Df Residuals: 4795 Prob (F-statistic): 0.00
R-squared: 0.317 Scale: 0.52557
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Coef. Std.Err. t P>|t| [0.025 0.975]
1 874.2126 1866.6100 0.4683 0.6396 -2785.1996 4533.6248
fixed_acidity 17.2438 25.1175 0.6865 0.4924 -31.9980 66.4856
volatile_acidity -735.9147 164.2593 -4.4802 0.0000 -1057.9383 -413.8911
citric_acid -375.2205 200.9788 -1.8670 0.0620 -769.2311 18.7900
residual_sugar 2.1457 13.7859 0.1556 0.8763 -24.8809 29.1723
chlorides -1219.9140 760.0849 -1.6050 0.1086 -2710.0291 270.2011
free_sulfur_dioxide 33.0684 8.6300 3.8318 0.0001 16.1496 49.9873
total_sulfur_dioxide 45.6122 23.6785 1.9263 0.0541 -0.8085 92.0328
density -1621.7821 721.4602 -2.2479 0.0246 -3036.1752 -207.3890
pH -123.6719 196.5043 -0.6294 0.5291 -508.9104 261.5667
sulphates -213.6188 172.6441 -1.2373 0.2160 -552.0806 124.8429
alcohol 274.6811 25.3731 10.8257 0.0000 224.9381 324.4241
fixed_acidity^2 -888.0924 1860.0506 -0.4775 0.6331 -4534.6449 2758.4602
fixed_acidity * volatile_acidity 213.0448 149.3410 1.4266 0.1538 -79.7320 505.8216
fixed_acidity * citric_acid -169.2454 191.2389 -0.8850 0.3762 -544.1614 205.6706
fixed_acidity * residual_sugar -2.3959 21.0911 -0.1136 0.9096 -43.7441 38.9523
fixed_acidity * chlorides 151.4367 661.2643 0.2290 0.8189 -1144.9447 1447.8180
fixed_acidity * free_sulfur_dioxide -13.3943 9.8122 -1.3651 0.1723 -32.6306 5.8421
fixed_acidity * total_sulfur_dioxide -12.3144 22.1599 -0.5557 0.5784 -55.7580 31.1291
fixed_acidity * density -228.1023 1055.2972 -0.2161 0.8289 -2296.9691 1840.7644
fixed_acidity * pH 263.5729 260.8085 1.0106 0.3123 -247.7314 774.8773
fixed_acidity * sulphates 210.1261 147.0152 1.4293 0.1530 -78.0912 498.3434
fixed_acidity * alcohol -102.4573 26.1357 -3.9202 0.0001 -153.6952 -51.2193
volatile_acidity^2 -1256.2263 1979.4050 -0.6346 0.5257 -5136.7684 2624.3158
volatile_acidity * citric_acid 2503.6940 1629.0043 1.5369 0.1244 -689.9019 5697.2899
volatile_acidity * residual_sugar -304.5840 139.4970 -2.1834 0.0291 -578.0621 -31.1060
volatile_acidity * chlorides 6503.3193 4276.6479 1.5207 0.1284 -1880.8729 14887.5116
volatile_acidity * free_sulfur_dioxide 176.1465 65.4348 2.6919 0.0071 47.8643 304.4288
volatile_acidity * total_sulfur_dioxide 541.9165 145.6814 3.7199 0.0002 256.3141 827.5190
volatile_acidity * density -5408.0462 5170.0682 -1.0460 0.2956 -15543.7521 4727.6598
volatile_acidity * pH 1591.6613 1576.5785 1.0096 0.3128 -1499.1559 4682.4786
volatile_acidity * sulphates -3066.2318 1347.2069 -2.2760 0.0229 -5707.3755 -425.0882
volatile_acidity * alcohol 611.6934 183.2814 3.3375 0.0009 252.3778 971.0089
citric_acid^2 861.9664 2070.4337 0.4163 0.6772 -3197.0336 4920.9664
citric_acid * residual_sugar -307.3877 171.8498 -1.7887 0.0737 -644.2921 29.5168
citric_acid * chlorides -8483.4913 6547.3948 -1.2957 0.1951 -21319.3893 4352.4067
citric_acid * free_sulfur_dioxide 150.5489 83.1030 1.8116 0.0701 -12.3711 313.4689
citric_acid * total_sulfur_dioxide 300.7497 178.4947 1.6849 0.0921 -49.1819 650.6813
citric_acid * density 14067.7740 7800.2113 1.8035 0.0714 -1224.2191 29359.7672
citric_acid * pH -5133.5558 2077.6861 -2.4708 0.0135 -9206.7738 -1060.3378
citric_acid * sulphates -2372.2746 1576.8448 -1.5044 0.1325 -5463.6139 719.0647
citric_acid * alcohol 708.8006 236.3385 2.9991 0.0027 245.4687 1172.1325
residual_sugar^2 -910.1293 1867.0943 -0.4875 0.6260 -4570.4908 2750.2323
residual_sugar * chlorides 1971.4865 757.4887 2.6027 0.0093 486.4611 3456.5118
residual_sugar * free_sulfur_dioxide -7.6328 5.0273 -1.5183 0.1290 -17.4886 2.2230
residual_sugar * total_sulfur_dioxide 2.8665 12.6000 0.2275 0.8200 -21.8354 27.5684
residual_sugar * density 1429.2194 705.7754 2.0250 0.0429 45.5757 2812.8631
residual_sugar * pH -287.7160 203.0709 -1.4168 0.1566 -685.8281 110.3961
residual_sugar * sulphates -189.7045 168.1916 -1.1279 0.2594 -519.4371 140.0282
residual_sugar * alcohol -18.0129 19.5540 -0.9212 0.3570 -56.3477 20.3219
chlorides^2 10142.2002 7074.9790 1.4335 0.1518 -3728.0049 24012.4054
chlorides * free_sulfur_dioxide -201.9536 331.1123 -0.6099 0.5419 -851.0857 447.1785
chlorides * total_sulfur_dioxide 1197.2260 682.4747 1.7542 0.0795 -140.7375 2535.1896
chlorides * density 20265.9335 23064.5613 0.8787 0.3796 -24951.1898 65483.0568
chlorides * pH -12226.8108 6517.5063 -1.8760 0.0607 -25004.1137 550.4920
chlorides * sulphates -3613.4768 4978.7999 -0.7258 0.4680 -13374.2092 6147.2555
chlorides * alcohol 2196.0436 843.4185 2.6037 0.0092 542.5563 3849.5310
free_sulfur_dioxide^2 -909.6884 1867.1743 -0.4872 0.6261 -4570.2067 2750.8299
free_sulfur_dioxide * total_sulfur_dioxide -24.9437 7.6926 -3.2426 0.0012 -40.0247 -9.8627
free_sulfur_dioxide * density 549.2329 298.2374 1.8416 0.0656 -35.4492 1133.9150
free_sulfur_dioxide * pH -13.8185 79.9507 -0.1728 0.8628 -170.5585 142.9216
free_sulfur_dioxide * sulphates 59.9612 69.6414 0.8610 0.3893 -76.5679 196.4903
free_sulfur_dioxide * alcohol -60.9958 9.6344 -6.3310 0.0000 -79.8838 -42.1079
total_sulfur_dioxide^2 -915.2771 1867.4927 -0.4901 0.6241 -4576.4197 2745.8655
total_sulfur_dioxide * density 856.3795 643.6237 1.3306 0.1834 -405.4183 2118.1773
total_sulfur_dioxide * pH 172.8750 176.1471 0.9814 0.3264 -172.4541 518.2041
total_sulfur_dioxide * sulphates 233.8040 154.5694 1.5126 0.1304 -69.2230 536.8310
total_sulfur_dioxide * alcohol -208.7613 22.8714 -9.1276 0.0000 -253.5998 -163.9229
density^2 2377.5655 18554.7387 0.1281 0.8980 -33998.2360 38753.3671
density * pH 11008.3563 10291.6307 1.0696 0.2848 -9167.9622 31184.6748
density * sulphates -6367.0944 6065.2219 -1.0498 0.2939 -18257.7123 5523.5234
density * alcohol -2160.1700 944.6336 -2.2868 0.0223 -4012.0853 -308.2546
pH^2 -4031.2392 1137.0986 -3.5452 0.0004 -6260.4741 -1802.0043
pH * sulphates 1604.8538 1639.6859 0.9788 0.3277 -1609.6830 4819.3905
pH * alcohol 766.9861 249.3365 3.0761 0.0021 278.1721 1255.8001
sulphates^2 -2563.1016 2045.6050 -1.2530 0.2103 -6573.4261 1447.2228
sulphates * alcohol 596.8511 210.7614 2.8319 0.0046 183.6620 1010.0402
alcohol^2 -1033.7653 1865.5885 -0.5541 0.5795 -4691.1748 2623.6442
\n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "\n", " \n", "\n", "
Omnibus: 74.655 Durbin-Watson: 2.012
Prob(Omnibus): 0.000 Jarque-Bera (JB): 119.025
Skew: 0.141 Prob(JB): 0.000
Kurtosis: 3.712 Condition No.: 4390541685726112

\n", "Notes:
\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
\n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that there are strong multicollinearity problems or that the design matrix is singular." ], "text/latex": [ "\\begin{table}\n", "\\caption{Results: Ordinary least squares}\n", "\\label{}\n", "\\begin{center}\n", "\\begin{tabular}{llll}\n", "\\hline\n", "Model: & OLS & Adj. R-squared: & 0.306 \\\\\n", "Dependent Variable: & quality & AIC: & 10768.5223 \\\\\n", "Date: & 2024-01-23 00:09 & BIC: & 11268.3493 \\\\\n", "No. Observations: & 4872 & Log-Likelihood: & -5307.3 \\\\\n", "Df Model: & 76 & F-statistic: & 29.30 \\\\\n", "Df Residuals: & 4795 & Prob (F-statistic): & 0.00 \\\\\n", "R-squared: & 0.317 & Scale: & 0.52557 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\n", "\\begin{center}\n", "\\begin{tabular}{lrrrrrr}\n", "\\hline\n", " & Coef. & Std.Err. & t & P$> |$t$|$ & [0.025 & 0.975] \\\\\n", "\\hline\n", "1 & 874.2126 & 1866.6100 & 0.4683 & 0.6396 & -2785.1996 & 4533.6248 \\\\\n", "fixed\\_acidity & 17.2438 & 25.1175 & 0.6865 & 0.4924 & -31.9980 & 66.4856 \\\\\n", "volatile\\_acidity & -735.9147 & 164.2593 & -4.4802 & 0.0000 & -1057.9383 & -413.8911 \\\\\n", "citric\\_acid & -375.2205 & 200.9788 & -1.8670 & 0.0620 & -769.2311 & 18.7900 \\\\\n", "residual\\_sugar & 2.1457 & 13.7859 & 0.1556 & 0.8763 & -24.8809 & 29.1723 \\\\\n", "chlorides & -1219.9140 & 760.0849 & -1.6050 & 0.1086 & -2710.0291 & 270.2011 \\\\\n", "free\\_sulfur\\_dioxide & 33.0684 & 8.6300 & 3.8318 & 0.0001 & 16.1496 & 49.9873 \\\\\n", "total\\_sulfur\\_dioxide & 45.6122 & 23.6785 & 1.9263 & 0.0541 & -0.8085 & 92.0328 \\\\\n", "density & -1621.7821 & 721.4602 & -2.2479 & 0.0246 & -3036.1752 & -207.3890 \\\\\n", "pH & -123.6719 & 196.5043 & -0.6294 & 0.5291 & -508.9104 & 261.5667 \\\\\n", "sulphates & -213.6188 & 172.6441 & -1.2373 & 0.2160 & -552.0806 & 124.8429 \\\\\n", "alcohol & 274.6811 & 25.3731 & 10.8257 & 0.0000 & 224.9381 & 324.4241 \\\\\n", "fixed\\_acidity^2 & -888.0924 & 1860.0506 & -0.4775 & 0.6331 & -4534.6449 & 2758.4602 \\\\\n", "fixed\\_acidity * volatile\\_acidity & 213.0448 & 149.3410 & 1.4266 & 0.1538 & -79.7320 & 505.8216 \\\\\n", "fixed\\_acidity * citric\\_acid & -169.2454 & 191.2389 & -0.8850 & 0.3762 & -544.1614 & 205.6706 \\\\\n", "fixed\\_acidity * residual\\_sugar & -2.3959 & 21.0911 & -0.1136 & 0.9096 & -43.7441 & 38.9523 \\\\\n", "fixed\\_acidity * chlorides & 151.4367 & 661.2643 & 0.2290 & 0.8189 & -1144.9447 & 1447.8180 \\\\\n", "fixed\\_acidity * free\\_sulfur\\_dioxide & -13.3943 & 9.8122 & -1.3651 & 0.1723 & -32.6306 & 5.8421 \\\\\n", "fixed\\_acidity * total\\_sulfur\\_dioxide & -12.3144 & 22.1599 & -0.5557 & 0.5784 & -55.7580 & 31.1291 \\\\\n", "fixed\\_acidity * density & -228.1023 & 1055.2972 & -0.2161 & 0.8289 & -2296.9691 & 1840.7644 \\\\\n", "fixed\\_acidity * pH & 263.5729 & 260.8085 & 1.0106 & 0.3123 & -247.7314 & 774.8773 \\\\\n", "fixed\\_acidity * sulphates & 210.1261 & 147.0152 & 1.4293 & 0.1530 & -78.0912 & 498.3434 \\\\\n", "fixed\\_acidity * alcohol & -102.4573 & 26.1357 & -3.9202 & 0.0001 & -153.6952 & -51.2193 \\\\\n", "volatile\\_acidity^2 & -1256.2263 & 1979.4050 & -0.6346 & 0.5257 & -5136.7684 & 2624.3158 \\\\\n", "volatile\\_acidity * citric\\_acid & 2503.6940 & 1629.0043 & 1.5369 & 0.1244 & -689.9019 & 5697.2899 \\\\\n", "volatile\\_acidity * residual\\_sugar & -304.5840 & 139.4970 & -2.1834 & 0.0291 & -578.0621 & -31.1060 \\\\\n", "volatile\\_acidity * chlorides & 6503.3193 & 4276.6479 & 1.5207 & 0.1284 & -1880.8729 & 14887.5116 \\\\\n", "volatile\\_acidity * free\\_sulfur\\_dioxide & 176.1465 & 65.4348 & 2.6919 & 0.0071 & 47.8643 & 304.4288 \\\\\n", "volatile\\_acidity * total\\_sulfur\\_dioxide & 541.9165 & 145.6814 & 3.7199 & 0.0002 & 256.3141 & 827.5190 \\\\\n", "volatile\\_acidity * density & -5408.0462 & 5170.0682 & -1.0460 & 0.2956 & -15543.7521 & 4727.6598 \\\\\n", "volatile\\_acidity * pH & 1591.6613 & 1576.5785 & 1.0096 & 0.3128 & -1499.1559 & 4682.4786 \\\\\n", "volatile\\_acidity * sulphates & -3066.2318 & 1347.2069 & -2.2760 & 0.0229 & -5707.3755 & -425.0882 \\\\\n", "volatile\\_acidity * alcohol & 611.6934 & 183.2814 & 3.3375 & 0.0009 & 252.3778 & 971.0089 \\\\\n", "citric\\_acid^2 & 861.9664 & 2070.4337 & 0.4163 & 0.6772 & -3197.0336 & 4920.9664 \\\\\n", "citric\\_acid * residual\\_sugar & -307.3877 & 171.8498 & -1.7887 & 0.0737 & -644.2921 & 29.5168 \\\\\n", "citric\\_acid * chlorides & -8483.4913 & 6547.3948 & -1.2957 & 0.1951 & -21319.3893 & 4352.4067 \\\\\n", "citric\\_acid * free\\_sulfur\\_dioxide & 150.5489 & 83.1030 & 1.8116 & 0.0701 & -12.3711 & 313.4689 \\\\\n", "citric\\_acid * total\\_sulfur\\_dioxide & 300.7497 & 178.4947 & 1.6849 & 0.0921 & -49.1819 & 650.6813 \\\\\n", "citric\\_acid * density & 14067.7740 & 7800.2113 & 1.8035 & 0.0714 & -1224.2191 & 29359.7672 \\\\\n", "citric\\_acid * pH & -5133.5558 & 2077.6861 & -2.4708 & 0.0135 & -9206.7738 & -1060.3378 \\\\\n", "citric\\_acid * sulphates & -2372.2746 & 1576.8448 & -1.5044 & 0.1325 & -5463.6139 & 719.0647 \\\\\n", "citric\\_acid * alcohol & 708.8006 & 236.3385 & 2.9991 & 0.0027 & 245.4687 & 1172.1325 \\\\\n", "residual\\_sugar^2 & -910.1293 & 1867.0943 & -0.4875 & 0.6260 & -4570.4908 & 2750.2323 \\\\\n", "residual\\_sugar * chlorides & 1971.4865 & 757.4887 & 2.6027 & 0.0093 & 486.4611 & 3456.5118 \\\\\n", "residual\\_sugar * free\\_sulfur\\_dioxide & -7.6328 & 5.0273 & -1.5183 & 0.1290 & -17.4886 & 2.2230 \\\\\n", "residual\\_sugar * total\\_sulfur\\_dioxide & 2.8665 & 12.6000 & 0.2275 & 0.8200 & -21.8354 & 27.5684 \\\\\n", "residual\\_sugar * density & 1429.2194 & 705.7754 & 2.0250 & 0.0429 & 45.5757 & 2812.8631 \\\\\n", "residual\\_sugar * pH & -287.7160 & 203.0709 & -1.4168 & 0.1566 & -685.8281 & 110.3961 \\\\\n", "residual\\_sugar * sulphates & -189.7045 & 168.1916 & -1.1279 & 0.2594 & -519.4371 & 140.0282 \\\\\n", "residual\\_sugar * alcohol & -18.0129 & 19.5540 & -0.9212 & 0.3570 & -56.3477 & 20.3219 \\\\\n", "chlorides^2 & 10142.2002 & 7074.9790 & 1.4335 & 0.1518 & -3728.0049 & 24012.4054 \\\\\n", "chlorides * free\\_sulfur\\_dioxide & -201.9536 & 331.1123 & -0.6099 & 0.5419 & -851.0857 & 447.1785 \\\\\n", "chlorides * total\\_sulfur\\_dioxide & 1197.2260 & 682.4747 & 1.7542 & 0.0795 & -140.7375 & 2535.1896 \\\\\n", "chlorides * density & 20265.9335 & 23064.5613 & 0.8787 & 0.3796 & -24951.1898 & 65483.0568 \\\\\n", "chlorides * pH & -12226.8108 & 6517.5063 & -1.8760 & 0.0607 & -25004.1137 & 550.4920 \\\\\n", "chlorides * sulphates & -3613.4768 & 4978.7999 & -0.7258 & 0.4680 & -13374.2092 & 6147.2555 \\\\\n", "chlorides * alcohol & 2196.0436 & 843.4185 & 2.6037 & 0.0092 & 542.5563 & 3849.5310 \\\\\n", "free\\_sulfur\\_dioxide^2 & -909.6884 & 1867.1743 & -0.4872 & 0.6261 & -4570.2067 & 2750.8299 \\\\\n", "free\\_sulfur\\_dioxide * total\\_sulfur\\_dioxide & -24.9437 & 7.6926 & -3.2426 & 0.0012 & -40.0247 & -9.8627 \\\\\n", "free\\_sulfur\\_dioxide * density & 549.2329 & 298.2374 & 1.8416 & 0.0656 & -35.4492 & 1133.9150 \\\\\n", "free\\_sulfur\\_dioxide * pH & -13.8185 & 79.9507 & -0.1728 & 0.8628 & -170.5585 & 142.9216 \\\\\n", "free\\_sulfur\\_dioxide * sulphates & 59.9612 & 69.6414 & 0.8610 & 0.3893 & -76.5679 & 196.4903 \\\\\n", "free\\_sulfur\\_dioxide * alcohol & -60.9958 & 9.6344 & -6.3310 & 0.0000 & -79.8838 & -42.1079 \\\\\n", "total\\_sulfur\\_dioxide^2 & -915.2771 & 1867.4927 & -0.4901 & 0.6241 & -4576.4197 & 2745.8655 \\\\\n", "total\\_sulfur\\_dioxide * density & 856.3795 & 643.6237 & 1.3306 & 0.1834 & -405.4183 & 2118.1773 \\\\\n", "total\\_sulfur\\_dioxide * pH & 172.8750 & 176.1471 & 0.9814 & 0.3264 & -172.4541 & 518.2041 \\\\\n", "total\\_sulfur\\_dioxide * sulphates & 233.8040 & 154.5694 & 1.5126 & 0.1304 & -69.2230 & 536.8310 \\\\\n", "total\\_sulfur\\_dioxide * alcohol & -208.7613 & 22.8714 & -9.1276 & 0.0000 & -253.5998 & -163.9229 \\\\\n", "density^2 & 2377.5655 & 18554.7387 & 0.1281 & 0.8980 & -33998.2360 & 38753.3671 \\\\\n", "density * pH & 11008.3563 & 10291.6307 & 1.0696 & 0.2848 & -9167.9622 & 31184.6748 \\\\\n", "density * sulphates & -6367.0944 & 6065.2219 & -1.0498 & 0.2939 & -18257.7123 & 5523.5234 \\\\\n", "density * alcohol & -2160.1700 & 944.6336 & -2.2868 & 0.0223 & -4012.0853 & -308.2546 \\\\\n", "pH^2 & -4031.2392 & 1137.0986 & -3.5452 & 0.0004 & -6260.4741 & -1802.0043 \\\\\n", "pH * sulphates & 1604.8538 & 1639.6859 & 0.9788 & 0.3277 & -1609.6830 & 4819.3905 \\\\\n", "pH * alcohol & 766.9861 & 249.3365 & 3.0761 & 0.0021 & 278.1721 & 1255.8001 \\\\\n", "sulphates^2 & -2563.1016 & 2045.6050 & -1.2530 & 0.2103 & -6573.4261 & 1447.2228 \\\\\n", "sulphates * alcohol & 596.8511 & 210.7614 & 2.8319 & 0.0046 & 183.6620 & 1010.0402 \\\\\n", "alcohol^2 & -1033.7653 & 1865.5885 & -0.5541 & 0.5795 & -4691.1748 & 2623.6442 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\n", "\\begin{center}\n", "\\begin{tabular}{llll}\n", "\\hline\n", "Omnibus: & 74.655 & Durbin-Watson: & 2.012 \\\\\n", "Prob(Omnibus): & 0.000 & Jarque-Bera (JB): & 119.025 \\\\\n", "Skew: & 0.141 & Prob(JB): & 0.000 \\\\\n", "Kurtosis: & 3.712 & Condition No.: & 4390541685726112 \\\\\n", "\\hline\n", "\\end{tabular}\n", "\\end{center}\n", "\\end{table}\n", "\\bigskip\n", "Notes: \\newline \n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline \n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that there are strong multicollinearity problems or that the design matrix is singular." ], "text/plain": [ "\n", "\"\"\"\n", " Results: Ordinary least squares\n", "=======================================================================================================\n", "Model: OLS Adj. R-squared: 0.306 \n", "Dependent Variable: quality AIC: 10768.5223\n", "Date: 2024-01-23 00:09 BIC: 11268.3493\n", "No. Observations: 4872 Log-Likelihood: -5307.3 \n", "Df Model: 76 F-statistic: 29.30 \n", "Df Residuals: 4795 Prob (F-statistic): 0.00 \n", "R-squared: 0.317 Scale: 0.52557 \n", "-------------------------------------------------------------------------------------------------------\n", " Coef. Std.Err. t P>|t| [0.025 0.975] \n", "-------------------------------------------------------------------------------------------------------\n", "1 874.2126 1866.6100 0.4683 0.6396 -2785.1996 4533.6248\n", "fixed_acidity 17.2438 25.1175 0.6865 0.4924 -31.9980 66.4856\n", "volatile_acidity -735.9147 164.2593 -4.4802 0.0000 -1057.9383 -413.8911\n", "citric_acid -375.2205 200.9788 -1.8670 0.0620 -769.2311 18.7900\n", "residual_sugar 2.1457 13.7859 0.1556 0.8763 -24.8809 29.1723\n", "chlorides -1219.9140 760.0849 -1.6050 0.1086 -2710.0291 270.2011\n", "free_sulfur_dioxide 33.0684 8.6300 3.8318 0.0001 16.1496 49.9873\n", "total_sulfur_dioxide 45.6122 23.6785 1.9263 0.0541 -0.8085 92.0328\n", "density -1621.7821 721.4602 -2.2479 0.0246 -3036.1752 -207.3890\n", "pH -123.6719 196.5043 -0.6294 0.5291 -508.9104 261.5667\n", "sulphates -213.6188 172.6441 -1.2373 0.2160 -552.0806 124.8429\n", "alcohol 274.6811 25.3731 10.8257 0.0000 224.9381 324.4241\n", "fixed_acidity^2 -888.0924 1860.0506 -0.4775 0.6331 -4534.6449 2758.4602\n", "fixed_acidity * volatile_acidity 213.0448 149.3410 1.4266 0.1538 -79.7320 505.8216\n", "fixed_acidity * citric_acid -169.2454 191.2389 -0.8850 0.3762 -544.1614 205.6706\n", "fixed_acidity * residual_sugar -2.3959 21.0911 -0.1136 0.9096 -43.7441 38.9523\n", "fixed_acidity * chlorides 151.4367 661.2643 0.2290 0.8189 -1144.9447 1447.8180\n", "fixed_acidity * free_sulfur_dioxide -13.3943 9.8122 -1.3651 0.1723 -32.6306 5.8421\n", "fixed_acidity * total_sulfur_dioxide -12.3144 22.1599 -0.5557 0.5784 -55.7580 31.1291\n", "fixed_acidity * density -228.1023 1055.2972 -0.2161 0.8289 -2296.9691 1840.7644\n", "fixed_acidity * pH 263.5729 260.8085 1.0106 0.3123 -247.7314 774.8773\n", "fixed_acidity * sulphates 210.1261 147.0152 1.4293 0.1530 -78.0912 498.3434\n", "fixed_acidity * alcohol -102.4573 26.1357 -3.9202 0.0001 -153.6952 -51.2193\n", "volatile_acidity^2 -1256.2263 1979.4050 -0.6346 0.5257 -5136.7684 2624.3158\n", "volatile_acidity * citric_acid 2503.6940 1629.0043 1.5369 0.1244 -689.9019 5697.2899\n", "volatile_acidity * residual_sugar -304.5840 139.4970 -2.1834 0.0291 -578.0621 -31.1060\n", "volatile_acidity * chlorides 6503.3193 4276.6479 1.5207 0.1284 -1880.8729 14887.5116\n", "volatile_acidity * free_sulfur_dioxide 176.1465 65.4348 2.6919 0.0071 47.8643 304.4288\n", "volatile_acidity * total_sulfur_dioxide 541.9165 145.6814 3.7199 0.0002 256.3141 827.5190\n", "volatile_acidity * density -5408.0462 5170.0682 -1.0460 0.2956 -15543.7521 4727.6598\n", "volatile_acidity * pH 1591.6613 1576.5785 1.0096 0.3128 -1499.1559 4682.4786\n", "volatile_acidity * sulphates -3066.2318 1347.2069 -2.2760 0.0229 -5707.3755 -425.0882\n", "volatile_acidity * alcohol 611.6934 183.2814 3.3375 0.0009 252.3778 971.0089\n", "citric_acid^2 861.9664 2070.4337 0.4163 0.6772 -3197.0336 4920.9664\n", "citric_acid * residual_sugar -307.3877 171.8498 -1.7887 0.0737 -644.2921 29.5168\n", "citric_acid * chlorides -8483.4913 6547.3948 -1.2957 0.1951 -21319.3893 4352.4067\n", "citric_acid * free_sulfur_dioxide 150.5489 83.1030 1.8116 0.0701 -12.3711 313.4689\n", "citric_acid * total_sulfur_dioxide 300.7497 178.4947 1.6849 0.0921 -49.1819 650.6813\n", "citric_acid * density 14067.7740 7800.2113 1.8035 0.0714 -1224.2191 29359.7672\n", "citric_acid * pH -5133.5558 2077.6861 -2.4708 0.0135 -9206.7738 -1060.3378\n", "citric_acid * sulphates -2372.2746 1576.8448 -1.5044 0.1325 -5463.6139 719.0647\n", "citric_acid * alcohol 708.8006 236.3385 2.9991 0.0027 245.4687 1172.1325\n", "residual_sugar^2 -910.1293 1867.0943 -0.4875 0.6260 -4570.4908 2750.2323\n", "residual_sugar * chlorides 1971.4865 757.4887 2.6027 0.0093 486.4611 3456.5118\n", "residual_sugar * free_sulfur_dioxide -7.6328 5.0273 -1.5183 0.1290 -17.4886 2.2230\n", "residual_sugar * total_sulfur_dioxide 2.8665 12.6000 0.2275 0.8200 -21.8354 27.5684\n", "residual_sugar * density 1429.2194 705.7754 2.0250 0.0429 45.5757 2812.8631\n", "residual_sugar * pH -287.7160 203.0709 -1.4168 0.1566 -685.8281 110.3961\n", "residual_sugar * sulphates -189.7045 168.1916 -1.1279 0.2594 -519.4371 140.0282\n", "residual_sugar * alcohol -18.0129 19.5540 -0.9212 0.3570 -56.3477 20.3219\n", "chlorides^2 10142.2002 7074.9790 1.4335 0.1518 -3728.0049 24012.4054\n", "chlorides * free_sulfur_dioxide -201.9536 331.1123 -0.6099 0.5419 -851.0857 447.1785\n", "chlorides * total_sulfur_dioxide 1197.2260 682.4747 1.7542 0.0795 -140.7375 2535.1896\n", "chlorides * density 20265.9335 23064.5613 0.8787 0.3796 -24951.1898 65483.0568\n", "chlorides * pH -12226.8108 6517.5063 -1.8760 0.0607 -25004.1137 550.4920\n", "chlorides * sulphates -3613.4768 4978.7999 -0.7258 0.4680 -13374.2092 6147.2555\n", "chlorides * alcohol 2196.0436 843.4185 2.6037 0.0092 542.5563 3849.5310\n", "free_sulfur_dioxide^2 -909.6884 1867.1743 -0.4872 0.6261 -4570.2067 2750.8299\n", "free_sulfur_dioxide * total_sulfur_dioxide -24.9437 7.6926 -3.2426 0.0012 -40.0247 -9.8627\n", "free_sulfur_dioxide * density 549.2329 298.2374 1.8416 0.0656 -35.4492 1133.9150\n", "free_sulfur_dioxide * pH -13.8185 79.9507 -0.1728 0.8628 -170.5585 142.9216\n", "free_sulfur_dioxide * sulphates 59.9612 69.6414 0.8610 0.3893 -76.5679 196.4903\n", "free_sulfur_dioxide * alcohol -60.9958 9.6344 -6.3310 0.0000 -79.8838 -42.1079\n", "total_sulfur_dioxide^2 -915.2771 1867.4927 -0.4901 0.6241 -4576.4197 2745.8655\n", "total_sulfur_dioxide * density 856.3795 643.6237 1.3306 0.1834 -405.4183 2118.1773\n", "total_sulfur_dioxide * pH 172.8750 176.1471 0.9814 0.3264 -172.4541 518.2041\n", "total_sulfur_dioxide * sulphates 233.8040 154.5694 1.5126 0.1304 -69.2230 536.8310\n", "total_sulfur_dioxide * alcohol -208.7613 22.8714 -9.1276 0.0000 -253.5998 -163.9229\n", "density^2 2377.5655 18554.7387 0.1281 0.8980 -33998.2360 38753.3671\n", "density * pH 11008.3563 10291.6307 1.0696 0.2848 -9167.9622 31184.6748\n", "density * sulphates -6367.0944 6065.2219 -1.0498 0.2939 -18257.7123 5523.5234\n", "density * alcohol -2160.1700 944.6336 -2.2868 0.0223 -4012.0853 -308.2546\n", "pH^2 -4031.2392 1137.0986 -3.5452 0.0004 -6260.4741 -1802.0043\n", "pH * sulphates 1604.8538 1639.6859 0.9788 0.3277 -1609.6830 4819.3905\n", "pH * alcohol 766.9861 249.3365 3.0761 0.0021 278.1721 1255.8001\n", "sulphates^2 -2563.1016 2045.6050 -1.2530 0.2103 -6573.4261 1447.2228\n", "sulphates * alcohol 596.8511 210.7614 2.8319 0.0046 183.6620 1010.0402\n", "alcohol^2 -1033.7653 1865.5885 -0.5541 0.5795 -4691.1748 2623.6442\n", "-------------------------------------------------------------------------------------------------------\n", "Omnibus: 74.655 Durbin-Watson: 2.012 \n", "Prob(Omnibus): 0.000 Jarque-Bera (JB): 119.025 \n", "Skew: 0.141 Prob(JB): 0.000 \n", "Kurtosis: 3.712 Condition No.: 4390541685726112\n", "=======================================================================================================\n", "Notes:\n", "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n", "[2] The smallest eigenvalue is 7.09e-28. This might indicate that there are strong\n", "multicollinearity problems or that the design matrix is singular.\n", "\"\"\"" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results.summary2(xname=pft.columns)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On ne garde que celles dont la [p-value](https://sdpython.github.io/doc/mlstatpy/dev/c_metric/pvalues.html) est inférieur à 0.05." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "x2 7.630773e-06\n", "x6 1.288486e-04\n", "x8 2.462679e-02\n", "x11 5.327338e-27\n", "x22 8.970958e-05\n", "x25 2.905133e-02\n", "x27 7.128466e-03\n", "x28 2.016020e-04\n", "x31 2.289041e-02\n", "x32 8.519337e-04\n", "x39 1.351547e-02\n", "x41 2.721775e-03\n", "x43 9.278805e-03\n", "x46 4.291916e-02\n", "x56 9.249665e-03\n", "x58 1.192719e-03\n", "x62 2.657965e-10\n", "x67 1.010018e-19\n", "x71 2.225201e-02\n", "x72 3.960632e-04\n", "x74 2.109039e-03\n", "x76 4.646809e-03\n", "dtype: float64" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pval = results.pvalues.copy()\n", "pval[pval <= 0.05]" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "volatile_acidity 7.630773e-06\n", "free_sulfur_dioxide 1.288486e-04\n", "density 2.462679e-02\n", "alcohol 5.327338e-27\n", "fixed_acidity * alcohol 8.970958e-05\n", "volatile_acidity * residual_sugar 2.905133e-02\n", "volatile_acidity * free_sulfur_dioxide 7.128466e-03\n", "volatile_acidity * total_sulfur_dioxide 2.016020e-04\n", "volatile_acidity * sulphates 2.289041e-02\n", "volatile_acidity * alcohol 8.519337e-04\n", "citric_acid * pH 1.351547e-02\n", "citric_acid * alcohol 2.721775e-03\n", "residual_sugar * chlorides 9.278805e-03\n", "residual_sugar * density 4.291916e-02\n", "chlorides * alcohol 9.249665e-03\n", "free_sulfur_dioxide * total_sulfur_dioxide 1.192719e-03\n", "free_sulfur_dioxide * alcohol 2.657965e-10\n", "total_sulfur_dioxide * alcohol 1.010018e-19\n", "density * alcohol 2.225201e-02\n", "pH^2 3.960632e-04\n", "pH * alcohol 2.109039e-03\n", "sulphates * alcohol 4.646809e-03\n", "dtype: float64" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pval.index = pft.columns\n", "pval[pval <= 0.05]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Le modèle fonctionne mieux mais il est plus compliqué de savoir si la contribution de l'alcool est corrélée positivement avec la qualité car l'alcool apparaît dans plus d'une variable." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }