mlinsights.metrics#
- mlinsights.metrics.correlations.non_linear_correlations(df, model, draws=5, minmax=False)[source]#
Computes non linear correlations.
- Parameters:
df –
pandas.DataFrame
ornumpy.array
model – machine learned model used to compute the correlations
draws – number of tries for bootstrap, the correlation is the average of the results obtained at each draw
minmax – if True, returns three matrices correlations, min, max, only the correlation matrix if False
- Returns:
see parameter minmax
If variables are centered, , it becomes:
If rescaled, , then it becomes . Let’s assume we try to find a coefficient such as minimizes the standard deviation of noise :
It is like if coefficient comes from a a linear regression which minimizes . If variable , are centered and rescaled: . We extend that definition to function of parameter defined as: . is not linear anymore. Let’s assume parameter minimizes quantity . Then and we choose such as . Let’s define a non linear correlation bounded by as:
We can verify that this value is in interval`:math:[0,1]`. That also means that there is no negative correlation. is a machine learned model and most of them usually overfit the data. The database is split into two parts, one is used to train the model, the other one to compute the correlation. The same split are used for every coefficient. The returned matrix is not necessarily symmetric.
Compute non linear correlations
The following example compute non linear correlations on Iris datasets based on a RandomForestRegressor model.
<<<
import pandas from sklearn import datasets from sklearn.ensemble import RandomForestRegressor from mlinsights.metrics import non_linear_correlations iris = datasets.load_iris() X = iris.data[:, :4] df = pandas.DataFrame(X) df.columns = ["X1", "X2", "X3", "X4"] cor = non_linear_correlations(df, RandomForestRegressor()) print(cor)
>>>
X1 X2 X3 X4 X1 0.998697 0.086262 0.845006 0.774358 X2 0.000000 0.996680 0.223144 0.193046 X3 0.867133 0.561329 0.998474 0.952098 X4 0.764144 0.681532 0.969139 0.999661