mlstatpy.ml.neural_tree#
- class mlstatpy.ml.neural_tree.BaseNeuralTreeNet(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)[source][source]#
Classifier or regressor following scikit-learn API.
- Paramètres:
estimator – instance of
NeuralTreeNet
.X – training set
y – training labels
optimizer – optimizer, by default, it is
SGDOptimizer
.max_iter – number maximum of iterations
early_th – early stopping threshold
verbose – more verbose
lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)
lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)
l1 – L1 regularization if optimizer is None (unused otherwise)
l2 – L2 regularization if optimizer is None (unused otherwise)
momentum – used if optimizer is None
- decision_function(X)[source][source]#
Returns the classification probabilities.
- Paramètres:
X – inputs
- Renvoie:
probabilities
- fit(X, y, sample_weights=None)[source][source]#
Trains the estimator.
- Paramètres:
X – input features
y – expected classes (binary)
sample_weights – sample weights
- Renvoie:
self
- static onnx_shape_calculator()[source][source]#
Shape calculator when converting this model into ONNX. See sklearn-onnx.
- set_fit_request(*, sample_weights: bool | None | str = '$UNCHANGED$') BaseNeuralTreeNet [source]#
Request metadata passed to the
fit
method.Note that this method is only relevant if
enable_metadata_routing=True
(seesklearn.set_config()
). Please see User Guide on how the routing mechanism works.The options for each parameter are:
True
: metadata is requested, and passed tofit
if provided. The request is ignored if metadata is not provided.False
: metadata is not requested and the meta-estimator will not pass it tofit
.None
: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str
: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED
) retains the existing request. This allows you to change the request for some parameters and not others.Ajouté dans la version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a
Pipeline
. Otherwise it has no effect.
- class mlstatpy.ml.neural_tree.NeuralTreeNet(dim, empty=True)[source][source]#
Node ensemble.
- Paramètres:
dim – space dimension
empty – empty network, other adds an identity node
<<<
import numpy from mlstatpy.ml.neural_tree import NeuralTreeNode, NeuralTreeNet w1 = numpy.array([-0.5, 0.8, -0.6]) neu = NeuralTreeNode(w1[1:], bias=w1[0], activation="sigmoid") net = NeuralTreeNet(2, empty=True) net.append(neu, numpy.arange(2)) ide = NeuralTreeNode(numpy.array([1.0]), bias=numpy.array([0.0]), activation="identity") net.append(ide, numpy.arange(2, 3)) X = numpy.abs(numpy.random.randn(10, 2)) pred = net.predict(X) print(pred)
>>>
/home/xadupre/github/mlstatpy/mlstatpy/ml/_neural_tree_node.py:184: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.) self.coef[0] = bias [[2.004 0.783 0.653 0.653] [0.635 0.857 0.376 0.376] [1.023 0.894 0.446 0.446] [0.703 0.856 0.389 0.389] [0.545 0.853 0.36 0.36 ] [1.287 1.173 0.457 0.457] [0.822 0.703 0.434 0.434] [0.382 1.022 0.308 0.308] [0.44 1.75 0.232 0.232] [1.678 0.502 0.632 0.632]]
- append(node, inputs)[source][source]#
Appends a node into the graph.
- Paramètres:
node – node to add
inputs – index of input nodes
- static create_from_tree(tree, k=1.0, arch='one')[source][source]#
Creates a
NeuralTreeNet
instance from a DecisionTreeClassifier- Paramètres:
tree – DecisionTreeClassifier
k – slant of the sigmoïd
arch – architecture, see below
- Renvoie:
The function only works for binary problems. Available architecture:
“one”: the method adds nodes with one output, there is no soecific definition of layers,
“compact”: the adds two nodes, the first computes the threshold, the second one computes the leaves output, a final node merges all outputs into one
See notebook Un arbre de décision en réseaux de neurones for examples.
- gradient_backward(graddx, X, inputs=False, cache=None)[source][source]#
Computes the gradient in X.
- Paramètres:
graddx – existing gradient against the inputs
X – computes the gradient in X
inputs – if False, derivative against the coefficients, otherwise against the inputs.
cache – cache intermediate results to avoid more computation
- Renvoie:
gradient
- property shape#
Returns the shape of the coefficients.
- to_dot(X=None)[source][source]#
Exports the neural network into dot.
- Paramètres:
X – input as an example
- property training_weights#
Returns the weights.
- class mlstatpy.ml.neural_tree.NeuralTreeNetClassifier(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)[source][source]#
Classifier following scikit-learn API.
- Paramètres:
estimator – instance of
NeuralTreeNet
.optimizer – optimizer, by default, it is
SGDOptimizer
.max_iter – number maximum of iterations
early_th – early stopping threshold
verbose – more verbose
lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)
lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)
l1 – L1 regularization if optimizer is None (unused otherwise)
l2 – L2 regularization if optimizer is None (unused otherwise)
momentum – used if optimizer is None
- predict_proba(X)[source][source]#
Returns the classification probabilities.
- Paramètres:
X – inputs
- Renvoie:
probabilities
- set_fit_request(*, sample_weights: bool | None | str = '$UNCHANGED$') NeuralTreeNetClassifier [source]#
Request metadata passed to the
fit
method.Note that this method is only relevant if
enable_metadata_routing=True
(seesklearn.set_config()
). Please see User Guide on how the routing mechanism works.The options for each parameter are:
True
: metadata is requested, and passed tofit
if provided. The request is ignored if metadata is not provided.False
: metadata is not requested and the meta-estimator will not pass it tofit
.None
: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str
: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED
) retains the existing request. This allows you to change the request for some parameters and not others.Ajouté dans la version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a
Pipeline
. Otherwise it has no effect.
- set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') NeuralTreeNetClassifier [source]#
Request metadata passed to the
score
method.Note that this method is only relevant if
enable_metadata_routing=True
(seesklearn.set_config()
). Please see User Guide on how the routing mechanism works.The options for each parameter are:
True
: metadata is requested, and passed toscore
if provided. The request is ignored if metadata is not provided.False
: metadata is not requested and the meta-estimator will not pass it toscore
.None
: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str
: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED
) retains the existing request. This allows you to change the request for some parameters and not others.Ajouté dans la version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a
Pipeline
. Otherwise it has no effect.
- class mlstatpy.ml.neural_tree.NeuralTreeNetRegressor(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)[source][source]#
Regressor following scikit-learn API.
- Paramètres:
estimator – instance of
NeuralTreeNet
.optimizer – optimizer, by default, it is
SGDOptimizer
.max_iter – number maximum of iterations
early_th – early stopping threshold
verbose – more verbose
lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)
lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)
l1 – L1 regularization if optimizer is None (unused otherwise)
l2 – L2 regularization if optimizer is None (unused otherwise)
momentum – used if optimizer is None
- set_fit_request(*, sample_weights: bool | None | str = '$UNCHANGED$') NeuralTreeNetRegressor [source]#
Request metadata passed to the
fit
method.Note that this method is only relevant if
enable_metadata_routing=True
(seesklearn.set_config()
). Please see User Guide on how the routing mechanism works.The options for each parameter are:
True
: metadata is requested, and passed tofit
if provided. The request is ignored if metadata is not provided.False
: metadata is not requested and the meta-estimator will not pass it tofit
.None
: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str
: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED
) retains the existing request. This allows you to change the request for some parameters and not others.Ajouté dans la version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a
Pipeline
. Otherwise it has no effect.
- set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') NeuralTreeNetRegressor [source]#
Request metadata passed to the
score
method.Note that this method is only relevant if
enable_metadata_routing=True
(seesklearn.set_config()
). Please see User Guide on how the routing mechanism works.The options for each parameter are:
True
: metadata is requested, and passed toscore
if provided. The request is ignored if metadata is not provided.False
: metadata is not requested and the meta-estimator will not pass it toscore
.None
: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str
: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED
) retains the existing request. This allows you to change the request for some parameters and not others.Ajouté dans la version 1.3.
Note
This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a
Pipeline
. Otherwise it has no effect.
- mlstatpy.ml.neural_tree.label_class_to_softmax_output(y_label)[source][source]#
Converts a binary class label into a matrix with two columns of probabilities.
<<<
import numpy from mlstatpy.ml.neural_tree import label_class_to_softmax_output y_label = numpy.array([0, 1, 0, 0]) soft_y = label_class_to_softmax_output(y_label) print(soft_y)
>>>
[[1. 0.] [0. 1.] [1. 0.] [1. 0.]]
- class mlstatpy.ml._neural_tree_node.NeuralTreeNode(weights, bias=None, activation='sigmoid', nodeid=-1, tag=None)[source][source]#
One node in a neural network.
- Paramètres:
weights – weights
bias – bias, if None, draws a random number
activation – activation function
nodeid – node id
tag – unused but to add information on how this node was created
- property bias#
Returns the weights.
- fill_cache(X)[source][source]#
Creates a cache with intermediate results.
lX
is the results before the activation function,aX
is the results after the activation function, the prediction.
- static get_activation_dloss_function(activation)[source][source]#
Returns the derivative of the default loss function based on the activation function. It returns a function df(x,y)/dw, df(w)/dw where w are the weights.
- static get_activation_function(activation)[source][source]#
Returns the activation function. It returns a function y=f(x).
- static get_activation_gradient_function(activation)[source][source]#
Returns the activation function. It returns a function y=f”(x). About the sigmoid:
\[\begin{split}\begin{array}{rcl} f(x) &=& \frac{1}{1 + e^{-x}} \\ f'(x) &=& \frac{e^{-x}}{(1 + e^{-x})^2} = f(x)(1-f(x)) \end{array}\end{split}\]
- static get_activation_loss_function(activation)[source][source]#
Returns a default loss function based on the activation function. It returns two functions g=loss(x,y).
- gradient_backward(graddx, X, inputs=False, cache=None)[source][source]#
Computes the gradients at point X.
- Paramètres:
graddx – existing gradient against the inputs
X – computes the gradient in X
inputs – if False, derivative against the coefficients, otherwise against the inputs.
cache – cache intermediate results
- Renvoie:
gradient
- property input_weights#
Returns the weights.
- property ndim#
Returns the input dimension.
- property ndim_out#
Returns the output dimension.
- property training_weights#
Returns the weights stored in the neuron.
- class mlstatpy.ml._neural_tree_api._TrainingAPI[source][source]#
Declaration of function needed to train a model.
- fit(X, y, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)[source][source]#
Fits a neuron.
- Paramètres:
X – training set
y – training labels
optimizer – optimizer, by default, it is
SGDOptimizer
.max_iter – number maximum of iterations
early_th – early stopping threshold
verbose – more verbose
lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)
lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)
l1 – L1 regularization if optimizer is None (unused otherwise)
l2 – L2 regularization if optimizer is None (unused otherwise)
momentum – used if optimizer is None
- Renvoie:
self
- gradient(X, y, inputs=False)[source][source]#
Computes the gradient in X knowing the expected value y.
- Paramètres:
X – computes the gradient in X
y – expected values
inputs – if False, derivative against the coefficients, otherwise against the inputs.
- Renvoie:
gradient
- gradient_backward(graddx, X, inputs=False, cache=None)[source][source]#
Computes the gradient in X.
- Paramètres:
graddx – existing gradient against the outputs
X – computes the gradient in X
inputs – if False, derivative against the coefficients, otherwise against the inputs.
cache – cache intermediate results to avoid more computation
- Renvoie:
gradient
- property training_weights#
Returns the weights.