Getting Fit Reports#
After SelfSupervised
or Supervised
fits are done, we display details about the fit process through
the method j.report
.
In this page, we’ll take a look on the information that is available.
Verbose Levels#
There’s 2 levels of verbose that set the amount of information that is retrieved.
The default value for default on the setup/fit method is verbose=1
, while using the j.report
the default value is verbose=2
.
And verbose=0
will just not return any information.
Some information is only displayed when accessing the dictionary returned on j.report(return_report=True)
.
Here are the information that is available with the reporting system:
verbose = 1#
Always printed when report method is used.
Loading from checkpoint: Specifies which epoch the model had the best performance and is used as the final model for inference.
Model Evaluation [Supervised only]: Metrics on the test set.
MSE and MAE for regression,
'quantile'=0.5
is used whenlabel={task: 'quantile_regression'}
.Scikit-learn’s classification_report (precision, recall and f1) when
label={task: 'classification'}
orlabel={task: 'metric_classification'}
.
verbose = 2#
Contains all content of verbose=1
plus:
Model Training: Plots the loss graph of the training. When
return_report=True
, returns the values of the loss for the training set and validation set for each epoch.
All content below is available only when return_report=True
Auto lr finder: If
learning_rate=0
, then we’ll try and find the best appropriate learning rate.
All content below is available only on Supervised models.
Metrics Train/Metrics Validation: The same metrics calculated on Model Evaluation but on the training set and validation set respectively.
Baseline Model: The same metrics of “Model Evaluation” on the test set, but using a Baseline Model (please check sklearn.dummy models). For regression cases, we evaluate the mean and the median as a baseline. For classification cases, we use
stratified
,uniform
andmost_frequent
models as baseline.Optimal Thresholds: List the probability thresholds that maximize
true positive rate - false negative rate
for each class. Since it’s calculated in a OneVsAll manner, the probabilities don’t sum up to one.