Getting Fit Reports#

After SelfSupervised or Supervised fits are done, we display details about the fit process through the method j.report. In this page, we’ll take a look on the information that is available.

Verbose Levels#

There’s 2 levels of verbose that set the amount of information that is retrieved. The default value for default on the setup/fit method is verbose=1, while using the j.report the default value is verbose=2. And verbose=0 will just not return any information.

Some information is only displayed when accessing the dictionary returned on j.report(return_report=True). Here are the information that is available with the reporting system:

verbose = 1#

Always printed when report method is used.

  • Loading from checkpoint: Specifies which epoch the model had the best performance and is used as the final model for inference.

  • Model Evaluation [Supervised only]: Metrics on the test set.

    • MSE and MAE for regression, 'quantile'=0.5 is used when label={task: 'quantile_regression'}.

    • Scikit-learn’s classification_report (precision, recall and f1) when label={task: 'classification'} or label={task: 'metric_classification'}.

verbose = 2#

Contains all content of verbose=1 plus:

  • Model Training: Plots the loss graph of the training. When return_report=True, returns the values of the loss for the training set and validation set for each epoch.

All content below is available only when return_report=True

  • Auto lr finder: If learning_rate=0, then we’ll try and find the best appropriate learning rate.

All content below is available only on Supervised models.

  • Metrics Train/Metrics Validation: The same metrics calculated on Model Evaluation but on the training set and validation set respectively.

  • Baseline Model: The same metrics of “Model Evaluation” on the test set, but using a Baseline Model (please check sklearn.dummy models). For regression cases, we evaluate the mean and the median as a baseline. For classification cases, we use stratified, uniform and most_frequent models as baseline.

  • Optimal Thresholds: List the probability thresholds that maximize true positive rate - false negative rate for each class. Since it’s calculated in a OneVsAll manner, the probabilities don’t sum up to one.