FairnessDrift.get_report_reduced

FairnessDrift.get_report_reduced()

Restituisce un report comprensivo dei risultati del drift delle metriche di fairness con warning assoluti e relativi.

Parameters:

None (metodo basato sulle metriche correnti e le soglie già impostate durante l'inizializzazione ed eventualmente le metriche storiche)

Returns:

Restituisce il report con le metriche e i gruppi di fairness

Return type:

pd.DataFrame

Note

  • Se lanciato dopo get_absolute_reduced() conterrà anche i warning assoluti sulle performance delle metriche correnti.

  • Se lanciato dopo get_relative_reduced() conterrà anche i warning relativi al drift sulle performance delle metriche correnti rispetto a quelle storiche.

  • Il report contiene per ogni gruppo di fairness, oltre al valore della metrica, anche la percentuale di campioni appartenenti al gruppo rispetto al numero totale di campioni.

  • Se l’attributo return_prop_true dell’__init__ della classe FairnessMeasures è True il dizionario delle metriche di fairness conterrà anche la percentuale di 1 (veri positivi) all’interno del gruppo in un contesto di classificazione binaria, e di conseguenza il report conterrà anche la colonna proportion_1_curr, e proportion_1_stor se chiamato dopo get_relative_reduced().

Dati utilizzati per gli esempi

>>> df_esempio_corr
feature_num feature_cat target_clf pred_clf target_reg pred_reg
0 2 B 0 0 7.652628 -1.427613
1 3 A 1 0 17.712800 3.589763
2 4 C 0 0 17.682912 32.339399
3 5 B 0 0 22.671351 20.413588
4 6 A 0 1 31.209811 31.885093
5 7 C 1 0 25.433599 11.186117
6 8 D 0 1 31.375411 25.931584
7 9 E 0 1 42.188562 43.297788
8 10 C 0 1 44.935844 33.425909
9 11 A 1 0 56.571237 60.328217 
>>> df_esempio_stor
feature_num feature_cat target_clf pred_clf target_reg pred_reg
0 1 A 0 0 -1.131079 13.795709
1 2 B 1 1 5.259965 12.331217
2 3 A 0 0 12.151730 -6.433172
3 4 C 0 0 15.114249 1.408011
4 5 B 0 1 21.146841 17.845778
5 6 A 1 1 29.831444 14.678544
6 7 C 1 0 29.835704 41.836306
7 8 A 1 0 45.712137 27.485945
8 9 B 1 0 41.951110 44.644955
9 10 C 1 0 57.347082 52.882838

Esempio 1:

>>> FM = FairnessMeasures(approach_type="supervised", model_type="classification")
>>> fair_dict_corr = FM.compute_metrics(
...     df_esempio_corr.pred_clf,
...     df_esempio_corr,
...     target=df_esempio_corr.target_clf,
...     fair_feat=['feature_cat']
... )
>>> fair_dict_stor = FM.compute_metrics(
...     df_esempio_stor.pred_clf,
...     df_esempio_stor,
...     target=df_esempio_stor.target_clf,
...     fair_feat=['feature_cat']
... )
>>> from model_monitoring.fairness_drift import FairnessDrift
>>> FD = FairnessDrift(fair_dict_corr)
>>> FD.get_report_reduced()
vars groups curr_perc_label proportion_1_curr equal_opportunity_difference_curr_perf average_odds_difference_curr_perf statistical_parity_difference_curr_perf disparate_impact_ratio_curr_perf predictive_parity_difference_curr_perf
1 feature_cat A 0.3 0.666667 0.0 0.25 -0.095238 0.777778 0.0
2 feature_cat C 0.3 0.333333 0.0 -0.05 -0.095238 0.777778 0.0
0 feature_cat B 0.2 0.000000 0.0 -0.40 -0.500000 0.000000 0.0
3 feature_cat D 0.1 0.000000 0.0 0.25 0.666667 3.000000 0.0
4 feature_cat E 0.1 0.000000 0.0 0.25 0.666667 3.000000 0.0

Esempio 2: Uso con get_absolute_reduced

>>> FD.get_absolute_reduced()
>>> FD.get_report_reduced()
vars groups curr_perc_label proportion_1_curr equal_opportunity_difference_curr_perf average_odds_difference_curr_perf statistical_parity_difference_curr_perf disparate_impact_ratio_curr_perf predictive_parity_difference_curr_perf absolute_warning
1 feature_cat A 0.3 0.666667 0.0 0.25 -0.095238 0.777778 0.0 Yellow Alert for average_odds_difference, disparate_impact_ratio
2 feature_cat C 0.3 0.333333 0.0 -0.05 -0.095238 0.777778 0.0 Yellow Alert for disparate_impact_ratio
0 feature_cat B 0.2 0.000000 0.0 -0.40 -0.500000 0.000000 0.0 Red Alert for statistical_parity_difference, disparate_impact_ratio
3 feature_cat D 0.1 0.000000 0.0 0.25 0.666667 3.000000 0.0 Red Alert for statistical_parity_difference, disparate_impact_ratio, Yellow Alert for average_odds_difference
4 feature_cat E 0.1 0.000000 0.0 0.25 0.666667 3.000000 0.0 Red Alert for statistical_parity_difference, disparate_impact_ratio, Yellow Alert for average_odds_difference

Esempio 3: Uso con get_relative_reduced

>>> FD.get_relative_reduced(fair_dict_stor)
>>> FD.get_report_reduced()
vars groups curr_perc_label proportion_1_curr equal_opportunity_difference_curr_perf statistical_parity_difference_curr_perf predictive_parity_difference_curr_perf average_odds_difference_curr_perf disparate_impact_ratio_curr_perf absolute_warning stor_perc_label proportion_1_stor equal_opportunity_difference_stor_perf drift_perc_equal_opportunity_difference statistical_parity_difference_stor_perf drift_perc_statistical_parity_difference predictive_parity_difference_stor_perf drift_perc_predictive_parity_difference average_odds_difference_stor_perf drift_perc_average_odds_difference disparate_impact_ratio_stor_perf drift_perc_disparate_impact_ratio relative_warning
0 feature_cat A 0.3 0.666667 0.0 -0.095238 0.0 0.25 0.777778 Yellow Alert for average_odds_difference, disparate_impact_ratio 0.4 0.500000 0.25 -100.0 -0.083333 14.285714 0.500000 -100.0 -0.125000 100.0 0.750000 -11.111111 Red Alert for average_odds_difference
1 feature_cat C 0.3 0.333333 0.0 -0.095238 0.0 -0.05 0.777778 Yellow Alert for disparate_impact_ratio 0.3 0.666667 -0.50 -100.0 -0.428571 -77.777778 -0.666667 -100.0 -0.416667 -88.0 0.000000 -77.777778
2 feature_cat B 0.2 0.000000 0.0 -0.500000 0.0 -0.40 0.000000 Red Alert for statistical_parity_difference, average_odds_difference, disparate_impact_ratio 0.3 0.666667 0.25 -100.0 0.523810 -4.545455 -0.500000 -100.0 0.625000 -36.0 4.666667 -72.727273