# Machine learning evaluation metrics

## Metrics

Machine learning evaluation metrics, implemented in Python, R, Haskell, and MATLAB / Octave.

**Metrics** provides implementations of various supervised machine learning evaluation metrics in the following languages:

**Python**`easy_install ml_metrics`

**R**`install.packages("Metrics")`

from the R prompt**Haskell**`cabal install Metrics`

**MATLAB / Octave**(clone the repo & run setup from the MATLAB command line)

For more detailed installation instructions, see the README for each implementation.

## EVALUATION METRICS

Evaluation Metric | Python | R | Haskell | MATLAB / Octave |

Absolute Error (AE) | ✓ | ✓ | ✓ | ✓ |

Average Precision at K (APK, [email protected]) | ✓ | ✓ | ✓ | ✓ |

Area Under the ROC (AUC) | ✓ | ✓ | ✓ | ✓ |

Classification Error (CE) | ✓ | ✓ | ✓ | ✓ |

F1 Score (F1) | ✓ | |||

Gini | ✓ | |||

Levenshtein | ✓ | ✓ | ✓ | |

Log Loss (LL) | ✓ | ✓ | ✓ | ✓ |

Mean Log Loss (LogLoss) | ✓ | ✓ | ✓ | ✓ |

Mean Absolute Error (MAE) | ✓ | ✓ | ✓ | ✓ |

Mean Average Precision at K (MAPK, [email protected]) | ✓ | ✓ | ✓ | ✓ |

Mean Quadratic Weighted Kappa | ✓ | ✓ | ✓ | |

Mean Squared Error (MSE) | ✓ | ✓ | ✓ | ✓ |

Mean Squared Log Error (MSLE) | ✓ | ✓ | ✓ | ✓ |

Normalized Gini | ✓ | |||

Quadratic Weighted Kappa | ✓ | ✓ | ✓ | |

Relative Absolute Error (RAE) | ✓ | |||

Root Mean Squared Error (RMSE) | ✓ | ✓ | ✓ | ✓ |

Relative Squared Error (RSE) | ✓ | |||

Root Relative Squared Error (RRSE) | ✓ | |||

Root Mean Squared Log Error (RMSLE) | ✓ | ✓ | ✓ | ✓ |

Squared Error (SE) | ✓ | ✓ | ✓ | ✓ |

Squared Log Error (SLE) | ✓ | ✓ | ✓ | ✓ |

## TO IMPLEMENT

- F1 score
- Multiclass log loss
- Lift
- Average Precision for binary classification
- precision / recall break-even point
- cross-entropy
- True Pos / False Pos / True Neg / False Neg rates
- precision / recall / sensitivity / specificity
- mutual information

## HIGHER LEVEL TRANSFORMATIONS TO HANDLE

- GroupBy / Reduce
- Weight individual samples or groups

## PROPERTIES METRICS CAN HAVE

(Nonexhaustive and to be added in the future)

- Min or Max (optimize through minimization or maximization)
- Binary Classification
- Scores predicted class labels
- Scores predicted ranking (most likely to least likely for being in one class)
- Scores predicted probabilities

- Multiclass Classification
- Scores predicted class labels
- Scores predicted probabilities

- Regression
- Discrete Rater Comparison (confusion matrix)