sklearn – Cross validation with multiple scores

Now in scikit-learn: cross_validate is a new function that can evaluate a model on multiple metrics.
This feature is also available in GridSearchCV and RandomizedSearchCV (doc).
It has been merged recently in master and will be available in v0.19.

From the scikit-learn doc:

The cross_validate function differs from cross_val_score in two ways: 1. It allows specifying multiple metrics for evaluation. 2.
It returns a dict containing training scores, fit-times and score-times in addition to the test score.

The typical use case goes by:

from sklearn.svm import SVC
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_validate
iris = load_iris()
scoring = ['precision', 'recall', 'f1']
clf = SVC(kernel="linear", C=1, random_state=0)
scores = cross_validate(clf, iris.data, iris.target == 1, cv=5,
                        scoring=scoring, return_train_score=False)

See also this example.