using confusion matrix as scoring metric in cross validation in scikit learn

You could use cross_val_predict(See the scikit-learn docs) instead of cross_val_score.

instead of doing :

from sklearn.model_selection import cross_val_score
scores = cross_val_score(clf, x, y, cv=10)

you can do :

from sklearn.model_selection import cross_val_predict
from sklearn.metrics import confusion_matrix
y_pred = cross_val_predict(clf, x, y, cv=10)
conf_mat = confusion_matrix(y, y_pred)