FRI Piškot Seminar series continues. In the summer semester the seminars will be held on Thursdays at 14.15 in lecture hall 22.

On Thursday, 9 March 2017, Jaka Demšar, the receiver of the Faculty level Prešeren Award for 2016, will present his research on machine learning.

Detecting concept drift using model explanation

Abstract

In the field of incremental learning, detecting and understanding changes (concept drift) is crucial for success of classification algorithms. To gain insight into the classification predictions, we employed incremental explanation techniques and used the resulting stream of explanations as a basis for a novel concept drift detection method. The majority of current change detection algorithms rely on either monitoring of classifier's performance or comparisons of data distributions where minimal attention is paid to the inner working of the algorithm itself (its comprehensibility). We devised a novel concept drift detection algorithm that relies on various techniques of measuring dissimilarities in vectors of the stream of explanations. The developed methods were evaluated on a new synthetic data stream generator where we control for common factors in incremental environments to confirm its use feasibility. The result is a robust yet sensitive generalized concept drift detector (it can act as a wrapper for an arbitrary classifier), which also diminishes the black-box nature of the learning process, especially with regards to understanding changes detected.