Continuous Live Monitoring of Machine Learning ModelsThe usual steps of developing a machine learning model are: training on a training set, tuning on a validation set and evaluating the performance on the test set. If the model is particularly good, it will be deployed to serve predictions in a production system.
In this talk we present what happens to a machine learning model after it is deployed in production at Zalando. We focus on the precautions that ensure that a model's predictions always stay at the high quality we expect.
In this talk, we present our solution, which consists of monitoring the similarity between the distributions of features in the live traffic and the test set on which the model was evaluated.
Vorkenntnisse
Some basic knowledge on how Machine Learning systems work
Lernziele
We want to make the audience aware of the aspects of monitoring machine learning applications that are running production and show our solutions for that.
//
Patrick Baier
arbeitet als Machine Learning Engineer im Payments Team von Zalando in Berlin. Schwerpunkt seiner Arbeit ist die Entwicklung statistischer Modelle zur Betrugserkennung. Davor arbeitete er als wissenschaftlicher Mitarbeiter an der Universität Stuttgart an probabilistischen Modellen zur Energieeffizienz in mobilen Systemen, wo er 2015 erfolgreich seine Dissertation abschloss.
//
Henning Esser
entwickelt als Teil von Zalando Payments das Machine-Learning-basierte Betrugserkennungssystem mit. Davor schloss er sein Studium der Mathematik an der TU Kaiserslautern mit einem Diplom ab.