Tech Talk - Monitoring for Machine Learning Analytics

Duration: 15 Minutes

Matt Mahowald Lead Data Science and Garrett Long VP Sales wanted to share a new way to think about a “single pane of glass” for monitoring your analytics.  Any machine learning or analytics platform that scales will require the ability to see what’s happening, real time, in the production analytics.  But it’s not enough to worry only about the model itself.  In our talk, we will take you through what an analytic operations center dashboard might look like including monitoring:

  • The infrastructure - records per second, CPU, memory and other critical factors that may impact performance (And costs)
  • The model - monitoring for values exceeding thresholds and scores out of range (e.g. drift)
  • The data - including checking if data has drifted away from expected ranges

We will use FastScore for model execution / monitoring, AWS for the compute infrastructure, and Grafana as the dashboard in this tech talk.