Meetup #41

Metaflow: Supercharging Our Data Scientist Productivity

Netflix's unique culture affords its data scientists an extraordinary amount of freedom. They are expected to build, deploy, and operate large machine learning workflows autonomously without the need to be significantly experienced with systems or data engineering. Metaflow, our ML framework (now open-source at metaflow.org), provides them with delightful abstractions to manage their project's lifecycle end-to-end, leveraging the strengths of the cloud: elastic compute and high-throughput storage. In this talk, we preface with our experience working alongside data scientists, present our human-centric design principles when building Machine Learning Infrastructure, and showcase how you can adopt these yourself with ease with open-source Metaflow.

Take-aways

- MLI is a stack of infrastructure. ML is just a (small) piece in this complex stack. - Metaflow reuses several existing key infrastructure pieces, cloud-first and is targeted at data-scientist productivity. - Philosophy behind Metaflow and basic how-to(s).

In this episode

Ravi Kiran Chirravuri

Ravi Kiran Chirravuri

Software Engineer, Netflix

Ravi is an individual contributor on the Machine Learning Infrastructure (MLI) team at Netflix. With almost a decade of industry experience, he has been building large scale systems focusing on performance, simplified user journeys and intuitive APIs in MLI and previously Search Indexing and Tensorflow at Google.

Twitter

LinkedIn

Demetrios Brinkmann

Demetrios Brinkmann

Host

Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.