Apache Spark and Kubernetes have been established as de facto standards for data processing and container orchestration respectively. This talk will cover how these technologies can be integrated under the orchestration of Kubeflow. (Kubeflow has been emerging as a platform to make ML workflows easy to work with and deploy.) All codes are available here: https://github.com/sbakiu/kubeflow-spark/blob/main/kubeflow_pipeline.py
In this episode
Freelance ML Engineer, Freelance
Sadik is a Freelance ML Engineer focused on creating production-grade ML workflows. Since the early beginning of his career, more than a decade ago, he was fascinated by Data and Information management systems and has been working in this field ever since. Sadik also writes occasionally about technology topics.
Demetrios is one of the main organizers of the MLOps community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveller who taught English as a second language to see the world and learn about new cultures. Demetrios fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and ML. Since diving into the nitty-gritty of Machine Learning Operations he felt a strong calling to explore the ethical issues surrounding ML. When he is not conducting interviews you can find him making stone stacking with his daughter in the woods or playing the ukulele by the campfire.