Scale and Host LLMs on AWS — Large Language Models (LLMs) and Generative AI are accelerating Machine Learning growth across various industries. With LLMs the scope for Machine Learning has increased to incredible heights, but has also been accompanied with a new set of challenges. The size of LLMs lead to difficult problems in both the Training…

Harness SageMaker Pipelines With Batch Inference — MLOPs continues to be a popular topic in the Machine Learning world and for good reason. As ML continues to mature it’s become crucial that we apply Software Engineering best practices to make reusable, end to end workflows tailored for Data Science. A tool we explored in my previous article…

Applying MLOps best practices to advanced serving options — MLOps is an essential practice to productionizing your Machine Learning workflows. With MLOps you can establish workflows that are catered for the ML lifecycle. These make it easier to centrally maintain resources, update/track models, and in general simplify the process as your ML experimentation scales up. A key MLOps tool…

Infrastructure as Code With Terraform — Infrastructure as Code (IaC) is an essential concept to optimize and take your resources and infrastructure to production. IaC is an age old DevOps/Software practice and has a few key benefits: Resources are maintained centrally via code, which in turn optimizes the speed and collaboration required to take your architecture…

Test TensorFlow ResNet50 on SageMaker Real-Time Endpoints — In the past I’ve written extensively about the importance of load testing your Machine Learning models before deploying them into production. When it comes to real-time inference use-cases in specific it’s essential to ensure your solution meets your target latency and throughput. We’ve also explored how we can use the…

Utilize Locust to Distribute Traffic Weight Across Models — Productionizing Machine Learning models is a complicated practice. There’s a lot of iteration around different model parameters, hardware configurations, traffic patterns that you will have to test to try to finalize a production grade deployment. …

SageMaker Distributed Training Data Parallel — As Machine Learning continues to evolve we’re seeing larger models with more and more parameters. At the same time we also see incredibly large datasets, at the end of the day any model is only as good as the data that it’s trained on. Working with large models and datasets…

Organizing and Tracking your ML Training and Experimentation — MLOps is a growing and crucial domain in the field of Machine Learning. I’ve been in Data Science and Medium for a few years now and have never seen such an implosion of tools in this area ranging from MLFlow to SageMaker Pipelines. A key part of MLOps is experimentation…

Rapid production-grade deployment of Transformers models — A constant theme in my articles has been the deployment of your Machine Learning models. As Machine Learning grows in popularity so has the range of model deployment options for users. …