Open-source Metaflow makes it quick and easy to build and manage real-life ML, AI, and data science projects.
Modeling
Use any Python libraries for models and business logic. Metaflow helps manage library dependencies, locally and in the cloud.
Deployment
Deploy workflows to production with a single command and integrate with other systems through events.
Versioning
Metaflow tracks and stores variables inside the flow automatically for easy experiment tracking and debugging.
Orchestration
Create robust workflows in plain Python. Develop and debug them locally, deploy to production without changes.
Compute
Leverage the cloud to execute functions at scale. Use GPUs, multiple cores, and large amounts of memory as needed.
Data
Access data from data warehouses. Metaflow flows data across steps, versioning everything on the way.
Explore with notebooks, develop with Metaflow, and test and debug locally. Results are stored and tracked automatically for easy analysis.
Break out from the confines of a laptop or a single notebook. Scale out easily to the cloud, utilizing GPUs, multiple cores, and multiple instances in parallel. Metaflow organizes the work for easy collaboration on the way.
Deploy experiments to production with a single click without changing anything in the code. Make flows react to updating data and other events automatically.
Get started easily on a laptop. When you are ready to scale, deploy the Metaflow stack on your cloud account or on-premise Kubernetes cluster. Metaflow integrates seamlessly with your existing infrastructure, security, and data governance policies.
To get a taste of Metaflow in the cloud, try Metaflow Sandbox in the browser.
Metaflow was originally developed at Netflix to address the needs of developers and data scientists who work on demanding real-life ML, AI, and data projects. Netflix open-sourced Metaflow in 2019.
Today, Metaflow is used by hundreds of companies across industries, powering diverse projects from state-of-the-art GenAI and compute vision to business-oriented data science, statistics, and operations research.
Our complex, multi-stage workflows are codified and orchestrated using Metaflow.
Our data science team believes they were able to test twice as many models in Q1 2021 as they did in all of 2020.
Metaflow helped us avoid the anti-pattern of needing to push code to find out if something works.
The team has shaved months off the time it takes to build a productionized machine learning model.
Setup the full Metaflow stack on your laptop with one click
Checkpoint long-running model training and other tasks with the new @checkpoint decorator
Configure flows freely with the new Config object
New APIs allow you to run and deploy Metaflow in notebooks and scripts
Learn about various patterns of scalable compute with Metaflow.
Train and fine-tune large language models and other generative AI models on AWS Trainium.
Build observable ML/AI systems with cards that update in real-time.
Install dependencies from PyPI as well as Conda in your Metaflow steps.
Connect to external services securely using the new @secrets decorator.
Metaflow 2.9 allows you to trigger workflows based on real-time events.
Apache Arrow and Metaflow.S3 make it easy to process data fast.
Learn how to use Metaflow for demanding GPU tasks.
Develop with Metaflow, deploy on your existing Apache Airflow servers.
