A curated list of Metaflow extensions, plugins, integrations, and resources.
Metaflow is a human-friendly Python/R framework for real-life ML, AI, and data science. Originally built at Netflix and open-sourced in 2019.
- Core Infrastructure
- Infrastructure & IaC
- Dependency Management
- Extension Mechanism & Templates
- Orchestration & Scheduling
- Distributed Compute & Training
- Model & Artifact Management
- Observability & Monitoring
- Cards & Visualization
- Third-Party Integrations
- Developer Tooling
- Examples & Tutorials
- metaflow - Core framework with built-in
@kubernetes,@batch, Argo, Airflow, and Step Functions support. - metaflow-local-service - Track Metaflow runs anywhere without a database — starts on demand, stops when idle.
- metaflow-serverless - Serverless Metaflow metadata service — free-tier Postgres, zero setup.
- metaflow-service - Metadata tracking REST API and UI backend.
- metaflow-ui - React web UI for real-time run monitoring with a plugin system.
- local_metaflow_deployment - Docker-based local stack: metadata service, PostgreSQL, and UI.
- metaflow-docker-deployment - Community Docker Compose: metadata service + UI + MinIO + PostgreSQL.
- metaflow-tools - CloudFormation, Terraform (AWS/Azure/GCP/Nebius), and Helm charts for Kubernetes.
- metaflow-with-airflow-minio - Metaflow + Airflow + MinIO on Minikube.
- terraform-aws-metaflow - Official Terraform module for AWS (Batch, S3, RDS, Step Functions, ECS, optional EKS+Argo).
- metaflow-nflx-extensions - Netflix's enhanced
@conda/@pypi: named environments, mixed conda+pip, and faster resolving via micromamba. Requires Metaflow ≥ 2.8.3. - metaflow_extensions - Adds
@pipand apreinstallshell-hook for system-level deps on remote nodes.⚠️ Built against Metaflow 2.7.x; verify compatibility.
- config-examples - Examples using Metaflow's
Configobject for flow-level configuration. - custom-decorator-examples - Examples of user-defined decorators and mutators, including
@memoizeandBaseFlowpatterns. - metaflow-extensions-template - Official template for building
metaflow_extensionsnamespace packages. - metaflow-orchestrator-kit - Compliance test suite and scaffold for building correct Metaflow orchestrators.
- metaflow-dagster - Dagster scheduling, observability, and UI for Metaflow pipelines.
- metaflow-flyte - Schedule and monitor Metaflow pipelines through Flyte without rewriting them.
- metaflow-kestra - Kestra scheduling, triggers, and UI for Metaflow pipelines.
- metaflow-kubeflow - Deploy Metaflow flows to Kubeflow Pipelines with no infra changes required.
- metaflow-mage - Mage pipeline orchestration and UI for Metaflow flows.
- metaflow-prefect - Prefect scheduling, deployments, and UI for Metaflow pipelines.
- metaflow-slurm - Run steps on HPC Slurm clusters via SSH +
sbatch. Beta. - metaflow-temporal - Temporal scheduling and durable workflows for Metaflow pipelines.
- metaflow-windmill - Windmill workflow automation and UI for Metaflow pipelines.
- metaflow-deepspeed -
@deepspeedfor multi-node DeepSpeed training with S3/Azure checkpoint uploads. Experimental. - metaflow-mpi -
@mpifor multi-node MPI programs (C/Fortran/mpi4py). Experimental. - metaflow-pyspark - PySpark decorator for Metaflow steps. Experimental, low adoption.
- metaflow-ray - Ephemeral Ray clusters on AWS Batch or Kubernetes. Supports Ray Core, Train, Tune, and Data.
- metaflow-sandbox - Metaflow steps in millisecond-start sandboxes with cloud-scale fanout and consistent deps.
- metaflow-tensorflow -
@tensorflowthat auto-configuresTF_CONFIGfortf.distribute.Strategy. Experimental. - metaflow-torchrun - Run tasks as nodes in a
torchrunDDP job on Batch or Kubernetes.
- metaflow-checkpoint -
@checkpoint,@model, and@huggingface_hubfor fault-tolerant training and model lineage. APIs may change. - metaflow-checkpoint-examples - Examples covering PyTorch, Keras, Lightning, and distributed DDP.
- metaflow-gpu-profile -
@gpu_profiledecorator that renders GPU utilization as a Metaflow card. - metaflow-measure - Emit step metrics to Datadog and other backends via a
measureAPI. - metaflow-profiler - Flamegraph profiling card for Metaflow steps.
- metaflow-sentry-logger - Sentry logging via
@sentry.⚠️ Relies on an unsupported extension API; broken as of Metaflow 2.7.20. - metaflowbot - Slack bot for real-time flow monitoring with CloudFormation deploy.
⚠️ Older; verify against current Metaflow. - resource-tracker - Zero-dependency CPU/memory/GPU tracker with Metaflow card output and cloud cost recommendations.
- dynamic-card-examples - Live cards with progress bars, Altair/Vega charts, maps, and custom JS.
- metaflow-card-altair - Render Altair charts in Metaflow cards.
- metaflow-card-html - Render raw HTML as a card. Reference implementation for custom card authors.
⚠️ Community reports of breakage. - metaflow-card-notebook - Render Jupyter Notebooks as Metaflow cards.
⚠️ Last commit 2022. - metaflow-card-scatter3d - Live 2D and 3D scatter plot card.
- metaflow-card-template - Starter template for building new card types.
- metaflow-dataprofiler - Instant EDA reports on every DataFrame in your Metaflow steps — zero code changes.
- metaflow-traincard - Live loss curves, GPU telemetry, and checkpoints in your Metaflow run card.
- airflow-metaflow-demo - Metaflow + Airflow in Docker Compose with
KubernetesPodOperatorsteps. - comet_ml -
@comet_flowand@comet_stepship in thecomet_mlpackage, including automatic Card export. - hamilton-metaflow - Hamilton as a feature engineering layer inside Metaflow steps.
- sap-ai-core-metaflow - Generates Argo Workflow Templates from Metaflow flows for SAP AI Core.
- wandb -
@wandb_logships in thewandbpackage for experiment tracking. - zdatasets - Zillow's Dataset SDK with a
DatasetParameterintegration for Metaflow flows.
- gha-metaflow - GitHub Actions workflows that trigger Metaflow runs on push/PR.
- metaflow-contracts - Catch bad data between Metaflow steps before it corrupts your pipeline.
- metaflow-dev-vscode - VS Code extension with shortcuts for running flows and steps from the editor.
- metaflow-diff - Diff your working directory against a past Metaflow run.
- metaflow-mcp-server - MCP server for inspecting runs, logs, and artifacts from any AI agent.
- metaflow-optuna - Parallel hyperparameter tuning with true adaptive TPE — no sequential bottleneck.
metaflow-stubs- Type stubs for IDE autocompletion.pip install metaflow-stubs
- diffusion-metaflow - Stable Diffusion text-to-image and text-to-video pipelines.
- dsbook - Code for Effective Data Science Infrastructure by Ville Tuulos.
- full-stack-ML-metaflow-tutorial - Workshop: prototype to production ML with dbt, Great Expectations, W&B, and SageMaker.
- hacker-news-sentiment - LLM-powered topic and sentiment analysis on Hacker News data.
- metaflow-instruction-tuning - LLM instruction tuning (Alpaca-LoRA, LLaMA 7B) with
@torchrun. - metaflow-trainium - AWS Trainium examples: LLaMA-2 pretraining, BERT fine-tuning.
- metaflow-transformers-tutorials - HuggingFace DistilBERT fine-tuning tutorials by Chip Huyen.
- post-modern-stack - Modern data stack + ML stack using dbt, SageMaker, and Metaflow.
- rag-demo - End-to-end RAG pipeline with LlamaIndex and LanceDB/Pinecone.
- recs-at-reasonable-scale - Recommendations at scale with dbt, NVIDIA Merlin, and Metaflow.
- triton-metaflow-starter-pack - NVIDIA Triton Inference Server starter pack.
- tutorials - Official tutorials on workflow graphs, versioning, data access, and scheduling.
- user2020-metaflow-tutorial - R tutorial from useR! 2020.
- you-dont-need-a-bigger-boat - End-to-end RecSys with Metaflow, Snowflake, SageMaker, dbt, and W&B.
- Project must be Metaflow-related with a clear install path
- One-line description only
- Flag experimental/broken items with
⚠️ - No item without a working link
