Open-Source MLOps Tools: A Practical Production Stack

A self-hosted open-source stack must cover the same five responsibilities as any production stack: data versioning, experiment tracking, pipeline orchestration, model serving and monitoring. The difference is not the responsibilities but who owns them — you do, for every component.

Coverage is not the same as adoption. A stack that tracks experiments but never registers a model, or serves a model but never monitors drift, has gaps that become incidents. The stack must cover the full loop, including a tested rollback path, before it is called production.

The smallest practical self-hosted stack is DVC for data versioning, MLflow for tracking and registry, one orchestrator (Kubeflow Pipelines, Airflow, Dagster or Prefect), Ray Serve or a containerised serving process, and Evidently for drift. Five components, five owners, five typed handoffs.

Smaller than five means one component owns two stages. MLflow is not an orchestrator; Evidently is not a serving runtime. Collapsing stages into one tool removes the failure boundary that contains incidents.

The reference architecture runs each component on a single cluster with shared object storage. DVC pointers live in the data registry; MLflow stores runs and registry aliases in a Postgres backend with artefacts in object storage; the orchestrator executes pipeline steps as containers; Ray Serve loads the artefact behind an MLflow alias; Evidently reads prediction logs and reference data to compute drift.

Every handoff is a typed artefact: a dataset version, an MLflow run ID, a registered model alias, a container digest, a metrics export. No handoff depends on a human copying a file. This is the architecture documented in the School of Core AI reference implementation.

The stack stays composable because every handoff is a typed artefact rather than an ad-hoc file. DVC passes a dataset pointer; MLflow passes a run ID and a registered model alias; the orchestrator passes step outputs through a pipeline root in object storage; serving loads the artefact behind the alias; monitoring consumes prediction logs and reference data.

When a handoff is untyped — a hardcoded path, a manual copy — the stack stops being composable and starts being brittle. The first sign is that an upgrade to one component breaks another. Typed handoffs are the contract that makes each component replaceable.

Kubernetes is justified when you need multi-tenant scheduling, GPU sharing across teams, or declarative rollouts with KServe. It is not justified by tool popularity or by the presence of Kubeflow in the stack — Kubeflow Pipelines can run on a single cluster without a full Kubernetes platform team.

The trigger is operational: when ad-hoc deployment, manual retries and single-tenant resource allocation cost more than a Kubernetes control plane, move the orchestrator and serving layers to Kubernetes. Before that point, Kubernetes adds burden without leverage.

Self-hosting means you own authentication and secrets. MLflow and the orchestrator need their own auth and RBAC; cloud object storage needs scoped credentials; serving needs request auth; monitoring needs read access to prediction logs. None of these come from the open-source license.

The network boundary is the trust boundary. In an air-gapped deployment, every component runs without outbound internet, which rules out managed dependencies and telemetry. Design the secret and network model before adopting components, not after the first incident.

In a self-hosted stack, the operator owns backup, upgrade and disaster recovery for every component. The MLflow Postgres backend, the object storage artefacts, the orchestrator's run history and the serving configuration all need tested backup and restore procedures.

Upgrade is the hidden cost. Each component releases on its own cadence, and an upgrade to one can break the handoff to another. A managed platform offloads this; a self-hosted stack does not. Budget for upgrade testing as a recurring platform task, not a one-off.

A self-hosted stack earns its keep until one of three triggers fires. Multi-tenant scheduling or GPU sharing outgrows ad-hoc deployment, which points to Kubernetes. The platform team's capacity falls below the operational burden, which points to a managed platform. A data-residency or cost constraint makes one stage cheaper to buy than to operate, which points to hybrid.

Replacement, not addition, is the healthy response. When a trigger fires, replace the one component that no longer fits and keep the typed handoffs so the rest of the stack stays intact. Adding a sixth tool to mask a broken handoff compounds the problem.

The reference implementation wires DVC, MLflow, Kubeflow Pipelines, Ray Serve and Evidently into one reproducible pipeline on a shared cluster, with each handoff typed and each component replaceable. The end-to-end project demonstrates the loop from versioned data to rollback on a real workload.

The implementation documents the integration contracts and the operational tasks the team owns. It is not a benchmark; it is a working stack that survived upgrade, backup and rollback testing.