MLOps Roadmap

You can build the model; the missing half is everything that happens after model.fit(). Notebooks do not survive contact with production: they run on your laptop, depend on your local Python, and nobody can reproduce the run that produced your "best" model. Learn four things in this order:

1. Docker — ship the model + preprocessing together so it runs identically on your laptop, CI and production.
2. MLflow — log every run's params, metrics and artefact so any result is reproducible and traceable.
3. CI/CD (GitHub Actions) — automate training, tests and deployment so a merge does not silently change behaviour.
4. Monitoring — track drift and prediction quality after release, because a model that scored 0.93 at training can be 0.71 in production two months later and you will not know unless you measure.

You do not need to become a Kubernetes admin. This roadmap is built around exactly that gap. Use the start-path selector above to enter at data contracts and versioning.

Yes. CI/CD, containers, IaC, monitoring, rollback, alerting: already in your toolkit. The missing layer is ML literacy, and it is the part that will quietly break your pipelines if you skip it.

The three things that trip up DevOps engineers:
1. A model is not deterministic code. The same build can produce different behaviour because the data changed. Your standard "green build = safe to deploy" mental model does not hold.
2. You need to read a confusion matrix, understand precision vs. recall, and know what data leakage looks like — not to build models, but to reason about why a deployed model degraded.
3. Evaluation gates are not unit tests. A candidate model must beat the current production model on held-out data, not just pass a correctness check.

Add: data validation (schema + distribution checks), MLflow for experiment tracking, and model evaluation gates. The MLOps layer on top of your existing skills is thin — but it is the thin layer that makes or breaks production ML. Use the start-path selector above — you can enter at serving patterns.

Yes, but not by skipping ML literacy. You do not need to design neural networks — you need enough ML understanding to treat a model as a deployable artefact that behaves differently from code.

Five concepts before the build stages: (1) training vs. inference (a model learns weights, then uses them — it is not "run the function"); (2) metrics — accuracy, precision, recall, AUC, and why accuracy alone lies on imbalanced data; (3) overfitting — a model that memorises training data and fails on new data; (4) data leakage — when test data sneaks into training and inflates your score; (5) why a model that scored 0.95 in the notebook can score 0.70 in production (distribution shift).

Budget the time these five concepts take rather than a fixed number of weeks — being comfortable reading an evaluation report is the readiness signal, not a calendar. The MLOps skills themselves — pipelines, serving, monitoring — are closer to DevOps than to ML. Skip the literacy step and you will build pipelines that look correct but miss every real failure mode.

Closer to platform engineering for ML than to model research. A typical week breaks into four buckets:

Pipeline work (~40%) — maintain data validation, training, packaging and deployment. A scientist hands you a notebook; you turn it into a reproducible, tested pipeline that runs on schedule and produces a registered model.
Incidents (~25%) — a drift alert fires at 2am: investigate, decide rollback vs. retrain, write the runbook so the next on-call does not debug from scratch.
Collaboration (~20%) — review data scientists' code for leakage, help them log experiments properly, ship their model behind a versioned API.
Reliability improvements (~15%) — add evaluation gates, improve monitoring coverage, tighten the rollback path.

If you want to build novel models, that is an ML Engineer role. MLOps is the engineering that makes those models reliably reach and stay in production.

Three roles, one artefact, different halves of its life:

DevOps ships deterministic code. Same build → same behaviour. CI/CD, IaC, rollback. The artefact does not change after deploy.
ML Engineering builds the model — features, validation strategy, serving contracts, model architecture. The artefact is the model itself.
MLOps ships a model that depends on data, and the data drifts. So MLOps adds three things DevOps does not need: data validation (schema + distribution checks before training), evaluation gates (candidate must beat the current model on held-out data, not just pass tests), and model monitoring (track prediction quality, not just service health).

In small teams one person does all three. In larger teams they split: ML Engineers build models, MLOps engineers make them reach and stay in production, DevOps owns the underlying platform.

Five, in this order — each maps to one lifecycle stage:

1. DVC — version your datasets and preprocessing the way Git versions code. Without this, "retrain on the latest data" is not reproducible.
2. MLflow — log params, metrics and the model artefact for every run, and keep a model registry. This is how you prove which run was best.
3. Docker — package the model + preprocessing together so it runs identically in dev, CI and prod.
4. FastAPI — serve the containerised model behind a REST endpoint with a /health check.
5. GitHub Actions — wire training, tests, evaluation gates and deployment into a pipeline that runs on every merge.

Add Prometheus + Grafana for monitoring once something is in production. Do not start with Kubernetes — it is the most over-prescribed tool in MLOps and will cost you weeks before you need it. Airflow and feature stores come later, only when a real problem demands them.

No. Containers — yes, because the model and its preprocessing must ship together. Kubernetes — no, unless you have a specific reason.

Most production ML systems run on a managed service (AWS SageMaker, GCP Vertex AI, Azure ML) or even a single beefy host. Reach for Kubernetes only when you hit one of: distributed training that one machine cannot handle, multi-tenant serving with autoscaling, or shared GPU infrastructure across teams.

Rule of thumb: if your model trains in under an hour on a single machine and serves under 100 QPS, Kubernetes is overhead you do not need. Adding it before you need it will slow your delivery more than it helps — you will spend more time on cluster YAML than on the model.

Only when you have a specific problem it solves. A feature store adds real infrastructure cost — do not adopt one on FOMO.

You need one when:
• Multiple teams share feature definitions and you are tired of three teams recomputing the same feature three different ways.
• You serve in real time and need online/offline consistency (the feature value at serving time must match what was used in training).
• You need point-in-time correctness for training (avoiding future leakage into historical features).

For a single model or a small team, versioned datasets (DVC) plus a preprocessing pipeline packaged inside the model container are enough. Adopt a feature store when the pain of not having one is concrete, not because a conference talk made it sound modern.

Data drift = the input distribution changed between training and production. The world moved; your model did not. A churn model trained on 2024 customer behaviour starts mispredicting in 2026 because usage patterns shifted — the code is fine, the data is not.

Detect it three ways:
1. Feature-level statistical tests — KS test or Population Stability Index (PSI) on each feature; flag when a feature's distribution shifts beyond a threshold.
2. Prediction distribution shift — if the model's output distribution changes (e.g. suddenly 30% more "churn" predictions), inputs likely shifted.
3. Embedding distance — for text/image inputs, track distance between production embeddings and a reference set.

Tools: Evidently and WhyLabs compute these automatically and generate reports. Drift is a signal to investigate and retrain a candidate — not to auto-promote a new model.

No. This is the single most expensive mistake in MLOps.

Drift means the data changed. It does not mean a model retrained on the new data will be better — it might overfit to a temporary shift, or the new data might be noisy.

Split it:
• Auto-trigger candidate retraining — yes. Run the pipeline, produce a candidate, log it.
• Auto-promote to production — no. The candidate must pass evaluation gates first: held-out metrics against a fixed test set, head-to-head comparison with the current production model, and operational checks (latency, memory, footprint).

Auto-retrain + auto-promote causes silent quality oscillation: the model flips between versions every few hours, nobody can explain why, and the team stops trusting the pipeline. Gate the promotion; automate the candidate.

Two layers, two different questions. You need both.

Service monitoring asks: "is the endpoint alive and fast?" — CPU, memory, latency, error rate, pod restarts. This is standard DevOps monitoring (Prometheus + Grafana). It catches a crashed pod or a memory leak.

Model monitoring asks: "are the predictions still correct?" — input drift, prediction distribution shift, and delayed ground-truth comparison once real labels arrive. This is MLOps-specific. It catches a model that is happily serving 200ms responses that are silently wrong because the input data shifted last month.

The trap: a model can be perfectly healthy as a service and wrong as a predictor. Service monitoring alone will never catch silent model degradation. Model monitoring alone will never catch a crashed pod. Run both; wire both to alerting.

Three signals, in priority order. Tool names are teachable in a week; judgement about ML in production takes months — hire for the judgement.

1. A built pipeline, not installed tools. "I set up MLflow" tells me nothing. "I built a pipeline that catches a bad model before it reaches users and rolls back when serving breaks" tells me everything. I ask: "walk me through what happens when a data contract breaks at 2am." If they can narrate detect → alert → rollback → investigate → fix without hand-waving, they have run production ML.

2. Failure reasoning. I ask about three scenarios: a candidate model regresses on the gate, the serving endpoint OOMs, ground-truth labels are delayed two weeks. Can they reason through each, or do they freeze?

3. ML literacy. Can they read a confusion matrix, explain precision vs. recall trade-off, and say why a model degrades in production? They do not need to build models — they need to reason about the ones they ship.

Indicative timeline: For someone with Python, Git and basic model training/evaluation, allow about 10–12 weeks at 8–10 hours per week to complete the roadmap builds and capstone. If you need those foundations first, budget additional preparation time; the amount varies by starting point. Completing this self-study roadmap is not a guarantee of job readiness and is not equivalent to the separate five-month live programme.

What to build: one capstone — a reproducible training pipeline, a containerised serving endpoint, a CI/CD pipeline with evaluation gates, a monitoring dashboard, and a runbook for three failure scenarios. That artefact is worth more on a CV than any certificate.

Where to go next:
• Deeper model development — feature engineering, validation, serving contracts → ML Engineer roadmap
• LLM-specific operations — prompt versioning, LLM evaluation, token cost → LLMOps roadmap
• AI-assisted incident detection → AIOps roadmap
• Structured practice with instructor feedback and a reviewed capstone → SCAI's MLOps course covers the full lifecycle live.