Build vs Buy an AI Platform: Managed, Open-Source or Hybrid

A team needs an AI platform when the coordination cost of building AI infrastructure ad hoc exceeds the cost of building or buying a shared platform. The signal is shadow operations: when multiple teams are each building their own model registries, training pipelines, serving infrastructure and observability stacks, the organization is paying for the same work multiple times and getting lower quality than a centralized platform would produce. The threshold, from platform engineering practice, is roughly 20–30 platform users — below that, a small number of teams can share infrastructure informally; above it, the shadow-operations tax exceeds the platform investment.

The need for a platform is not driven by team size alone but by the diversity and complexity of AI workloads. A team running a single LLM endpoint does not need a platform — it needs a serving framework. A team running multiple models, training pipelines, RAG systems and agent workflows across multiple environments needs a platform that centralizes the shared capabilities. The more diverse the workloads, the more value a platform provides, because the capabilities (registry, serving, evaluation, observability, governance) are shared across workloads. A team with one workload type can specialize; a team with five workload types needs a platform that generalizes.

The need is also driven by compliance and governance requirements. An organization that must produce audit evidence for every model decision, enforce access controls on every artefact, and maintain lineage across the lifecycle needs a platform that provides these capabilities centrally. Building these controls ad hoc for each team produces inconsistent evidence and gaps; a centralized platform produces consistent evidence as a by-product of the controls it enforces. The compliance dimension may push the platform decision earlier than the efficiency dimension — a small team with strict compliance requirements may need a platform before the 20–30 user threshold.

Before choosing build, buy or hybrid, the team must enumerate the capabilities the platform must provide. The twelve-layer model — infrastructure, environments, model registry, training pipelines, serving, evaluation, observability, deployment, security, governance, cost control, developer experience — is a checklist. For each layer, the team asks: do we need this capability, who provides it, and what is the maturity gap? A team that only needs serving and observability does not need a platform with twelve layers — it needs a serving framework and an observability stack. A team that needs training, serving, evaluation, observability, governance and cost control needs most of the layers and should evaluate a full platform.

The capability requirements are driven by the workload types. An ML team that trains models from scratch needs training pipelines, a model registry, and evaluation. An LLM team that fine-tunes and serves needs serving, evaluation and token-level cost control but may not need training pipelines. A RAG team needs vector store management, retrieval evaluation, and serving. An agent team needs tool registries, checkpoint stores, and agent observability. Each workload type has a different capability set, and the platform must cover the union. A team that only does LLM serving should not buy a platform optimized for ML training — it would pay for capabilities it does not use.

The capability gap analysis is the input to the build-vs-buy decision. For each required capability, the team asks: is there a managed service that provides it, is there an open-source tool we can self-host, or must we build it? A capability with a mature managed service and a mature open-source alternative is a buy-or-compose decision; a capability with no mature option is a build decision. The gap analysis reveals which capabilities are commoditized (buy/compose) and which are differentiating (build). The platform strategy should commoditize the commoditized and invest in the differentiating — not the reverse.

A managed AI platform is a vendor-provided system that delivers AI capabilities as a service: model training, serving, evaluation, observability, governance and cost control, operated by the vendor. The team consumes the capabilities via APIs and a console; the vendor operates the infrastructure, handles scaling, patching, and availability. Examples include cloud-provider AI platforms (AWS SageMaker, Google Vertex AI, Azure ML) and specialized AI platforms (Hugging Face Endpoints, Replicate, Databricks). The value proposition is time-to-production: a team can deploy a model in hours, not weeks, because the vendor has already built the platform.

The trade-off is control and lock-in. A managed platform abstracts the infrastructure, which means the team cannot customize capabilities the platform does not expose. If the platform's serving framework does not support a custom batching strategy, the team must accept the platform's default or leave the platform. The lock-in is operational: the team's workflows, model artefacts, and observability data live in the vendor's system, and migrating to another platform requires rebuilding the workflows and re-registering the artefacts. The exit cost is high — not prohibitive, but high enough that teams typically stay on a managed platform for years after the initial choice.

The managed platform is the right choice when time-to-production is the dominant constraint, the team does not have platform engineering capacity, or the workloads fit the platform's supported patterns. A team of 5 data scientists who need to deploy a model in two weeks should use a managed platform — building a platform in two weeks is not feasible, and the managed platform's limitations are acceptable for the time savings. A team of 50 engineers with diverse workloads and strict governance requirements may find the managed platform's limitations unacceptable — the platform cannot enforce custom governance controls, and the team has the capacity to build. The decision is not managed vs self-hosted in the abstract; it is managed vs self-hosted for this team, these workloads, and this timeline.

A composable open-source platform is a self-hosted system assembled from open-source components: a model registry (MLflow, W&B), a serving framework (vLLM, TGI, Triton), an evaluation framework (promptfoo, DeepEval), an observability stack (OpenTelemetry, Prometheus, Grafana, Phoenix), a deployment system (Argo CD, Flux), and a governance layer (custom or Open Policy Agent). The team assembles the components, operates them, and provides the golden paths that make them consumable. The value proposition is control: the team can customize any capability, enforce any policy, and avoid vendor lock-in.

The trade-off is operational responsibility. The team must operate every component: patching, scaling, upgrading, debugging, and on-call. Each component has its own release cycle, its own configuration, and its own failure modes. A composable platform with 10 open-source components has 10 upgrade paths, 10 configuration surfaces, and 10 potential failure points. The operational load is significant — a team without dedicated platform engineers will struggle to keep the platform healthy, and the platform will degrade into a collection of unmaintained components. The composable approach requires platform engineering capacity, not just data science capacity.

The composable platform is the right choice when control is the dominant constraint, the team has platform engineering capacity, or the workloads require customization that no managed platform provides. A team with strict data residency requirements (data cannot leave the data centre), custom governance controls (sector-specific compliance), or specialised workloads (research models with custom architectures) may have no managed option and must compose. A team with platform engineers who want to build a platform as a product — with golden paths, documentation, and adoption metrics — may choose composable for the control and the career development. The decision assumes the team can operate the platform; if it cannot, the composable approach produces a platform that is more broken than a managed one.

A hybrid architecture uses a managed platform for some capabilities and a self-hosted platform for others. The principle is: commoditize the commoditized, invest in the differentiating. A team that needs standard LLM serving (commoditized — many managed options) but custom governance controls (differentiating — no managed platform provides sector-specific compliance) uses a managed serving layer with a self-hosted governance layer. The hybrid approach captures the time-to-production of managed for the standard capabilities and the control of self-hosted for the differentiating capabilities.

The hybrid boundary is drawn at the capability, not the layer. Within a single layer — say, observability — the team may use a managed metrics store (commoditized) with a custom quality dashboard (differentiating). The boundary is: where the managed capability meets the team's needs, use managed; where it does not, build or compose. The gap analysis from earlier in this page is the input: capabilities with mature managed options and no differentiating need are managed; capabilities with no managed option or a differentiating need are built or composed. The hybrid architecture is not a compromise — it is a deliberate choice that optimizes each capability independently.

The hybrid architecture's risk is integration complexity. The managed and self-hosted components must interoperate: the self-hosted governance layer must read from the managed serving layer's logs, the self-hosted cost dashboard must pull from the managed platform's billing API, the custom evaluation framework must deploy models to the managed serving endpoint. Each integration is a custom interface that must be maintained. The hybrid approach reduces operational responsibility (the managed layers are operated by the vendor) but adds integration responsibility (the interfaces between managed and self-hosted must be maintained). The team must have enough platform engineering capacity to maintain the integrations, even if it does not operate the full stack.

Total cost of ownership (TCO) for an AI platform includes the platform cost (managed subscription or self-hosted infrastructure), the operational cost (platform engineers' time to operate the platform), the integration cost (building and maintaining integrations between components), and the opportunity cost (what the platform engineers could build instead of operating the platform). A managed platform that costs $50,000/year in subscription but requires no platform engineering time may be cheaper than a self-hosted platform that costs $10,000/year in infrastructure but requires two full-time platform engineers — the engineer cost ($300,000+/year) dwarfs the infrastructure cost. TCO must include people cost, not just cloud cost.

The TCO calculation must account for utilisation. A managed platform's per-use pricing is cheap at low utilisation (you pay for what you use) and expensive at high utilisation (the per-use cost scales with volume). A self-hosted platform's fixed cost is expensive at low utilisation (you pay for idle capacity) and cheap at high utilisation (the fixed cost is amortised across volume). The crossover point — where self-hosted becomes cheaper than managed — depends on the utilisation rate, the managed pricing, and the self-hosted infrastructure cost. The team must forecast utilisation and calculate TCO at the expected utilisation, not at peak or at zero.

The TCO must also account for exit cost. A managed platform's exit cost includes re-registering all model artefacts in a new platform, rebuilding all workflows in the new platform's APIs, and retraining or re-instrumenting the observability stack. A self-hosted platform's exit cost is lower because the components are open-source and portable — the model registry can be pointed at a new storage backend, the serving framework can be redeployed on new infrastructure. The exit cost is not paid at the decision point but at the migration point, and it must be estimated and included in the TCO comparison. A platform that is cheap to run but expensive to leave may have a higher TCO than a platform that is expensive to run but cheap to leave.

Vendor lock-in is the cost of leaving a managed platform — the effort to move artefacts, workflows and data to another platform. Lock-in is not inherently bad; it is a trade-off. A managed platform that provides high value, meets the team's needs, and has a reasonable exit path may be worth the lock-in. The evaluation is: what is the exit cost, what is the probability of needing to exit, and is the value worth the risk? A platform with high exit cost and high exit probability (the vendor may discontinue the product, the pricing may increase, the team may outgrow the platform) is a bad choice; a platform with high exit cost and low exit probability may be acceptable.

The exit risk is evaluated by analysing the portability of each artefact and workflow. Model artefacts stored in a standard format (ONNX, Safetensors) are portable — they can be loaded by any serving framework. Model artefacts stored in a vendor-proprietary format are not — they require conversion or retraining. Workflows defined in standard APIs (OpenAI-compatible, KServe) are portable; workflows defined in vendor-proprietary APIs are not. Observability data stored in a standard format (OpenTelemetry) is portable; data stored in a vendor-proprietary format is not. The team should assess each artefact and workflow for portability before committing to a managed platform, and prefer platforms that use standard formats.

The mitigation for lock-in is abstraction. A team that uses a managed serving platform behind an abstraction layer — a standard API that the application calls, which routes to the managed platform — can swap the platform without changing the application. The abstraction layer adds a small amount of complexity but reduces the exit cost from 'rewrite the application' to 'swap the backend.' The governance policy may require abstraction layers for critical workloads — a managed platform that the team cannot leave without rewriting the application is a strategic risk, and the abstraction layer is the control. The lock-in evaluation is not 'avoid all lock-in' — it is 'understand the exit cost, mitigate where the risk is high, and accept where the value justifies it.'

Team maturity — the team's size, platform engineering experience, and operational capacity — is the primary driver of the build-vs-buy decision. A small team (5–10 engineers) with limited platform experience and a fast time-to-production constraint should choose a managed platform: the vendor provides the platform, the team consumes the capabilities, and the team's limited capacity is spent on the product, not the platform. A managed platform for a small team is not a compromise — it is the correct choice for the constraint set.

A medium team (20–50 engineers) with some platform experience and mixed workloads should evaluate hybrid. The standard capabilities (serving, observability) can be managed; the differentiating capabilities (governance, custom evaluation) can be self-hosted. The hybrid approach matches the medium team's capacity — enough to operate the custom layers, not enough to operate the full stack. The hybrid decision is the most common for medium teams because it optimizes each capability independently, but it requires the gap analysis and the integration capacity to succeed.

A large team (50+ engineers) with dedicated platform engineers and diverse, custom workloads should evaluate composable open-source. The team has the capacity to operate a full platform, the workloads require customization that no managed platform provides, and the control is worth the operational cost. The composable decision is the correct choice for a team that treats the platform as a product — with a platform team, golden paths, and adoption metrics. A large team that chooses managed when it has the capacity to compose may be paying for capabilities it could build better and cheaper — the decision must match the team's maturity, not its habit.