Event-Driven Data Orchestration and Validation
ML pipelines break when upstream data changes. Without orchestration, validation, and quarantine paths, bad data silently corrupts training and serving.
An orchestrated, validated ingestion pipeline that separates clean data from rejected records.
Orchestrated pipeline definition, validation suite, quarantine logic, and a data-quality runbook.
Portfolio Output: Orchestration pipeline with validation and quarantine runbook
View Engineering Deep-Dive & Review Criteria
Choosing partitioning strategy, setting validation thresholds, and designing the quarantine recovery path.
Schema drift from upstream source causing validation failures, or Dagster job failure requiring retry logic.
Pipeline correctness, validation coverage, orchestration reliability, and quarantine explanation.