Data Pipeline Orchestration & Workflow
14 repos
Frameworks and tools for scheduling, executing, and monitoring data workflows and ETL pipelines at scale. The cluster centers on mature Python-based orchestration platforms (Airflow, Prefect, Dagster) that enable declarative DAG-based job scheduling, alongside supporting infrastructure in Go and Java for deployment, monitoring, and data lineage tracking. Repositories here span workflow definition languages, execution engines, observability tooling, and data integration platforms—collectively addressing how teams build, deploy, and maintain production data systems.
Python · 6
Java · 2
Go · 1
Astro · 1
C# · 1
Ruby · 1
workflow ·159,801
orchestration ·148,931
data-science ·133,484
python ·133,484
data-engineering ·133,484
workflow-engine ·129,200
data-pipelines ·124,129
automation ·118,188
etl ·110,508
scheduler ·110,508
