DTStack/Taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

Java

1,286

1,336 commits

updated Jul 31, 2026

See the code

README

Taier Logo

Taier

A distributed dispatching system

Office Website | Document

中文 | English

Introduction

Taier , spelling in chinese is 太阿, is one of the ancient chinese celebrated swords.

Taier is distributed dispatching system that focus on different tasks submitted and scheduled. It's aimed at reducing the ETL's cost, making the complex dependencies between tasks clearly and reducing the labor cost about submitting, scheduling and O&M.

It's unnecessary to concern about the complex dependencies between tasks and the underlying architecture about the big data platform at Taier, so that you can pay more attention into business.

Taier provide an one-stop big data platform for submitting tasks, scheduling tasks, O&M, presentation about indicators.

The core features for Taier are as follows:

  • Easy to distributed extend
  • Visualization config for DAG
  • With an IDE development platform designed for big-data users
  • Supports to develop your own plugins
  • Multiply task modes including guide mode and script mode
  • Supports to the dependencies between upstream/downstream tasks
  • Supports batch and stream tasks
  • Integrates various different versions of Hadoop
  • Easy to integrate Flink Standalone
  • Completely safe and non-intrusive to the cluster's environment
  • Isolation by tenants and clusters
  • Supports kerberos authentication
  • Different versions for tasks
  • Supports user-defined parameters for task
  • Real-time monitoring about cluster's resources
  • Real-time presented about data indicators
  • Restrict the task's resource

Architecture

architecture

Quick start

In Docker

$ wget https://raw.githubusercontent.com/DTStack/Taier/master/docker-compose.yml
$ docker-compose up -d

Online Preview

Develop Tasks

main

Tasks

TasksDocumentation
Work FlowDocumentation
Data SyncDocumentation
Data AcquisitionDocumentation
FlinkDocumentation
ShellDocumentation
PythonDocumentation
Spark SQLDocumentation
Hive SQLDocumentation
Flink SQLDocumentation
OceanBase SQLDocumentation
ClickHouse SQLDocumentation
Doris SQLDocumentation
TiDB SQLDocumentation
MySQL SQLDocumentation
Vertica SQLDocumentation
Postgre SQLDocumentation
SqlServer SQLDocumentation
Greenplum SQLDocumentation
MaxCompute SQLDocumentation
GaussDB SQLDocumentation
DataXDocumentation
User-defined TaskDocumentation

Questions

FAQ Reference For questions, bugs and supports please open an issue, we'll reply you in time.

Stay in touch

Contribution

Please make sure to read the Contributing Guide before making a pull request.

Contributor

License

Taier is under the Apache 2.0 license. See the LICENSE file for details.

azkaban
chunjun
cronjob-scheduler
dag
data-schedule
distributed-schedule-system
flink
hadoop
hive
job-scheduler
scheduler
spark
task-schedule
workflow-scheduling-system

DTStack/Taier

Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display

Java

1,286

1,336 commits

updated Jul 31, 2026

See the code

README

Taier Logo

Taier

A distributed dispatching system

Office Website | Document

中文 | English

Introduction

Taier , spelling in chinese is 太阿, is one of the ancient chinese celebrated swords.

Taier is distributed dispatching system that focus on different tasks submitted and scheduled. It's aimed at reducing the ETL's cost, making the complex dependencies between tasks clearly and reducing the labor cost about submitting, scheduling and O&M.

It's unnecessary to concern about the complex dependencies between tasks and the underlying architecture about the big data platform at Taier, so that you can pay more attention into business.

Taier provide an one-stop big data platform for submitting tasks, scheduling tasks, O&M, presentation about indicators.

The core features for Taier are as follows:

  • Easy to distributed extend
  • Visualization config for DAG
  • With an IDE development platform designed for big-data users
  • Supports to develop your own plugins
  • Multiply task modes including guide mode and script mode
  • Supports to the dependencies between upstream/downstream tasks
  • Supports batch and stream tasks
  • Integrates various different versions of Hadoop
  • Easy to integrate Flink Standalone
  • Completely safe and non-intrusive to the cluster's environment
  • Isolation by tenants and clusters
  • Supports kerberos authentication
  • Different versions for tasks
  • Supports user-defined parameters for task
  • Real-time monitoring about cluster's resources
  • Real-time presented about data indicators
  • Restrict the task's resource

Architecture

architecture

Quick start

In Docker

$ wget https://raw.githubusercontent.com/DTStack/Taier/master/docker-compose.yml
$ docker-compose up -d

Online Preview

Develop Tasks

main

Tasks

TasksDocumentation
Work FlowDocumentation
Data SyncDocumentation
Data AcquisitionDocumentation
FlinkDocumentation
ShellDocumentation
PythonDocumentation
Spark SQLDocumentation
Hive SQLDocumentation
Flink SQLDocumentation
OceanBase SQLDocumentation
ClickHouse SQLDocumentation
Doris SQLDocumentation
TiDB SQLDocumentation
MySQL SQLDocumentation
Vertica SQLDocumentation
Postgre SQLDocumentation
SqlServer SQLDocumentation
Greenplum SQLDocumentation
MaxCompute SQLDocumentation
GaussDB SQLDocumentation
DataXDocumentation
User-defined TaskDocumentation

Questions

FAQ Reference For questions, bugs and supports please open an issue, we'll reply you in time.

Stay in touch

Contribution

Please make sure to read the Contributing Guide before making a pull request.

Contributor

License

Taier is under the Apache 2.0 license. See the LICENSE file for details.

azkaban
chunjun
cronjob-scheduler
dag
data-schedule
distributed-schedule-system
flink
hadoop
hive
job-scheduler
scheduler
spark
task-schedule
workflow-scheduling-system

Project health

Latest release

4.1.3 · Feb 25, 2022

Releases this year

0

Significant stargazers

FuryBean

249 followers · starred Feb 2024

liuqiufeng

4 followers · starred Jun 2023

xieyinglin

1 followers · starred Feb 2022