Name | Domain | Distributed | Task overhead | Data parallel | Provenance | Dependency-time | Delete unused products | Correct wrt changes in dependencies | Memory-aware scheduling |
---|---|---|---|---|---|---|---|---|---|
funsies | Distributed | ||||||||
Luigi | Distributed | ||||||||
Reena | Computational science | Distributed | |||||||
Yadage | Computational science | Distributed | |||||||
Snakemake | Computational science | Distributed | |||||||
Popper | Computational science | ||||||||
Nix | Package management | Distributed | |||||||
Bazel | Build system | Distributed | |||||||
Buck | Build system | Single server | |||||||
Make | Build system | Single server | |||||||
Scons | Build system | Single server | |||||||
Rake | Build System | Single server | |||||||
Shake | BuildSystem | Single server |
CWL: Just a language for going to other workflow managers.
Apache Flink: Stream oriented.
Apache Airflow: Too complex, designed for big-data.