Apache Airflow Alternatives
Apache Airflow
Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.
Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
Principles
Dynamic: Airflow pipelines are configuration as code (Python), allowing for dynamic pipeline generation. This allows for writting code that instantiate pipelines dynamically.
Extensible: Easily define your own operators, executors and extend the library so that it fits the level of abstraction that suits your environment.
Elegant: Airflow pipelines are lean and explicit. Parameterizing your scripts is built in the core of Airflow using powerful Jinja templating engine.
Scalable: Airflow has a modular architecture and uses a message queue to talk to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity.
Best Free Apache Airflow Alternatives
For the best Apache Airflow alternative check out these alternatives which will work with your device & platform. There are several Apache Airflow-like software that you might want to check as well.
n8n.io
FreeOpen SourceSelf-HostedSoftware as a Service (SaaS)n8n is a free Node-based source-available Workflow Automation Tool. It can be self-hosted, easily extended, and so also used with internal tools.
Features:
- Airtable integration
- API Integration
- Asana integration
- Dropbox integration
- Sync with Dropbox
- Git integration
- GitHub integration
- Google Sheets integration
- Integration
- Integration with Pipedrive
- Nextcloud integration
- Pipedrive integration
- Slack integration
- Sync with Nextcloud
- Task Automation
- Trello integration
- Twilio integration
- Workflow
- Workflow Automation
RunDeck
FreeOpen SourceLinuxRunDeck is an open source automation service with a web console, command line tools and a WebAPI. It lets you easily run automation tasks across a set of nodes.
Features:
- Job scheduling
- Task Scheduling
- Workflow Automation
- Configuration
- Configuration Management
- Server Management
StackStorm
FreeOpen SourceLinuxStackStorm is a powerful open-source automation platform that wires together all of your apps, services and workflows. It’s extendable, flexible, and built with love for...
Features:
- Job scheduling
- REST API
- Scheduling
- SSH
- Workflow Automation
Apache Oozie
FreeOpen SourceLinuxOozie is a workflow scheduler system to manage Apache Hadoop jobs. Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions. Oozie Coordinator jobs are...
Azkaban
FreeOpen SourceLinuxAzkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provides an easy to use web...
Luigi
FreeOpen SourceLinuxLuigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes...
Metaflow
FreeOpen SourceSelf-HostedFramework for real-life data science; build, improve, and operate end-to-end workflows.
Features:
- Data science
- Workflow Automation
Upvote Comparison
Interest Trends
Apache Airflow Reviews
Add your reviews & share your experience when using Apache Airflow to the world. Your opinion will be useful to others who are looking for the best Apache Airflow alternatives.