Apache Airflow Alternatives
Apache Airflow
Airflow is a platform to programmaticaly author, schedule and monitor data pipelines.
Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
Principles
Dynamic: Airflow pipelines are configuration as code (Python), allowing for dynamic pipeline generation. This allows for writting code that instantiate pipelines dynamically.
Extensible: Easily define your own operators, executors and extend the library so that it fits the level of abstraction that suits your environment.
Elegant: Airflow pipelines are lean and explicit. Parameterizing your scripts is built in the core of Airflow using powerful Jinja templating engine.
Scalable: Airflow has a modular architecture and uses a message queue to talk to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity.
Best Apache Airflow Alternatives
Are you trying to find the best programs like Apache Airflow? Have a look at this post and weigh in on your thoughts. Cool!
n8n.io
FreeOpen SourceSelf-HostedSoftware as a Service (SaaS)n8n is a free Node-based source-available Workflow Automation Tool. It can be self-hosted, easily extended, and so also used with internal tools.
Features:
- Airtable integration
- API Integration
- Asana integration
- Dropbox integration
- Sync with Dropbox
- Git integration
- GitHub integration
- Google Sheets integration
- Integration
- Integration with Pipedrive
- Nextcloud integration
- Pipedrive integration
- Slack integration
- Sync with Nextcloud
- Task Automation
- Trello integration
- Twilio integration
- Workflow
- Workflow Automation
RunDeck
FreeOpen SourceLinuxRunDeck is an open source automation service with a web console, command line tools and a WebAPI. It lets you easily run automation tasks across a set of nodes.
Features:
- Job scheduling
- Task Scheduling
- Workflow Automation
- Configuration
- Configuration Management
- Server Management
Zenaton
FreemiumClever CloudSoftware as a Service (SaaS)HerokuA Workflow Builder for Developers. Build event-driven processes in days instead of months .
Features:
- Workflow
- Container Orchestration
- Error Handling
- Real-time monitoring
- Workflow Triggers
- API Integration
- Php
- Python
- Ruby
- Scheduling
- Task Automation
- Task Scheduling
- Workflow Automation
Shipyard App
FreemiumWebSoftware as a Service (SaaS)Workflow Automation Platform for Data Teams. Build, Monitor, and Share Data Solutions 10x faster without DevOps.
Features:
- Business Intelligence
- Data analytics
- Data-management
- Data science
- Developer Tools
- Workflow
- Workflow Automation
- Workflow Management
StackStorm
FreeOpen SourceLinuxStackStorm is a powerful open-source automation platform that wires together all of your apps, services and workflows. It’s extendable, flexible, and built with love for...
Features:
- Job scheduling
- REST API
- Scheduling
- SSH
- Workflow Automation
Apache Oozie
FreeOpen SourceLinuxOozie is a workflow scheduler system to manage Apache Hadoop jobs. Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions. Oozie Coordinator jobs are...
Azkaban
FreeOpen SourceLinuxAzkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provides an easy to use web...
Luigi
FreeOpen SourceLinuxLuigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes...
Metaflow
FreeOpen SourceSelf-HostedFramework for real-life data science; build, improve, and operate end-to-end workflows.
Features:
- Data science
- Workflow Automation
Upvote Comparison
Interest Trends
Apache Airflow Reviews
Add your reviews & share your experience when using Apache Airflow to the world. Your opinion will be useful to others who are looking for the best Apache Airflow alternatives.