Home
Astronomer Starship can send your Airflow workloads to new places!
What is it?¶
Starship is a utility to migrate Airflow metadata such as Airflow Variables, Connections, Environment Variables, Pools, and DAG History between two Airflow instances.
Installation¶
Usage¶
- Create a Workspace in Astro or Software to hold Astro Deployments
- Create an Astro Deployment matching the source Airflow deployment configuration as possible
- Run
astro dev initwith the Astro CLI to create a Astro Project locally in your terminal - Add any DAGs to the
/dagsfolder in the Astro Project - Complete any additional setup required to convert your existing Airflow deployment to an Astro Project
- Install Starship (and any additional Python Dependencies) to the Astro Project
- Install Starship to your existing Airflow Deployment
- Deploy the Astro Project to the Astro Deployment with
astro deploy - In the Airflow UI of the source Airflow deployment, navigate to the new
Astronomermenu and select theMigration Tool 🚀option - Follow the UI prompts to migrate, or if needed, look at the instructions to use the Operator
Compatability¶
| Source | Compatible |
|---|---|
| Airflow 1 | ❌ |
| GCC 1 - Airflow 2.x | Operator |
| GCC 2 - Airflow 2.x | ✅ |
| GCC 3 - Airflow 2.x & 3.x | ✅ |
| MWAA - Airflow 2.0 | Operator |
| MWAA - Airflow 2.2+ | ✅ |
| MWAA - Airflow 3.x | ✅ |
| OSS Airflow VM | ✅ |
| Astronomer Products | ✅ |
Supported Airflow Features¶
The following Airflow features are supported by the current version of Starship when running on the corresponding minimum Airflow version.
| Feature | Minimum Airflow Version | Supported |
|---|---|---|
| Connections | 2.0 | ✅ |
| Dag runs | 2.0 | ✅ |
| Environment variables | 2.0 | ✅ |
| Pools | 2.0 | ✅ |
| Task instances | 2.0 | ✅ |
| Variables | 2.0 | ✅ |
| Task Logs | 2.0 | ❌ |
| XComs | 2.0 | ❌ |
| Task instance history | 2.10 | ✅ |
| Assets | 3.0 | ❌ |
| Backfills | 3.0 | ❌ |
| Dag versions | 3.0 | ❌ |
| Task instance notes | 3.0 | ❌ |
| HITL | 3.1 | ❌ |
| Teams | 3.1 | ❌ |
Security Notice¶
This project is an Airflow Plugin that adds custom API routes. Ensure your environments are correctly secured.
Artwork Starship logo by Lorenzo used with permission from The Noun Project under Creative Commons.