Skip to content

Home

Logo of Spaceship

Astronomer Starship can send your Airflow workloads to new places!

What is it?

Starship is a utility to migrate Airflow metadata such as Airflow Variables, Connections, Environment Variables, Pools, and DAG History between two Airflow instances.

Logo of Spaceship

Installation

pip install astronomer-starship

Usage

  1. Create a Workspace in Astro or Software to hold Astro Deployments
  2. Create an Astro Deployment matching the source Airflow deployment configuration as possible
  3. Run astro dev init with the Astro CLI to create a Astro Project locally in your terminal
  4. Add any DAGs to the /dags folder in the Astro Project
  5. Complete any additional setup required to convert your existing Airflow deployment to an Astro Project
  6. Install Starship (and any additional Python Dependencies) to the Astro Project
  7. Install Starship to your existing Airflow Deployment
  8. Deploy the Astro Project to the Astro Deployment with astro deploy
  9. In the Airflow UI of the source Airflow deployment, navigate to the new Astronomer menu and select the Migration Tool 🚀 option
  10. Follow the UI prompts to migrate, or if needed, look at the instructions to use the Operator

Compatability

Source Compatible
Airflow 1 ❌
GCC 1 - Airflow 2.x Operator
GCC 2 - Airflow 2.x ✅
MWAA v2.0.2 Operator
MWAA ≥ v2.2.2 ✅
OSS Airflow VM ✅
Astronomer Products ✅

FAQ

  • I'm on Airflow 1, can I use Starship?

    No, Starship is only compatible with Airflow 2.x and above, see Compatibility

  • I'm on Airflow>=2.7 and can't test connections?

You must have AIRFLOW__CORE__TEST_CONNECTION set. See notes here

  • I'm using Google Cloud Composer 2.x and Airflow 2.x and do not see the Astronomer menu and/or the Starship Airflow Plugin?

    Run the following to ensure you are a privileged user. 

    gcloud config set project <PROJECT_NAME>
gcloud composer environments run <ENVIRONMENT_NAME> --location <LOCATION> users add-role -- -e <USER_EMAIL> -r Admin

Security Notice

This project is an Airflow Plugin that adds custom API routes. Ensure your environments are correctly secured.

Artwork Starship logo by Lorenzo used with permission from The Noun Project under Creative Commons.