Home

Astronomer Orbiter can land legacy workloads safely down in a new home on Apache Airflow!

What is Orbiter?

Orbiter is both a CLI and Framework for converting workflows from other orchestration tools to Apache Airflow.

Generally it can be thoughts of as:

flowchart LR
    origin{{ XML/JSON/YAML/Etc Workflows }}
    origin -->| ✨ Translations ✨ | airflow{{ Apache Airflow Project }}

Orbiter is primarily a set of Rules and Objects that can translate workflows from an Origin system to Airflow-native python code.

Installation

Contact an Astronomer representative to get access to the Orbiter binary or package.

pip install astronomer_orbiter-X.Y.Z-py3-none-any.whl

Usage

Quickstart

You can see more specifics on how to use Orbiter in the CLI section.

Supported Origins

The list of systems Orbiter has support for translating is listed at Origins

Customization

Orbiter can be extended to fit specific needs, patterns, or get support for additional origins or specific translations.

Read more specifics about how to use the framework at Rules and Objects

Example

1. Set up a folder like:

   .
   ├── override.py
   └── workflow
       └── my-sample-workflow.json
   

2. With contents like:
   override.py

   from __future__ import annotations
   from orbiter.objects import OrbiterRequirement
   from orbiter.objects.task import OrbiterTask
   from orbiter.rules import rule
   from orbiter.translator.control_m.json import translation_ruleset
   
   
   # A higher priority means it will be applied before other rules
   @rule(priority=99)
   def my_empty_operator_rule(key, val, global_dict) -> OrbiterTask | None:
       """Demonstration of overriding rulesets"""
       if "Job:" in val.get("Type", ""):
   
           # This creates an EmptyOperator (as it's the first import that has Operator or Sensor in the name)
           # with a task_id
           return OrbiterTask(
               imports=[
                   OrbiterRequirement(
                       package="apache-airflow",
                       module=" airflow.operators.empty",
                       names=["EmptyOperator"],
                   ),
               ],
               task_id=key,
           )
       else:
           return None
   
   
   #  add a custom rule to the default ruleset
   translation_ruleset.task_ruleset.ruleset.append(my_empty_operator_rule)
   

3. Invoke with your customization:

   orbiter translate workflow output -r override.translation_ruleset
   

FAQ

• Can this tool convert my workflows from tool X to Airflow?

  If you don't see your tool listed in Supported Origins, you can create an issue to request support for it. We also happily accept pull requests.

• Are the results of this tool under any guarantee of correctness?

  No. This tool is provided as-is, with no guarantee of correctness. It is your responsibility to verify the results. We accept Pull Requests for improving base parsing logic and strive to make the tool easily configurable to handle your specific use-case.

---

Artwork Orbiter logo by Ivan Colic used with permission from The Noun Project under Creative Commons.