Operators

Note

These operators are included and are intended to represent some of the most common Airflow Operators, but not all Airflow Operators.

Additional Operators can be created by subclassing OrbiterOperator or using OrbiterTask directly.

Review the Astronomer Registry to find additional Airflow Operators.

Modules:

Name	Description
`bash`
`empty`
`kubernetes_pod`
`livy`
`python`
`smtp`
`sql`
`ssh`

orbiter.objects.operators.bash ¶

Classes:

Name	Description
`OrbiterBashOperator`	An Airflow

orbiter.objects.operators.bash.OrbiterBashOperator ¶

Bases: OrbiterOperator

An Airflow BashOperator. Used to run shell commands.

>>> OrbiterBashOperator(task_id="foo", bash_command="echo 'hello world'")
foo_task = BashOperator(task_id='foo', bash_command="echo 'hello world'")

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`bash_command`	`str`	The shell command to execute
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.empty ¶

Classes:

Name	Description
`OrbiterEmptyOperator`	An Airflow EmptyOperator. Does nothing.

orbiter.objects.operators.empty.OrbiterEmptyOperator ¶

Bases: OrbiterOperator

An Airflow EmptyOperator. Does nothing.

>>> OrbiterEmptyOperator(task_id="foo")
foo_task = EmptyOperator(task_id='foo')

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.kubernetes_pod ¶

Classes:

Name	Description
`OrbiterKubernetesPodOperator`	An Airflow

orbiter.objects.operators.kubernetes_pod.OrbiterKubernetesPodOperator ¶

Bases: OrbiterOperator

An Airflow KubernetesPodOperator. Used to launch a Docker container in a Kubernetes cluster.

>>> OrbiterKubernetesPodOperator(
...     task_id="foo",
...     kubernetes_conn_id="KUBERNETES",
...     image="my-docker-image"
... )
foo_task = KubernetesPodOperator(task_id='foo', kubernetes_conn_id='KUBERNETES', image='my-docker-image')

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`kubernetes_conn_id`	`str, optional`	The Kubernetes connection to use. Defaults to "KUBERNETES"
`image`	`str`	The Docker image to launch
`cmds`	`list[str] \| None, optional`	The commands to run in the container, defaults container Entrypoint
`arguments`	`list[str] \| None, optional`	The arguments to pass to the commands, defaults container commands
`env_vars`	`dict[str, str] \| list[V1EnvVar] \| None, optional`	The environment variables to set in the container, defaults to None
`container_resources`	`V1ResourceRequirements \| None, optional`	The resource requirements for the container, defaults to None
`image_pull_secrets`	`list[V1LocalObjectReference] \| None, optional`	The secrets to use for pulling the Docker image, defaults to None
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.livy ¶

Classes:

Name	Description
`OrbiterLivyOperator`	An Airflow

orbiter.objects.operators.livy.OrbiterLivyOperator ¶

Bases: OrbiterOperator

An Airflow LivyOperator. Used to submit Spark Jobs.

>>> OrbiterLivyOperator(task_id="foo", file="/a/b/c.jar")
foo_task = LivyOperator(task_id='foo', file='/a/b/c.jar')

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.python ¶

Classes:

Name	Description
`OrbiterDecoratedPythonOperator`	An Airflow
`OrbiterPythonOperator`	An Airflow

orbiter.objects.operators.python.OrbiterDecoratedPythonOperator ¶

Bases: OrbiterPythonOperator

An Airflow TaskFlow @task. Used to execute any Python Function.

>>> def foo(a, b):
...    print(a + b)
>>> OrbiterDecoratedPythonOperator(task_id="foo", python_callable=foo)
@task()
def foo(a, b):
    print(a + b)

Parameters as per OrbiterPythonOperator

orbiter.objects.operators.python.OrbiterPythonOperator ¶

Bases: OrbiterOperator

An Airflow PythonOperator. Used to execute any Python Function.

>>> def foo(a, b):
...    print(a + b)
>>> OrbiterPythonOperator(task_id="foo", python_callable=foo)
def foo(a, b):
   print(a + b)
foo_task = PythonOperator(task_id='foo', python_callable=foo)

You can utilize the orbiter_includes and imports to include additional Python code and imports (or subclass to default).

>>> from orbiter.objects.include import OrbiterInclude
>>> OrbiterPythonOperator(
...     task_id="foo",
...     orbiter_includes={OrbiterInclude(filepath="include/bar.py", contents="def baz(): pass")},
...     imports=[OrbiterRequirement(module="include.bar", names=["baz"])],
...     python_callable="baz"
... )
foo_task = PythonOperator(task_id='foo', python_callable=baz)

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`python_callable`	`Callable`	The python function to execute
`op_args`	`list \| None, optional`	The arguments to pass to the python function, defaults to None
`op_kwargs`	`dict \| None, optional`	The keyword arguments to pass to the python function, defaults to None
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.smtp ¶

Classes:

Name	Description
`OrbiterEmailOperator`	An Airflow

orbiter.objects.operators.smtp.OrbiterEmailOperator ¶

Bases: OrbiterOperator

An Airflow EmailOperator. Used to send emails.

>>> OrbiterEmailOperator(
...   task_id="foo", to="humans@astronomer.io", subject="Hello", html_content="World!"
... )
foo_task = EmailOperator(task_id='foo', to='humans@astronomer.io', subject='Hello', html_content='World!', conn_id='SMTP')

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`to`	`str \| list[str]`	The recipient of the email
`subject`	`str`	The subject of the email
`html_content`	`str`	The content of the email
`files`	`list, optional`	The files to attach to the email, defaults to None
`conn_id`	`str, optional`	The SMTP connection to use. Defaults to "SMTP" and sets `orbiter_conns` property
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.sql ¶

Classes:

Name	Description
`OrbiterSQLExecuteQueryOperator`	An Airflow

orbiter.objects.operators.sql.OrbiterSQLExecuteQueryOperator ¶

Bases: OrbiterOperator

An Airflow Generic SQL Operator. Used to run SQL against any Database.

>>> OrbiterSQLExecuteQueryOperator(
...   task_id="foo", conn_id='sql', sql="select 1;"
... )
foo_task = SQLExecuteQueryOperator(task_id='foo', conn_id='sql', sql='select 1;')

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`conn_id`	`str`	The SQL connection to utilize. (Note: use the `**conn_id(...)` utility function)
`sql`	`str \| list[str]`	The SQL to execute
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties

orbiter.objects.operators.ssh ¶

Classes:

Name	Description
`OrbiterSSHOperator`	An Airflow

orbiter.objects.operators.ssh.OrbiterSSHOperator ¶

Bases: OrbiterOperator

An Airflow SSHOperator. Used to run shell commands over SSH.

>>> OrbiterSSHOperator(task_id="foo", ssh_conn_id="SSH", command="echo 'hello world'")
foo_task = SSHOperator(task_id='foo', ssh_conn_id='SSH', command="echo 'hello world'")

Parameters:

Name	Type	Description
`task_id`	`str`	The `task_id` for the operator
`ssh_conn_id`	`str`	The SSH connection to use. (Note: use the `**conn_id(...)` utility function)
`command`	`str`	The command to execute
`environment`	`dict, optional`	The environment variables to set, defaults to None
`**kwargs`		Extra arguments to pass to the operator
`**OrbiterBase`		OrbiterBase inherited properties