Trigger

class RayJobTrigger(*args: Any, **kwargs: Any)

Bases: BaseTrigger

Triggers and monitors the status of a Ray job.

This trigger periodically checks the status of a submitted job on a Ray cluster and yields events based on the job’s status. It handles timeouts and errors during the polling process.

Parameters:

• job_id – The unique identifier of the Ray job.

• conn_id – The connection ID for the Ray cluster.

• xcom_dashboard_url – Optional URL for the Ray dashboard.

• poll_interval – The interval in seconds at which to poll the job status. Defaults to 30 seconds.

• fetch_logs – Whether to fetch and stream logs. Defaults to True.

async cleanup() → None

Cleanup method to ensure resources are properly deleted. This will be called when the trigger encounters an exception.

Example scenario: A job is submitted using the @ray.task decorator with a Ray specification. After the cluster is started and the job is submitted, the trigger begins tracking its progress. However, if the job is stopped through the UI at this stage, the cluster resources are not deleted.

property hook: RayHook

Lazily initializes and returns a RayHook instance.

Returns:

An instance of RayHook configured with the connection ID and dashboard URL.

async run() → AsyncIterator[airflow.triggers.base.TriggerEvent]

Asynchronously polls the Ray job status and yields events based on the job’s state.

This method gets job status at each poll interval and streams logs if available. It yields a TriggerEvent upon job completion, cancellation, or failure.

Yield:

TriggerEvent containing the status, message, and job ID related to the job.

serialize() → tuple[str, dict[str, Any]]

Serializes the trigger’s configuration.

Returns:

A tuple containing the fully qualified class name and a dictionary of its parameters.