Durable.Telemetry (Durable v1.0.0)
View SourceThe :telemetry events Durable emits. This list is a contract: names,
measurements and metadata keys only change in a major version.
Spans follow :telemetry.span/3: a :start event, then exactly one of
:stop or :exception, with duration in native time units — the same
shape as Phoenix, Ecto and Oban, so existing tooling reads them as-is.
Run, step, compensation and schedule events include :durable (the
instance name) in their metadata.
Runs
A run executes in segments: it starts, runs until it finishes or suspends (sleep, wait for an event or input, parallel fan-out), and later resumes as a new segment, possibly on another node. Each segment is one span.
[:durable, :workflow, :start]—%{system_time, monotonic_time}; metadata:workflow_id,:workflow,:workflow_module,:queue,:trigger,:resumed(false on a run's first segment)[:durable, :workflow, :stop]—%{duration, monotonic_time}; adds:status::completed,:failed,:compensated,:compensation_failed,:waiting(suspended) or:superseded(cancelled or reclaimed by another node mid-segment)[:durable, :workflow, :exception]— the executor itself crashed; adds:kind,:reason,:stacktrace
Steps
One span per attempt. A step that raises or returns an error is a normal
:stop with result: :error — :exception means Durable itself failed.
[:durable, :step, :start]— metadata:workflow_id,:workflow,:step,:step_type,:attempt,:max_attempts[:durable, :step, :stop]— adds:result(:ok,:erroror:suspended) and, for errors,:error(%{type, message})[:durable, :step, :exception][:durable, :step, :retry]—%{backoff_ms}; the step metadata plus:error, emitted before sleeping for the next attempt
Compensations
[:durable, :compensation, :start | :stop | :exception]— metadata:workflow_id,:workflow,:compensation,:step(the step it undoes);:stopadds:result(:okor:error)
Queue, scheduler and waits
[:durable, :queue, :job_claimed]—%{count};:job_id,:queue,:node_id[:durable, :queue, :job_completed]—%{duration};:job_id,:queue,:workflow_module,:workflow_name,:status[:durable, :queue, :poll]—%{jobs_fetched};:queue,:available_slots[:durable, :queue, :heartbeat]—%{count};:job_id,:queue[:durable, :queue, :stale_recovered],[:durable, :queue, :zombie_recovered]—%{count}[:durable, :queue, :worker_fenced]—%{count, duration};:job_id,:queue[:durable, :queue, :ack_failed]—%{count, attempts};:job_id,:reason[:durable, :schedule, :trigger]—%{count};:schedule,:workflow_id,:workflow_module,:workflow,:queue[:durable, :wait, :sleep_woken]—%{count}[:durable, :pruner, :pruned]—%{count};:group(:completed,:failedor:cancelled), once per deleted batch
Reporting
metrics/0 returns Telemetry.Metrics definitions for any reporter
(Prometheus, StatsD, LiveDashboard) when :telemetry_metrics is installed:
def metrics, do: my_metrics() ++ Durable.Telemetry.metrics()attach_default_logger/1 logs one line per finished run and per failed step.
Summary
Functions
Logs one line per finished run (level, default :info) and one warning
per failed step attempt. Detach with :telemetry.detach("durable-default-logger").
Every event name Durable emits, for :telemetry.attach_many/4.
Telemetry.Metrics definitions for Durable's events. Tags are bounded —
workflow, step, queue and schedule names, never run ids.
Functions
@spec attach_default_logger(keyword()) :: :ok | {:error, :already_exists}
Logs one line per finished run (level, default :info) and one warning
per failed step attempt. Detach with :telemetry.detach("durable-default-logger").
@spec events() :: [[atom()]]
Every event name Durable emits, for :telemetry.attach_many/4.
@spec metrics() :: [Telemetry.Metrics.t()]
Telemetry.Metrics definitions for Durable's events. Tags are bounded —
workflow, step, queue and schedule names, never run ids.