> ## Documentation Index
> Fetch the complete documentation index at: https://docs.squasher.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Durable workflow traces

> Model retries, checkpoints, pauses, and resumptions without one long-running span.

Durable workflows can pause for minutes or days, resume in another process,
and retry one step several times. Do not keep one span open for the complete
workflow lifetime. Long spans are difficult to flush, can be lost when a
process stops, and do not describe retries clearly.

Use finite spans for active work, stable workflow attributes for search, and
span links when work resumes from an earlier execution context.

## Recommended model

Create one trace for each active execution attempt or short execution burst.
Record these attributes on its root span and important child spans:

| Attribute                         | Purpose                                      |
| --------------------------------- | -------------------------------------------- |
| `squasher.workflow.id`            | Stable workflow definition or logical name   |
| `squasher.workflow.run_id`        | Stable identifier for one workflow run       |
| `squasher.workflow.step`          | Stable step name                             |
| `squasher.workflow.attempt`       | Numeric attempt, starting at `1`             |
| `squasher.workflow.checkpoint_id` | Stable checkpoint or resume token identifier |

These are Squasher conventions for custom OpenTelemetry attributes. Do not put
credentials, full payloads, or customer content in them.

Use a low-cardinality value for `squasher.workflow.id` and
`squasher.workflow.step`. Use `run_id` and `checkpoint_id` only for direct
search, not dashboard grouping.

## Parent versus link

Use a parent-child relationship when work continues in one live causal call
and you can propagate the normal trace context.

Use a span link when:

* a retry starts after the earlier process or trace ended
* a checkpoint resumes in a new runtime instance
* one step combines several messages or upstream runs
* one completed step starts several independent branches

A span has one parent but can have several links. Squasher preserves OTLP span
links and shows them in the span **Links** tab.

## Retry example

The first attempt can end with an error. The next attempt starts a new trace
and links its root span to the failed attempt:

```typescript theme={null}
const retrySpan = tracer.startSpan("workflow.step", {
  attributes: {
    "squasher.workflow.id": "invoice-delivery",
    "squasher.workflow.run_id": workflowRunId,
    "squasher.workflow.step": "send-email",
    "squasher.workflow.attempt": 2,
    "squasher.workflow.checkpoint_id": checkpointId,
  },
  links: [
    {
      context: previousAttemptSpanContext,
      attributes: { "squasher.link.reason": "retry" },
    },
  ],
});
```

Use your OpenTelemetry SDK's normal context and link types. The exact constructor
names can differ by language.

## Checkpoints and state changes

Record a span event when a checkpoint occurs during an active span:

```typescript theme={null}
span.addEvent("workflow.checkpoint", {
  "squasher.workflow.checkpoint_id": checkpointId,
  "squasher.workflow.step": "await-approval",
});
```

If no span is active, emit a structured log with the workflow run, step, and
checkpoint attributes. Link the next execution span to the last stored span
context when it is safe and available.

Do not create a fake parent when the earlier context is missing. Stable workflow
attributes still let you find the related attempts.

## Status and errors

* End each attempt span when that attempt stops running.
* Set span status to `ERROR` when an attempt fails.
* Record `error.type` and a safe error message.
* Record the final workflow outcome as a structured event or finite completion span.
* Keep timeout, cancellation, retry, and terminal failure as different outcomes.

## Verify

1. Run one workflow that pauses and resumes.
2. Run one workflow that fails once and then succeeds.
3. Search by `squasher.workflow.run_id` and confirm every attempt appears.
4. Open the retry span and confirm its link opens the earlier attempt.
5. Confirm that no span remains open only because the workflow is waiting.

## References

* [OpenTelemetry span links](https://opentelemetry.io/docs/specs/otel/overview/#links-between-spans)
* [OpenTelemetry messaging trace structure](https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/#trace-structure)

## Agent handoff

```text theme={null}
Instrument durable workflow <workflow_name> with finite OpenTelemetry spans. Use stable workflow and run attributes, create one span for each active attempt, record checkpoints as events, link retries or resumed work to the prior span context, do not invent a parent, and verify one pause/resume and one failed retry in Squasher.
```