Document telemetry state serialization controls

Author: mdrideout

docs/hooks.rst

@@ -83,6 +83,12 @@ Hook callbacks receive one immutable event object. Useful fields include:
 * ``event.error``: original exception on failure hooks
 * ``event.reason``: cancellation reason on cancellation hooks
 
+Hook event state payloads are Python-side lifecycle data, not serialized
+OpenTelemetry payloads. ``event.state`` is a detached copied state object for
+in-process inspection inside your callback. That means telemetry-oriented state
+serialization choices such as field exclusion or truncation do not
+automatically change what appears on ``event.state``.
+
 For ``on_state_changed``, the executable identity fields describe the active
 workflow, subflow, node, or concurrent executable that actually triggered the
 state update. The parent executable fields describe the containing execution

docs/opentelemetry.rst

@@ -177,6 +177,80 @@ below:
 Cancelled spans do not set ``error.type`` and are not marked with ``Error``
 status unless they actually fail.
 
+State Serialization And Telemetry
+=================================
+
+Junjo intentionally records rich workflow state in telemetry by default. This
+is a debugging-oriented design choice: many AI workflows need full prompts,
+tool inputs, tool outputs, and intermediate state to be visible in traces.
+
+Workflow state telemetry is derived from your state model's normal Pydantic
+serialization:
+
+- ``junjo.workflow.state.start`` and ``junjo.workflow.state.end`` use the
+  serialized state JSON
+- ``junjo.state_json_patch`` is built from serialized before/after state dumps
+
+This means your state model controls what appears in OpenTelemetry state
+payloads. If you want to exclude, redact, or truncate fields for telemetry,
+shape that behavior in your state model serialization.
+
+This does **not** apply to ``junjo.workflow.execution_graph_snapshot``, which
+is generated from the compiled graph rather than from state serialization.
+
+Controlling Telemetry State Payloads
+------------------------------------
+
+If you need to keep a field in runtime state but remove it from serialized
+telemetry payloads, exclude it from Pydantic serialization:
+
+.. code-block:: python
+
+    from pydantic import Field
+    from junjo import BaseState
+
+
+    class ChatWorkflowState(BaseState):
+        user_message: str
+        llm_response: str | None = None
+        raw_api_key: str | None = Field(default=None, exclude=True)
+
+In this example, ``raw_api_key`` remains available in runtime state, but it is
+omitted from serialized OpenTelemetry state snapshots and JSON patches.
+
+If you want to keep a field but truncate or reshape it for telemetry, use a
+serializer on the state model:
+
+.. code-block:: python
+
+    from pydantic import field_serializer
+    from junjo import BaseState
+
+
+    class PromptWorkflowState(BaseState):
+        prompt: str
+        final_answer: str | None = None
+
+        @field_serializer("prompt")
+        def serialize_prompt_for_telemetry(self, value: str) -> str:
+            if len(value) <= 2000:
+                return value
+            return value[:2000] + "...[truncated]"
+
+In this example, runtime state still holds the full prompt, but Junjo's
+OpenTelemetry state fields and patches use the truncated serialized form.
+
+Hook Events Use Copied State Objects
+------------------------------------
+
+Hook event state payloads are separate from OpenTelemetry serialization.
+
+- OpenTelemetry state fields use serialized state from your model
+- hook ``event.state`` values use a copied in-memory state object
+
+So excluding or truncating a field for telemetry serialization does **not**
+automatically remove it from ``event.state`` inside hook callbacks.
+
 Workflow/Subflow Span Attributes
 ---------------------------------
 

docs/state_management.rst

@@ -34,6 +34,34 @@ The `BaseState` class, which is a Pydantic `BaseModel`, is used to define the st
 
 In this example, we've defined a state for a chat application. Any workflow that uses this state will have access to these fields, and Pydantic will ensure that the data conforms to the specified types.
 
+Because Junjo uses your state's normal Pydantic serialization for workflow
+state telemetry, this is also the place where you control telemetry-facing
+serialization behavior. If a field should be excluded, redacted, or truncated
+for OpenTelemetry state payloads, implement that at the state model layer.
+
+.. code-block:: python
+
+    from pydantic import Field, field_serializer
+    from junjo import BaseState
+
+
+    class AgentState(BaseState):
+        prompt: str
+        provider_api_key: str | None = Field(default=None, exclude=True)
+
+        @field_serializer("prompt")
+        def serialize_prompt_for_telemetry(self, value: str) -> str:
+            if len(value) <= 2000:
+                return value
+            return value[:2000] + "...[truncated]"
+
+In this example:
+
+- ``provider_api_key`` stays in runtime state but is omitted from serialized
+  telemetry state
+- ``prompt`` stays complete in runtime state but is truncated in serialized
+  telemetry state
+
 BaseStore: Managing Your State
 ==============================