Skip to content

Commit 1c5206e

Browse files
committed
docs: docs update on slice surface
1 parent 8400728 commit 1c5206e

3 files changed

Lines changed: 53 additions & 105 deletions

File tree

docs/reference/src/core/Flowthru.Core/Flowthru.Flows.FlowSliceStrategy.md

Lines changed: 30 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -31,17 +31,23 @@ execution validity. All slicing operations preserve the runnability guarantee:
3131
the resulting sub-DAG must be executable without missing dependencies.
3232
</p>
3333
<p>
34+
Because a Flowthru flow is a bipartite graph of steps and catalog items, all slice
35+
targets are addressed uniformly by labelwhether the label belongs to a step or a
36+
catalog item. The resolver checks the step index first; if no step matches, it falls
37+
back to the catalog item index and resolves to the relevant producer or consumer steps.
38+
</p>
39+
<p>
3440
<strong>Slicing Strategies:</strong>
3541
</p>
36-
<ul><li><strong>flows:</strong> Filter to nodes from specific named flows (in merged DAGs)</li><li><strong>FromNodes:</strong> Include specified nodes and all downstream dependents</li><li><strong>ToNodes:</strong> Include specified nodes and all upstream dependencies (run "up to" these nodes)</li><li><strong>FromData:</strong> Include nodes consuming specified catalog entries and all downstream dependents</li><li><strong>ToData:</strong> Include nodes producing specified catalog entries and all upstream dependencies</li><li><strong>OnlyNodes:</strong> Explicit allowlist of nodes plus minimal required dependencies</li></ul>
42+
<ul><li><strong>Flows:</strong> Filter to nodes from specific named flows (in merged DAGs)</li><li><strong>From:</strong> Include specified nodes and all downstream dependents. Accepts step labels or catalog item labels (resolves to consumers).</li><li><strong>To:</strong> Include specified nodes and all upstream dependencies. Accepts step labels or catalog item labels (resolves to producer).</li><li><strong>Only:</strong> Explicit allowlist plus minimal required dependencies. Accepts step labels or catalog item labels (resolves to producer).</li></ul>
3743
<p>
3844
<strong>Composition:</strong> Multiple strategies compose via intersection (additive filtering).
39-
For example, <code>--from-nodes A --to-data B</code> produces nodes in the downstream dependency
40-
tree of A that are also required to produce data B.
45+
For example, <code>--from A --to B</code> produces nodes in the downstream dependency
46+
tree of A that are also required to produce B.
4147
</p>
4248
<p>
4349
<strong>Runnability Guarantee:</strong> Slicing operations are additive only. Subtractive
44-
operations (<code>--from-nodes A --except B</code>) would break the runnability guarantee and
50+
operations (<code>--from A --except B</code>) would break the runnability guarantee and
4551
are not supported.
4652
</p>
4753

@@ -61,35 +67,16 @@ public IReadOnlySet<string>? Flows { get; init; }
6167

6268
#### Remarks
6369

64-
In merged flows, nodes are prefixed with their Flow name (e.g., "DataScience.TrainModel").
65-
This filter includes only nodes from the specified flows.
70+
In merged flows, steps are prefixed with their Flow name (e.g., "DataScience.TrainModel").
71+
This filter includes only steps from the specified flows.
6672
Flow names are case-insensitive.
6773

68-
### <a id="Flowthru_Flows_FlowSliceStrategy_FromData"></a> FromData
69-
70-
Start from nodes that consume these catalog entry labels, including all downstream dependents.
71-
72-
```csharp
73-
public IReadOnlySet<string>? FromData { get; init; }
74-
```
75-
76-
#### Property Value
77-
78-
[IReadOnlySet](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlyset\-1)<[string](https://learn.microsoft.com/dotnet/api/system.string)\>?
79-
80-
#### Remarks
81-
82-
Finds all nodes that read the specified catalog entries, then expands downstream.
83-
Useful for impact analysis - "what breaks if I change this data?"
84-
85-
TODO: Remove this, as it is now sufficient to reference both steps and data entries as nodes.
86-
87-
### <a id="Flowthru_Flows_FlowSliceStrategy_FromNodes"></a> FromNodes
74+
### <a id="Flowthru_Flows_FlowSliceStrategy_From"></a> From
8875

8976
Start from these nodes, including all downstream dependents.
9077

9178
```csharp
92-
public IReadOnlySet<string>? FromNodes { get; init; }
79+
public IReadOnlySet<string>? From { get; init; }
9380
```
9481

9582
#### Property Value
@@ -98,8 +85,10 @@ public IReadOnlySet<string>? FromNodes { get; init; }
9885

9986
#### Remarks
10087

101-
Expands to include all nodes that depend on these nodes (transitively).
102-
Useful for impact analysis - "what breaks if I change this node?"
88+
Each label is resolved against the step index first. If no step matches, the label is
89+
treated as a catalog item and resolved to all steps that consume it.
90+
Expands to include all transitively dependent steps.
91+
Useful for impact analysis: "what is affected if I change this step or item?"
10392

10493
### <a id="Flowthru_Flows_FlowSliceStrategy_IsSliced"></a> IsSliced
10594

@@ -113,29 +102,12 @@ public bool IsSliced { get; }
113102

114103
[bool](https://learn.microsoft.com/dotnet/api/system.boolean)
115104

116-
### <a id="Flowthru_Flows_FlowSliceStrategy_OnlyNodes"></a> OnlyNodes
117-
118-
Explicit allowlist of node names (dependencies auto-included).
119-
120-
```csharp
121-
public IReadOnlySet<string>? OnlyNodes { get; init; }
122-
```
123-
124-
#### Property Value
125-
126-
[IReadOnlySet](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlyset\-1)<[string](https://learn.microsoft.com/dotnet/api/system.string)\>?
127-
128-
#### Remarks
105+
### <a id="Flowthru_Flows_FlowSliceStrategy_Only"></a> Only
129106

130-
Specifies exactly which nodes to execute, then automatically includes any
131-
required dependencies to maintain DAG validity.
132-
133-
### <a id="Flowthru_Flows_FlowSliceStrategy_ToData"></a> ToData
134-
135-
End at nodes that produce these catalog entry labels, including all upstream dependencies.
107+
Explicit allowlist of nodes (dependencies auto-included).
136108

137109
```csharp
138-
public IReadOnlySet<string>? ToData { get; init; }
110+
public IReadOnlySet<string>? Only { get; init; }
139111
```
140112

141113
#### Property Value
@@ -144,17 +116,16 @@ public IReadOnlySet<string>? ToData { get; init; }
144116

145117
#### Remarks
146118

147-
Finds the nodes that write the specified catalog entries, then expands upstream.
148-
Useful for targeted execution - "run everything needed to produce this data".
149-
150-
TODO: Remove this, as it is now sufficient to reference both steps and data entries as nodes.
119+
Each label is resolved against the step index first. If no step matches, the label is
120+
treated as a catalog item and resolved to the step that produces it.
121+
Automatically includes all transitive upstream dependencies to maintain DAG validity.
151122

152-
### <a id="Flowthru_Flows_FlowSliceStrategy_ToNodes"></a> ToNodes
123+
### <a id="Flowthru_Flows_FlowSliceStrategy_To"></a> To
153124

154125
End at these nodes, including all upstream dependencies needed to produce them.
155126

156127
```csharp
157-
public IReadOnlySet<string>? ToNodes { get; init; }
128+
public IReadOnlySet<string>? To { get; init; }
158129
```
159130

160131
#### Property Value
@@ -163,9 +134,11 @@ public IReadOnlySet<string>? ToNodes { get; init; }
163134

164135
#### Remarks
165136

166-
Expands to include all transitive dependencies needed to run these nodes.
137+
Each label is resolved against the step index first. If no step matches, the label is
138+
treated as a catalog item and resolved to the step that produces it.
139+
Expands to include all transitive dependencies.
167140
Equivalent to "run everything up to and including these nodes".
168-
Useful for testing specific outputs without running the entire flow.
141+
Useful for targeted execution: "run everything needed to produce this step or item".
169142

170143
## Methods
171144

docs/reference/src/core/Flowthru.Core/Flowthru.Meta.Models.DagSliceMetadata.md

Lines changed: 22 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Namespace: [Flowthru.Meta.Models](Flowthru.Meta.Models.md)
44
Assembly: Flowthru.Core.dll
55

6-
Metadata describing how a pipeline was sliced during execution.
6+
Metadata describing how a flow was sliced during execution.
77

88
```csharp
99
public class DagSliceMetadata
@@ -26,90 +26,65 @@ public class DagSliceMetadata
2626
2727
## Remarks
2828

29-
Captures the criteria used to select a subset of nodes from the full pipeline DAG.
29+
Captures the criteria used to select a subset of steps from the full flow DAG.
3030
This information is essential for:
31-
<ul><li>Reproducibility - rerun the exact same slice</li><li>Debugging - understand what was included/excluded when failures occur</li><li>Auditing - track which pipeline subsets were executed in production</li><li>Visualization - indicate sliced vs. full DAG in metadata exports</li></ul>
31+
<ul><li>Reproducibility - rerun the exact same slice</li><li>Debugging - understand what was included/excluded when failures occur</li><li>Auditing - track which flow subsets were executed in production</li><li>Visualization - indicate sliced vs. full DAG in metadata exports</li></ul>
3232

3333
## Properties
3434

35-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_FromData"></a> FromData
35+
### <a id="Flowthru_Meta_Models_DagSliceMetadata_Flows"></a> Flows
3636

37-
Catalog entry labels whose consumers are included (expanded downstream).
37+
Flow names used to filter the merged DAG.
3838

3939
```csharp
40-
[JsonPropertyName("fromData")]
40+
[JsonPropertyName("flows")]
4141
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
42-
public string[]? FromData { get; init; }
42+
public string[]? Flows { get; init; }
4343
```
4444

4545
#### Property Value
4646

4747
[string](https://learn.microsoft.com/dotnet/api/system.string)\[\]?
4848
49-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_FromNodes"></a> FromNodes
49+
### <a id="Flowthru_Meta_Models_DagSliceMetadata_From"></a> From
5050

51-
Node names from which the slice expanded downstream (dependents included).
51+
Node labels from which the slice expanded downstream. Each label may be a step
52+
label or a catalog item label (resolved to its consumer steps).
5253

5354
```csharp
54-
[JsonPropertyName("fromNodes")]
55+
[JsonPropertyName("from")]
5556
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
56-
public string[]? FromNodes { get; init; }
57+
public string[]? From { get; init; }
5758
```
5859

5960
#### Property Value
6061

6162
[string](https://learn.microsoft.com/dotnet/api/system.string)\[\]?
6263
63-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_OnlyNodes"></a> OnlyNodes
64+
### <a id="Flowthru_Meta_Models_DagSliceMetadata_Only"></a> Only
6465

65-
Explicit allowlist of node names (with dependencies auto-included).
66+
Explicit allowlist of node labels (with upstream dependencies auto-included). Each label
67+
may be a step label or a catalog item label (resolved to its producer step).
6668

6769
```csharp
68-
[JsonPropertyName("onlyNodes")]
70+
[JsonPropertyName("only")]
6971
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
70-
public string[]? OnlyNodes { get; init; }
72+
public string[]? Only { get; init; }
7173
```
7274

7375
#### Property Value
7476

7577
[string](https://learn.microsoft.com/dotnet/api/system.string)\[\]?
7678
77-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_Pipelines"></a> Pipelines
79+
### <a id="Flowthru_Meta_Models_DagSliceMetadata_To"></a> To
7880

79-
Pipeline names to include in the merged DAG.
81+
Node labels to which the slice expanded upstream. Each label may be a step
82+
label or a catalog item label (resolved to its producer step).
8083

8184
```csharp
82-
[JsonPropertyName("pipelines")]
85+
[JsonPropertyName("to")]
8386
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
84-
public string[]? Pipelines { get; init; }
85-
```
86-
87-
#### Property Value
88-
89-
[string](https://learn.microsoft.com/dotnet/api/system.string)\[\]?
90-
91-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_ToData"></a> ToData
92-
93-
Catalog entry labels whose producers are included (expanded upstream).
94-
95-
```csharp
96-
[JsonPropertyName("toData")]
97-
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
98-
public string[]? ToData { get; init; }
99-
```
100-
101-
#### Property Value
102-
103-
[string](https://learn.microsoft.com/dotnet/api/system.string)\[\]?
104-
105-
### <a id="Flowthru_Meta_Models_DagSliceMetadata_ToNodes"></a> ToNodes
106-
107-
Node names to which the slice expanded upstream (dependencies included).
108-
109-
```csharp
110-
[JsonPropertyName("toNodes")]
111-
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
112-
public string[]? ToNodes { get; init; }
87+
public string[]? To { get; init; }
11388
```
11489

11590
#### Property Value

docs/reference/src/core/Flowthru.Core/Flowthru.Meta.Models.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Root metadata model representing a complete FlowthruService DAG (Directed Acycli
88

99
[DagSliceMetadata](Flowthru.Meta.Models.DagSliceMetadata.md)
1010

11-
Metadata describing how a pipeline was sliced during execution.
11+
Metadata describing how a flow was sliced during execution.
1212

1313
[EdgeMetadata](Flowthru.Meta.Models.EdgeMetadata.md)
1414

0 commit comments

Comments
 (0)