Skip to content

Handle specialties at inference #142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zmek merged 5 commits into from
Mar 2, 2026
Merged

Handle specialties at inference #142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bbb0e90
add aspirational flag to FlowInputs
zmek Feb 23, 2026
1ae1eba
limit inference specialties to training specialties
zmek Feb 23, 2026
2ec3323
correct specialty of admission plot
zmek Feb 25, 2026
22840da
Merge remote-tracking branch 'origin/main' into evaluate-by-service
zmek Mar 2, 2026
a95acc5
ruff reforat
zmek Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 65 additions & 48 deletions docs/notebooks/4a_Organise_predictions_for_a_production_pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ This notebook generates fake data on-the-fly, so you can run it immediately with
%autoreload 2
```

The autoreload extension is already loaded. To reload it, use:
%reload_ext autoreload

## Generate fake data and train simple models

We reuse the fake data generation from notebooks 3a and 3c. First, fake ED snapshots for training an admission probability model:
Expand Down Expand Up @@ -111,9 +114,10 @@ yta_model.fit(

Split sizes: [2214, 710, 1584]

<style>#sk-container-id-1 {
<style>#sk-container-id-2 {
/* Definition of color scheme common for light and dark mode */
--sklearn-color-text: black;
--sklearn-color-text: #000;
--sklearn-color-text-muted: #666;
--sklearn-color-line: gray;
/* Definition of color scheme for unfitted estimators */
--sklearn-color-unfitted-level-0: #fff5e6;
Expand Down Expand Up @@ -141,15 +145,15 @@ yta_model.fit(
}
}

#sk-container-id-1 {
#sk-container-id-2 {
color: var(--sklearn-color-text);
}

#sk-container-id-1 pre {
#sk-container-id-2 pre {
padding: 0;
}

#sk-container-id-1 input.sk-hidden--visually {
#sk-container-id-2 input.sk-hidden--visually {
border: 0;
clip: rect(1px 1px 1px 1px);
clip: rect(1px, 1px, 1px, 1px);
Expand All @@ -161,15 +165,15 @@ yta_model.fit(
width: 1px;
}

#sk-container-id-1 div.sk-dashed-wrapped {
#sk-container-id-2 div.sk-dashed-wrapped {
border: 1px dashed var(--sklearn-color-line);
margin: 0 0.4em 0.5em 0.4em;
box-sizing: border-box;
padding-bottom: 0.4em;
background-color: var(--sklearn-color-background);
}

#sk-container-id-1 div.sk-container {
#sk-container-id-2 div.sk-container {
/* jupyter's `normalize.less` sets `[hidden] { display: none; }`
but bootstrap.min.css set `[hidden] { display: none !important; }`
so we also need the `!important` here to be able to override the
Expand All @@ -179,7 +183,7 @@ yta_model.fit(
position: relative;
}

#sk-container-id-1 div.sk-text-repr-fallback {
#sk-container-id-2 div.sk-text-repr-fallback {
display: none;
}

Expand All @@ -195,43 +199,43 @@ div.sk-item {

/* Parallel-specific style estimator block */

#sk-container-id-1 div.sk-parallel-item::after {
#sk-container-id-2 div.sk-parallel-item::after {
content: "";
width: 100%;
border-bottom: 2px solid var(--sklearn-color-text-on-default-background);
flex-grow: 1;
}

#sk-container-id-1 div.sk-parallel {
#sk-container-id-2 div.sk-parallel {
display: flex;
align-items: stretch;
justify-content: center;
background-color: var(--sklearn-color-background);
position: relative;
}

#sk-container-id-1 div.sk-parallel-item {
#sk-container-id-2 div.sk-parallel-item {
display: flex;
flex-direction: column;
}

#sk-container-id-1 div.sk-parallel-item:first-child::after {
#sk-container-id-2 div.sk-parallel-item:first-child::after {
align-self: flex-end;
width: 50%;
}

#sk-container-id-1 div.sk-parallel-item:last-child::after {
#sk-container-id-2 div.sk-parallel-item:last-child::after {
align-self: flex-start;
width: 50%;
}

#sk-container-id-1 div.sk-parallel-item:only-child::after {
#sk-container-id-2 div.sk-parallel-item:only-child::after {
width: 0;
}

/* Serial-specific style estimator block */

#sk-container-id-1 div.sk-serial {
#sk-container-id-2 div.sk-serial {
display: flex;
flex-direction: column;
align-items: center;
Expand All @@ -249,38 +253,47 @@ clickable and can be expanded/collapsed.

/* Pipeline and ColumnTransformer style (default) */

#sk-container-id-1 div.sk-toggleable {
#sk-container-id-2 div.sk-toggleable {
/* Default theme specific background. It is overwritten whether we have a
specific estimator or a Pipeline/ColumnTransformer */
background-color: var(--sklearn-color-background);
}

/* Toggleable label */
#sk-container-id-1 label.sk-toggleable__label {
#sk-container-id-2 label.sk-toggleable__label {
cursor: pointer;
display: block;
display: flex;
width: 100%;
margin-bottom: 0;
padding: 0.5em;
box-sizing: border-box;
text-align: center;
align-items: start;
justify-content: space-between;
gap: 0.5em;
}

#sk-container-id-1 label.sk-toggleable__label-arrow:before {
#sk-container-id-2 label.sk-toggleable__label .caption {
font-size: 0.6rem;
font-weight: lighter;
color: var(--sklearn-color-text-muted);
}

#sk-container-id-2 label.sk-toggleable__label-arrow:before {
/* Arrow on the left of the label */
content: "▸";
float: left;
margin-right: 0.25em;
color: var(--sklearn-color-icon);
}

#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {
#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {
color: var(--sklearn-color-text);
}

/* Toggleable content - dropdown */

#sk-container-id-1 div.sk-toggleable__content {
#sk-container-id-2 div.sk-toggleable__content {
max-height: 0;
max-width: 0;
overflow: hidden;
Expand All @@ -289,92 +302,92 @@ clickable and can be expanded/collapsed.
background-color: var(--sklearn-color-unfitted-level-0);
}

#sk-container-id-1 div.sk-toggleable__content.fitted {
#sk-container-id-2 div.sk-toggleable__content.fitted {
/* fitted */
background-color: var(--sklearn-color-fitted-level-0);
}

#sk-container-id-1 div.sk-toggleable__content pre {
#sk-container-id-2 div.sk-toggleable__content pre {
margin: 0.2em;
border-radius: 0.25em;
color: var(--sklearn-color-text);
/* unfitted */
background-color: var(--sklearn-color-unfitted-level-0);
}

#sk-container-id-1 div.sk-toggleable__content.fitted pre {
#sk-container-id-2 div.sk-toggleable__content.fitted pre {
/* unfitted */
background-color: var(--sklearn-color-fitted-level-0);
}

#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {
#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {
/* Expand drop-down */
max-height: 200px;
max-width: 100%;
overflow: auto;
}

#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {
#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {
content: "▾";
}

/* Pipeline/ColumnTransformer-specific style */

#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {
#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {
color: var(--sklearn-color-text);
background-color: var(--sklearn-color-unfitted-level-2);
}

#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
background-color: var(--sklearn-color-fitted-level-2);
}

/* Estimator-specific style */

/* Colorize estimator box */
#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {
#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {
/* unfitted */
background-color: var(--sklearn-color-unfitted-level-2);
}

#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
/* fitted */
background-color: var(--sklearn-color-fitted-level-2);
}

#sk-container-id-1 div.sk-label label.sk-toggleable__label,
#sk-container-id-1 div.sk-label label {
#sk-container-id-2 div.sk-label label.sk-toggleable__label,
#sk-container-id-2 div.sk-label label {
/* The background is the default theme color */
color: var(--sklearn-color-text-on-default-background);
}

/* On hover, darken the color of the background */
#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {
#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {
color: var(--sklearn-color-text);
background-color: var(--sklearn-color-unfitted-level-2);
}

/* Label box, darken color on hover, fitted */
#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {
#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {
color: var(--sklearn-color-text);
background-color: var(--sklearn-color-fitted-level-2);
}

/* Estimator label */

#sk-container-id-1 div.sk-label label {
#sk-container-id-2 div.sk-label label {
font-family: monospace;
font-weight: bold;
display: inline-block;
line-height: 1.2em;
}

#sk-container-id-1 div.sk-label-container {
#sk-container-id-2 div.sk-label-container {
text-align: center;
}

/* Estimator-specific */
#sk-container-id-1 div.sk-estimator {
#sk-container-id-2 div.sk-estimator {
font-family: monospace;
border: 1px dotted var(--sklearn-color-border-box);
border-radius: 0.25em;
Expand All @@ -384,18 +397,18 @@ clickable and can be expanded/collapsed.
background-color: var(--sklearn-color-unfitted-level-0);
}

#sk-container-id-1 div.sk-estimator.fitted {
#sk-container-id-2 div.sk-estimator.fitted {
/* fitted */
background-color: var(--sklearn-color-fitted-level-0);
}

/* on hover */
#sk-container-id-1 div.sk-estimator:hover {
#sk-container-id-2 div.sk-estimator:hover {
/* unfitted */
background-color: var(--sklearn-color-unfitted-level-2);
}

#sk-container-id-1 div.sk-estimator.fitted:hover {
#sk-container-id-2 div.sk-estimator.fitted:hover {
/* fitted */
background-color: var(--sklearn-color-fitted-level-2);
}
Expand All @@ -416,7 +429,8 @@ a:visited.sk-estimator-doc-link {
height: 1em;
width: 1em;
text-decoration: none !important;
margin-left: 1ex;
margin-left: 0.5em;
text-align: center;
/* unfitted */
border: var(--sklearn-color-unfitted-level-1) 1pt solid;
color: var(--sklearn-color-unfitted-level-1);
Expand Down Expand Up @@ -482,7 +496,7 @@ div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,

/* "?"-specific style due to the `<a>` HTML tag */

#sk-container-id-1 a.estimator_doc_link {
#sk-container-id-2 a.estimator_doc_link {
float: right;
font-size: 1rem;
line-height: 1em;
Expand All @@ -497,25 +511,25 @@ div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,
border: var(--sklearn-color-unfitted-level-1) 1pt solid;
}

#sk-container-id-1 a.estimator_doc_link.fitted {
#sk-container-id-2 a.estimator_doc_link.fitted {
/* fitted */
border: var(--sklearn-color-fitted-level-1) 1pt solid;
color: var(--sklearn-color-fitted-level-1);
}

/* On hover */
#sk-container-id-1 a.estimator_doc_link:hover {
#sk-container-id-2 a.estimator_doc_link:hover {
/* unfitted */
background-color: var(--sklearn-color-unfitted-level-3);
color: var(--sklearn-color-background);
text-decoration: none;
}

#sk-container-id-1 a.estimator_doc_link.fitted:hover {
#sk-container-id-2 a.estimator_doc_link.fitted:hover {
/* fitted */
background-color: var(--sklearn-color-fitted-level-3);
}
</style><div id="sk-container-id-1" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>ParametricIncomingAdmissionPredictor(filters={})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-1" type="checkbox" checked><label for="sk-estimator-id-1" class="sk-toggleable__label sk-toggleable__label-arrow ">&nbsp;ParametricIncomingAdmissionPredictor<span class="sk-estimator-doc-link ">i<span>Not fitted</span></span></label><div class="sk-toggleable__content "><pre>ParametricIncomingAdmissionPredictor(filters={})</pre></div> </div></div></div></div>
</style><div id="sk-container-id-2" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>ParametricIncomingAdmissionPredictor(filters={})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-2" type="checkbox" checked><label for="sk-estimator-id-2" class="sk-toggleable__label sk-toggleable__label-arrow"><div><div>ParametricIncomingAdmissionPredictor</div></div><div><span class="sk-estimator-doc-link ">i<span>Not fitted</span></span></div></label><div class="sk-toggleable__content "><pre>ParametricIncomingAdmissionPredictor(filters={})</pre></div> </div></div></div></div>

We also need the aspirational curve parameters for the parametric model. These are loaded from the config file in the repository:

Expand Down Expand Up @@ -596,11 +610,12 @@ This is exactly the kind of result we produced in the 3x\_ notebooks — a numpy

## Step 2: Wrap the PMF in a `FlowInputs` container

A `FlowInputs` object is a lightweight, immutable container that holds a single source of patient flow. It adds three pieces of metadata to the raw distribution:
A `FlowInputs` object is a lightweight, immutable container that holds a single source of patient flow. It adds metadata to the raw distribution:

- **`flow_id`** — a unique identifier (e.g. `"ed_current"`)
- **`flow_type`** — either `"pmf"` or `"poisson"`, so downstream code knows how to handle it
- **`display_name`** — an optional human-readable label
- **`aspirational`** — whether the flow reflects demand under target performance assumptions rather than empirically observed patterns (default `False`; see notebook 4d for evaluation implications)

The distribution itself is stored in the `distribution` attribute — exactly the same numpy array we just plotted.

Expand Down Expand Up @@ -766,6 +781,8 @@ print(f'Distribution: {ed_yta_flow.distribution} (Poisson lambda)')

Both PMF and Poisson flows use the same `FlowInputs` container, distinguished only by `flow_type`. This uniform interface means downstream code can handle any flow without special-casing.

Because we used a `ParametricIncomingAdmissionPredictor` here, the Poisson rate reflects demand as if ED performance targets are met — an _aspirational_ prediction. When `build_service_data()` constructs this flow automatically, it sets `aspirational=True` on the `ed_yta` `FlowInputs` so that downstream code can detect this. See notebook 4d for how this affects evaluation.

## Step 4: Group flows into a `ServicePredictionInputs`

A hospital service (e.g. "medical") has multiple sources of incoming patients and potentially outgoing patients too. `ServicePredictionInputs` groups all of these into a single object, organised as **inflows** and **outflows**.
Expand Down Expand Up @@ -997,7 +1014,7 @@ Every call to `predictor.predict_service()` returns a `PredictionBundle`. This b
- **departures** — the combined outflow distribution
- **net_flow** — the difference (arrivals minus departures)

It also records which `FlowSelection` was used, so you can always trace what went into the prediction.
It also records which `FlowSelection` was used, so you can always trace what went into the prediction. The bundle's **`is_aspirational`** flag is `True` when any contributing inflow was generated under aspirational assumptions (e.g. ED performance targets), indicating that comparison against observed admissions requires care — see notebook 4d.

```python
print(combined_bundle)
Expand Down
Loading
Loading