Skip to content

Include notebooks in generated documentation site#57

Merged
zmek merged 12 commits intoUCL-CORU:mainfrom
matt-graham:mmg/notebooks-in-docs
Apr 1, 2025
Merged

Include notebooks in generated documentation site#57
zmek merged 12 commits intoUCL-CORU:mainfrom
matt-graham:mmg/notebooks-in-docs

Conversation

@matt-graham
Copy link
Copy Markdown
Contributor

Uses nbconvert to convert Jupyter notebooks in notebooks directory to Markdown files under docs/notebooks and runs a script docs/notebooks/update_notebook_links.py to change internal links between notebooks to point to .md Markdown files rather than .ipynb files. Also updates the link in docs/notebooks/README.md to be relative and fixes a few broken internal links in notebooks due to inconsistent capitalisation.

As alternative to the manual approach here which is quite hacky some combination of the mkdocs-jupyter plugin, mkdocs-awesome-nav plugin (for including directory of files in navigation without individually listing files) and a MkDocs hook for dealing with link conversion might be cleaner.

@matt-graham matt-graham requested a review from zmek as a code owner March 27, 2025 17:12
@zmek
Copy link
Copy Markdown
Collaborator

zmek commented Apr 1, 2025

Hi @matt-graham - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.

Are you suggesting this could be a better solution but would require some further development:

some combination of the mkdocs-jupyter plugin, mkdocs-awesome-nav plugin (for including directory of files in navigation without individually listing files) and a MkDocs hook for dealing with link conversion might be cleaner.

In the meantime, I've changed all links to be absolute links - another hacky solution.

Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673

@matt-graham
Copy link
Copy Markdown
Contributor Author

matt-graham commented Apr 1, 2025

Hi @matt-graham - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.

Ah sorry I'd forgotten when testing this locally I had copied across the notebooks/README.md file to docs/notebooks to replace the placeholder files which directly includes, now added a step to tox docs environment which does as part of tox -e docs command and links seem to work locally for me.

Are you suggesting this could be a better solution but would require some further development:

some combination of the mkdocs-jupyter plugin, mkdocs-awesome-nav plugin (for including directory of files in navigation without individually listing files) and a MkDocs hook for dealing with link conversion might be cleaner.

In general relying on plugins would probably be a nicer option than doing something manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example mkdocs-jupyter-plugin seems to only directly support having notebooks listed at top-level of navigation tree.

Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673

These were being generated I think as you are pinning the mkdocstrings version and its dependencies - I assume possibly because of an error you might have got when a new version was released as we updated the template recently because of this. I've applied those changes locally here and removed the version pins in pyproject.toml and the docs now build without errors and without the deprecation warnings. There are still warnings because some of the notebooks have unrecognised relative links, a few missing type annotations and the notebooks not being directly included in navigation (probably could update the script to add these to mkdocs.yml automatically or use the mkdocs-awesome-nav plugin for this mentioned above)

WARNING -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
             - notebooks/0_Set_up_your_environment.md
             - notebooks/1_Meet_the_users_of_our_predictions.md
             - notebooks/2a_Create_patient_snapshots.md
             - notebooks/2b_Create_group_snapshots.md
             - notebooks/3_Explore_the_datasets.md
             - notebooks/4_Specify_emergency_demand_model.md
             - notebooks/4a_Predict_probability_of_admission_from_ED.md
             - notebooks/4b_Predict_demand_from_patients_in_ED.md
             - notebooks/4c_Predict_probability_of_admission_to_specialty.md
             - notebooks/4d_Predict_demand_from_patients_yet_to_arrive.md
             - notebooks/4e_Predict_probabiity_of_admission_using_minimal_data.md
             - notebooks/4f_Bring_it_all_together.md
             - notebooks/5_Evaluate_model_performance.md
             - notebooks/7_Visualise_un-delayed_demand.md
WARNING -  griffe: src/patientflow/prepare.py:319: No type or annotation for returned value 'tuple'
WARNING -  griffe: src/patientflow/prepare.py:300: No type or annotation for returned value 'dict'
WARNING -  griffe: src/patientflow/prepare.py:291: No type or annotation for returned value 'bool'
WARNING -  griffe: src/patientflow/prepare.py:275: No type or annotation for returned value 'bool'
WARNING -  Doc file 'notebooks/0_Set_up_your_environment.md' contains an absolute link '/README.md', it was left as is.
WARNING -  Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING -  Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING -  Doc file 'notebooks/3_Explore_the_datasets.md' contains a link '../data-dictionaries/ed_visits_data_dictionary.csv', but the target 'data-dictionaries/ed_visits_data_dictionary.csv' is not found among documentation files.
WARNING -  Doc file 'notebooks/4f_Bring_it_all_together.md' contains an unrecognized relative link '../trained-models/', it was left as is.

@zmek
Copy link
Copy Markdown
Collaborator

zmek commented Apr 1, 2025 via email

@zmek
Copy link
Copy Markdown
Collaborator

zmek commented Apr 1, 2025 via email

@zmek zmek merged commit e51d434 into UCL-CORU:main Apr 1, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants