Include notebooks in generated documentation site#57
Conversation
|
Hi @matt-graham - I've tried running this. It seems like the links in the documentation are still broken in the documentation site. Are you suggesting this could be a better solution but would require some further development:
In the meantime, I've changed all links to be absolute links - another hacky solution. Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs |
Ah sorry I'd forgotten when testing this locally I had copied across the
In general relying on plugins would probably be a nicer option than doing something manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example
These were being generated I think as you are pinning the |
|
Ah OK I see. Yes, that does build correctly for me now.
I've tried to install the awesome-pages plug in. I added mkdocs-awesome-pages-plugin>=2.8.0 in requirements.txt and it shows up in the pip list. I changed mkdocs.yml, but I'm getting an error when I try to build the docs.
docs: commands[1]> cp /Users/zellaking/Repos/patientflow/notebooks/README.md /Users/zellaking/Repos/patientflow/docs/notebooks/README.md
docs: commands[2]> python /Users/zellaking/Repos/patientflow/docs/notebooks/update_notebook_links.py /Users/zellaking/Repos/patientflow/docs/notebooks
docs: commands[3]> mkdocs build
ERROR - Config value 'plugins': The "awesome-pages" plugin is not installed
I tried to add these changes to your pull request but I didn't follow the right steps (probably just as well) so I've ended up creating a branch called pr-57 https://github.com/UCL-CORU/patientflow/tree/pr-57. I will delete that branch once you've looked at it.
Thanks
Zella
…________________________________
From: Matt Graham ***@***.***>
Sent: 01 April 2025 09:55
To: UCL-CORU/patientflow ***@***.***>
Cc: King, Zella ***@***.***>; Review requested ***@***.***>
Subject: Re: [UCL-CORU/patientflow] Include notebooks in generated documentation site (PR #57)
⚠ Caution: External sender
Hi @matt-graham<https://github.com/matt-graham> - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.
Ah sorry I'd forgotten when testing this locally I had copied across the notebooks/README.md file to docs/notebooks to replace the placeholder files which directly includes, now added a step to tox docs environment which does as part of tox -e docs command and links seem to work locally for me.
Are you suggesting this could be a better solution but would require some further development:
some combination of the mkdocs-jupyter plugin<https://github.com/danielfrg/mkdocs-jupyter>, mkdocs-awesome-nav plugin<https://lukasgeiter.github.io/mkdocs-awesome-nav/> (for including directory of files in navigation without individually listing files) and a MkDocs hook<https://www.mkdocs.org/user-guide/configuration/#hooks> for dealing with link conversion might be cleaner.
In general relying on plugins would probably be a nicer option than doing some manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example mkdocs-jupyter-plugin seems to only directly support having notebooks listed at top-level of navigation tree.
Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673
These were being generated I think as you are pinning the mkdocstrings version and its dependencies - I assume possibly because of an error you might have got when a new version was released as we updated the template recently<UCL-ARC/python-tooling@925fce9> because of this. I've applied those changes locally here and removed the version pins in pyproject.toml and the docs now build without errors and without the deprecation warnings. There are still warnings because some of the notebooks have unrecognised relative links, a few missing type annotations and the notebooks not being directly included in navigation (probably could update the script to add these to mkdocs.yml automatically or use the mkdocs-awesome-nav plugin for this mentioned above)
WARNING - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- notebooks/0_Set_up_your_environment.md
- notebooks/1_Meet_the_users_of_our_predictions.md
- notebooks/2a_Create_patient_snapshots.md
- notebooks/2b_Create_group_snapshots.md
- notebooks/3_Explore_the_datasets.md
- notebooks/4_Specify_emergency_demand_model.md
- notebooks/4a_Predict_probability_of_admission_from_ED.md
- notebooks/4b_Predict_demand_from_patients_in_ED.md
- notebooks/4c_Predict_probability_of_admission_to_specialty.md
- notebooks/4d_Predict_demand_from_patients_yet_to_arrive.md
- notebooks/4e_Predict_probabiity_of_admission_using_minimal_data.md
- notebooks/4f_Bring_it_all_together.md
- notebooks/5_Evaluate_model_performance.md
- notebooks/7_Visualise_un-delayed_demand.md
WARNING - griffe: src/patientflow/prepare.py:319: No type or annotation for returned value 'tuple'
WARNING - griffe: src/patientflow/prepare.py:300: No type or annotation for returned value 'dict'
WARNING - griffe: src/patientflow/prepare.py:291: No type or annotation for returned value 'bool'
WARNING - griffe: src/patientflow/prepare.py:275: No type or annotation for returned value 'bool'
WARNING - Doc file 'notebooks/0_Set_up_your_environment.md' contains an absolute link '/README.md', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains a link '../data-dictionaries/ed_visits_data_dictionary.csv', but the target 'data-dictionaries/ed_visits_data_dictionary.csv' is not found among documentation files.
WARNING - Doc file 'notebooks/4f_Bring_it_all_together.md' contains an unrecognized relative link '../trained-models/', it was left as is.
—
Reply to this email directly, view it on GitHub<#57 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACKYPEOLTYUQH7BIQPPLHGT2XJIB3AVCNFSM6AAAAABZ5R2UZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRYGY2TSNBRG4>.
You are receiving this because your review was requested.Message ID: ***@***.***>
[matt-graham]matt-graham left a comment (UCL-CORU/patientflow#57)<#57 (comment)>
Hi @matt-graham<https://github.com/matt-graham> - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.
Ah sorry I'd forgotten when testing this locally I had copied across the notebooks/README.md file to docs/notebooks to replace the placeholder files which directly includes, now added a step to tox docs environment which does as part of tox -e docs command and links seem to work locally for me.
Are you suggesting this could be a better solution but would require some further development:
some combination of the mkdocs-jupyter plugin<https://github.com/danielfrg/mkdocs-jupyter>, mkdocs-awesome-nav plugin<https://lukasgeiter.github.io/mkdocs-awesome-nav/> (for including directory of files in navigation without individually listing files) and a MkDocs hook<https://www.mkdocs.org/user-guide/configuration/#hooks> for dealing with link conversion might be cleaner.
In general relying on plugins would probably be a nicer option than doing some manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example mkdocs-jupyter-plugin seems to only directly support having notebooks listed at top-level of navigation tree.
Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673
These were being generated I think as you are pinning the mkdocstrings version and its dependencies - I assume possibly because of an error you might have got when a new version was released as we updated the template recently<UCL-ARC/python-tooling@925fce9> because of this. I've applied those changes locally here and removed the version pins in pyproject.toml and the docs now build without errors and without the deprecation warnings. There are still warnings because some of the notebooks have unrecognised relative links, a few missing type annotations and the notebooks not being directly included in navigation (probably could update the script to add these to mkdocs.yml automatically or use the mkdocs-awesome-nav plugin for this mentioned above)
WARNING - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- notebooks/0_Set_up_your_environment.md
- notebooks/1_Meet_the_users_of_our_predictions.md
- notebooks/2a_Create_patient_snapshots.md
- notebooks/2b_Create_group_snapshots.md
- notebooks/3_Explore_the_datasets.md
- notebooks/4_Specify_emergency_demand_model.md
- notebooks/4a_Predict_probability_of_admission_from_ED.md
- notebooks/4b_Predict_demand_from_patients_in_ED.md
- notebooks/4c_Predict_probability_of_admission_to_specialty.md
- notebooks/4d_Predict_demand_from_patients_yet_to_arrive.md
- notebooks/4e_Predict_probabiity_of_admission_using_minimal_data.md
- notebooks/4f_Bring_it_all_together.md
- notebooks/5_Evaluate_model_performance.md
- notebooks/7_Visualise_un-delayed_demand.md
WARNING - griffe: src/patientflow/prepare.py:319: No type or annotation for returned value 'tuple'
WARNING - griffe: src/patientflow/prepare.py:300: No type or annotation for returned value 'dict'
WARNING - griffe: src/patientflow/prepare.py:291: No type or annotation for returned value 'bool'
WARNING - griffe: src/patientflow/prepare.py:275: No type or annotation for returned value 'bool'
WARNING - Doc file 'notebooks/0_Set_up_your_environment.md' contains an absolute link '/README.md', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains a link '../data-dictionaries/ed_visits_data_dictionary.csv', but the target 'data-dictionaries/ed_visits_data_dictionary.csv' is not found among documentation files.
WARNING - Doc file 'notebooks/4f_Bring_it_all_together.md' contains an unrecognized relative link '../trained-models/', it was left as is.
—
Reply to this email directly, view it on GitHub<#57 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACKYPEOLTYUQH7BIQPPLHGT2XJIB3AVCNFSM6AAAAABZ5R2UZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRYGY2TSNBRG4>.
You are receiving this because your review was requested.Message ID: ***@***.***>
|
|
Cancel that - my stupid mistake - hadn't saved requirements.txt.
Now fixed.
…________________________________
From: King, Zella ***@***.***>
Sent: 01 April 2025 10:54
To: UCL-CORU/patientflow ***@***.***>
Subject: Re: [UCL-CORU/patientflow] Include notebooks in generated documentation site (PR #57)
Ah OK I see. Yes, that does build correctly for me now.
I've tried to install the awesome-pages plug in. I added mkdocs-awesome-pages-plugin>=2.8.0 in requirements.txt and it shows up in the pip list. I changed mkdocs.yml, but I'm getting an error when I try to build the docs.
docs: commands[1]> cp /Users/zellaking/Repos/patientflow/notebooks/README.md /Users/zellaking/Repos/patientflow/docs/notebooks/README.md
docs: commands[2]> python /Users/zellaking/Repos/patientflow/docs/notebooks/update_notebook_links.py /Users/zellaking/Repos/patientflow/docs/notebooks
docs: commands[3]> mkdocs build
ERROR - Config value 'plugins': The "awesome-pages" plugin is not installed
I tried to add these changes to your pull request but I didn't follow the right steps (probably just as well) so I've ended up creating a branch called pr-57 https://github.com/UCL-CORU/patientflow/tree/pr-57. I will delete that branch once you've looked at it.
Thanks
Zella
________________________________
From: Matt Graham ***@***.***>
Sent: 01 April 2025 09:55
To: UCL-CORU/patientflow ***@***.***>
Cc: King, Zella ***@***.***>; Review requested ***@***.***>
Subject: Re: [UCL-CORU/patientflow] Include notebooks in generated documentation site (PR #57)
⚠ Caution: External sender
Hi @matt-graham<https://github.com/matt-graham> - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.
Ah sorry I'd forgotten when testing this locally I had copied across the notebooks/README.md file to docs/notebooks to replace the placeholder files which directly includes, now added a step to tox docs environment which does as part of tox -e docs command and links seem to work locally for me.
Are you suggesting this could be a better solution but would require some further development:
some combination of the mkdocs-jupyter plugin<https://github.com/danielfrg/mkdocs-jupyter>, mkdocs-awesome-nav plugin<https://lukasgeiter.github.io/mkdocs-awesome-nav/> (for including directory of files in navigation without individually listing files) and a MkDocs hook<https://www.mkdocs.org/user-guide/configuration/#hooks> for dealing with link conversion might be cleaner.
In general relying on plugins would probably be a nicer option than doing some manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example mkdocs-jupyter-plugin seems to only directly support having notebooks listed at top-level of navigation tree.
Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673
These were being generated I think as you are pinning the mkdocstrings version and its dependencies - I assume possibly because of an error you might have got when a new version was released as we updated the template recently<UCL-ARC/python-tooling@925fce9> because of this. I've applied those changes locally here and removed the version pins in pyproject.toml and the docs now build without errors and without the deprecation warnings. There are still warnings because some of the notebooks have unrecognised relative links, a few missing type annotations and the notebooks not being directly included in navigation (probably could update the script to add these to mkdocs.yml automatically or use the mkdocs-awesome-nav plugin for this mentioned above)
WARNING - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- notebooks/0_Set_up_your_environment.md
- notebooks/1_Meet_the_users_of_our_predictions.md
- notebooks/2a_Create_patient_snapshots.md
- notebooks/2b_Create_group_snapshots.md
- notebooks/3_Explore_the_datasets.md
- notebooks/4_Specify_emergency_demand_model.md
- notebooks/4a_Predict_probability_of_admission_from_ED.md
- notebooks/4b_Predict_demand_from_patients_in_ED.md
- notebooks/4c_Predict_probability_of_admission_to_specialty.md
- notebooks/4d_Predict_demand_from_patients_yet_to_arrive.md
- notebooks/4e_Predict_probabiity_of_admission_using_minimal_data.md
- notebooks/4f_Bring_it_all_together.md
- notebooks/5_Evaluate_model_performance.md
- notebooks/7_Visualise_un-delayed_demand.md
WARNING - griffe: src/patientflow/prepare.py:319: No type or annotation for returned value 'tuple'
WARNING - griffe: src/patientflow/prepare.py:300: No type or annotation for returned value 'dict'
WARNING - griffe: src/patientflow/prepare.py:291: No type or annotation for returned value 'bool'
WARNING - griffe: src/patientflow/prepare.py:275: No type or annotation for returned value 'bool'
WARNING - Doc file 'notebooks/0_Set_up_your_environment.md' contains an absolute link '/README.md', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains a link '../data-dictionaries/ed_visits_data_dictionary.csv', but the target 'data-dictionaries/ed_visits_data_dictionary.csv' is not found among documentation files.
WARNING - Doc file 'notebooks/4f_Bring_it_all_together.md' contains an unrecognized relative link '../trained-models/', it was left as is.
—
Reply to this email directly, view it on GitHub<#57 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACKYPEOLTYUQH7BIQPPLHGT2XJIB3AVCNFSM6AAAAABZ5R2UZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRYGY2TSNBRG4>.
You are receiving this because your review was requested.Message ID: ***@***.***>
[matt-graham]matt-graham left a comment (UCL-CORU/patientflow#57)<#57 (comment)>
Hi @matt-graham<https://github.com/matt-graham> - I've tried running this. It seems like the links in the documentation are still broken in the documentation site.
Ah sorry I'd forgotten when testing this locally I had copied across the notebooks/README.md file to docs/notebooks to replace the placeholder files which directly includes, now added a step to tox docs environment which does as part of tox -e docs command and links seem to work locally for me.
Are you suggesting this could be a better solution but would require some further development:
some combination of the mkdocs-jupyter plugin<https://github.com/danielfrg/mkdocs-jupyter>, mkdocs-awesome-nav plugin<https://lukasgeiter.github.io/mkdocs-awesome-nav/> (for including directory of files in navigation without individually listing files) and a MkDocs hook<https://www.mkdocs.org/user-guide/configuration/#hooks> for dealing with link conversion might be cleaner.
In general relying on plugins would probably be a nicer option than doing some manual as I'm suggesting here but what I wasn't sure is if you can actually do what you want with what the plugins support. For example mkdocs-jupyter-plugin seems to only directly support having notebooks listed at top-level of navigation tree.
Now, the only documentation warnings are deprecation warnings: DeprecationWarning: autorefs span elements are deprecated in favor of autoref elements. see https://github.com/UCL-CORU/patientflow/actions/runs/14190818876/job/39754883673
These were being generated I think as you are pinning the mkdocstrings version and its dependencies - I assume possibly because of an error you might have got when a new version was released as we updated the template recently<UCL-ARC/python-tooling@925fce9> because of this. I've applied those changes locally here and removed the version pins in pyproject.toml and the docs now build without errors and without the deprecation warnings. There are still warnings because some of the notebooks have unrecognised relative links, a few missing type annotations and the notebooks not being directly included in navigation (probably could update the script to add these to mkdocs.yml automatically or use the mkdocs-awesome-nav plugin for this mentioned above)
WARNING - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- notebooks/0_Set_up_your_environment.md
- notebooks/1_Meet_the_users_of_our_predictions.md
- notebooks/2a_Create_patient_snapshots.md
- notebooks/2b_Create_group_snapshots.md
- notebooks/3_Explore_the_datasets.md
- notebooks/4_Specify_emergency_demand_model.md
- notebooks/4a_Predict_probability_of_admission_from_ED.md
- notebooks/4b_Predict_demand_from_patients_in_ED.md
- notebooks/4c_Predict_probability_of_admission_to_specialty.md
- notebooks/4d_Predict_demand_from_patients_yet_to_arrive.md
- notebooks/4e_Predict_probabiity_of_admission_using_minimal_data.md
- notebooks/4f_Bring_it_all_together.md
- notebooks/5_Evaluate_model_performance.md
- notebooks/7_Visualise_un-delayed_demand.md
WARNING - griffe: src/patientflow/prepare.py:319: No type or annotation for returned value 'tuple'
WARNING - griffe: src/patientflow/prepare.py:300: No type or annotation for returned value 'dict'
WARNING - griffe: src/patientflow/prepare.py:291: No type or annotation for returned value 'bool'
WARNING - griffe: src/patientflow/prepare.py:275: No type or annotation for returned value 'bool'
WARNING - Doc file 'notebooks/0_Set_up_your_environment.md' contains an absolute link '/README.md', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains an unrecognized relative link '../data-dictionaries/', it was left as is.
WARNING - Doc file 'notebooks/3_Explore_the_datasets.md' contains a link '../data-dictionaries/ed_visits_data_dictionary.csv', but the target 'data-dictionaries/ed_visits_data_dictionary.csv' is not found among documentation files.
WARNING - Doc file 'notebooks/4f_Bring_it_all_together.md' contains an unrecognized relative link '../trained-models/', it was left as is.
—
Reply to this email directly, view it on GitHub<#57 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACKYPEOLTYUQH7BIQPPLHGT2XJIB3AVCNFSM6AAAAABZ5R2UZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRYGY2TSNBRG4>.
You are receiving this because your review was requested.Message ID: ***@***.***>
|
Uses
nbconvertto convert Jupyter notebooks innotebooksdirectory to Markdown files underdocs/notebooksand runs a scriptdocs/notebooks/update_notebook_links.pyto change internal links between notebooks to point to.mdMarkdown files rather than.ipynbfiles. Also updates the link indocs/notebooks/README.mdto be relative and fixes a few broken internal links in notebooks due to inconsistent capitalisation.As alternative to the manual approach here which is quite hacky some combination of the
mkdocs-jupyterplugin,mkdocs-awesome-navplugin (for including directory of files in navigation without individually listing files) and a MkDocs hook for dealing with link conversion might be cleaner.