Add "Edit on GitHub" links

[abitrolly]

Allow to easily jump to documentation page contents in repository.

The explanation is here https://stackoverflow.com/a/62904217
Related readthedocs/sphinx_rtd_theme#1635

Pull Request Checklist

• implement the feature
• write the documentation
• extend the test coverage
• update the specification
• adjust plugin docstring
• modify the json schema
• mention the version
• include a release note

[happz]

Good idea, but it's not going to be that simple. It will work for pages that exist as ReST files in the repo, and that's just a subset of pages Sphinx consumes and renders. Specification, stories, plugins, plus a couple more, all these are generated either from Python sources or, in the case of large content in specification and stories, from fmf files. So, a simple link to $PAGE.rst will not work for these because the source file literally does not exist in the repository, and is conjured from thin air by make -C docs generate run by ReadTheDocs.

https://tmt--3398.org.readthedocs.build/en/3398/spec/core.html -> 404.

Edit: on top of that, one page generated by Sphinx is often constructed from multiple fmf stories, multiple fmf files. "Edit on Github" should then most likely point to a directory rather than a single file. https://tmt.readthedocs.io/en/stable/spec/core.html -> https://github.com/teemtee/tmt/tree/main/spec/core

[abitrolly]

Template uses Sphinx hasdoc function to check if the page has source. Possible solution is to let Sphinx generate the docs with extension, like https://www.sphinx-doc.org/en/master/tutorial/automatic-doc-generation.html

Or save a copy of generated sources in the repo.

[abitrolly]

All right. The feature can be turned on and off on page by page basis. Setting display_github to False on generated pages should do this. Now I need to find the place where to patch this in page generator. It is also possible to generate exact URL and set github_url.

[abitrolly]

https://tmt--3398.org.readthedocs.build/en/3398/spec/core.html is generated from https://github.com/teemtee/tmt/tree/main/spec/core but I don't see what generates it https://github.com/teemtee/tmt/blob/main/docs/Makefile

EDIT: Okay, it is generated by https://github.com/teemtee/tmt/blob/main/docs/scripts/generate-stories.py

[happz]

https://tmt--3398.org.readthedocs.build/en/3398/spec/core.html is generated from https://github.com/teemtee/tmt/tree/main/spec/core but I don't see what generates it https://github.com/teemtee/tmt/blob/main/docs/Makefile

https://github.com/teemtee/tmt/blob/main/docs/Makefile#L133:

$ make docs
...
mkdir -p spec
scripts/generate-lint-checks.py templates/lint-checks.rst.j2 spec/lint.rst
scripts/generate-template-filters.py templates/template-filters.rst.j2 code/template-filters.rst
scripts/generate-plugins.py discover templates/plugins.rst.j2 plugins/discover.rst
scripts/generate-plugins.py provision templates/plugins.rst.j2 plugins/provision.rst
scripts/generate-plugins.py prepare templates/plugins.rst.j2 plugins/prepare.rst
scripts/generate-plugins.py execute templates/plugins.rst.j2 plugins/execute.rst
scripts/generate-plugins.py finish templates/plugins.rst.j2 plugins/finish.rst
scripts/generate-plugins.py report templates/plugins.rst.j2 plugins/report.rst
warn: report/reportportal.log_size_limit: could not render default value, '1 MB'
warn: report/reportportal.traceback_size_limit: could not render default value, '50 kB'
scripts/generate-plugins.py test-checks templates/plugins.rst.j2 plugins/test-checks.rst
scripts/generate-hardware-matrix.py templates/hardware-matrix.rst.j2 plugins/hardware-matrix.rst
Generating reST file for HW requirement support matrix
mkdir -p stories
scripts/generate-stories.py templates/story.rst.j2
Generating rst files from /stories/docs
Generating rst files from /stories/cli
Generating rst files from /stories/install
Generating rst files from /stories/features
Generating rst files from /stories/deferred
Generating rst files from /spec/core
Generating rst files from /spec/tests
Generating rst files from /spec/plans
Generating rst files from /spec/stories
Generating rst files from /spec/context
Generating rst files from /spec/hardware

[martinhoyer]

It would be cool to have it as part of #3088
Also with a matrix room button.
Just thinking aloud ;)

[abitrolly]

@martinhoyer it would be nice to get proper standard for "Edit on GitHub" links in Spinx, but folks on @readthedocs probably have different priorities. I would be happy to take this as a freelance job, just to focus.

[abitrolly]

@happz spec link works https://tmt--3398.org.readthedocs.build/en/3398/spec/core.html

[happz]

1. The URL is invalid for forks and every branch beyond main
2. Some areas are directories, some are fmf files. It's not important to fmf, which can handle both correctly, but here we would need to make a distinction. https://github.com/teemtee/tmt/blob/main/stories/install does not exist, https://github.com/teemtee/tmt/blob/main/stories/install.fmf does.

Every story object should have web_link() method, which returns a link to the corresponding fmf file, that would be perfect, but that's for "edit on github" per section rather than per page. My initial idea would be to collect all web links for the area stories, and find their common, shared prefix and use that one. I don't like it, but right now I don't have a better one.

[abitrolly]

The URL points to main, but it should be valid as long as file path in repo doesn't change. Does it makes sense to propose edits to branches?

For .fmf vs directory, the generate script knows which template to invoke, and it can set the URL correctly. What is needed, is the list of this generated entries to check, and generators that produce them. To make sure nothing falls apart.

[graphite-app]

The :github_url: metadata field is missing a newline character at the end. Consider modifying the string to include \n at the end:

doc.write(
    f':github_url: https://github.com/teemtee/tmt/blob/main{area}\n'
)

This ensures proper separation between the metadata field and subsequent content in the generated RST file, preventing potential formatting issues in the documentation.

Suggested change

	doc.write(
	f':github_url: https://github.com/teemtee/tmt/blob/main{area}'
	)
	doc.write(
	f':github_url: https://github.com/teemtee/tmt/blob/main{area}\n'
	)

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

[LecrisUT]

What is the state of this? I am working on refactoring the sphinx implementation which automatically adds the reference to the original docstrings as best as it can. Isn't there a sphinx extension that can hook into those and add the element to the individual items?

I can help with this feature if you can help me figure out how to create that edit element.

[abitrolly]

@LecrisUT the problem is missing web_link() method that should help trace generated content back to its source.

[LecrisUT]

@LecrisUT the problem is missing web_link() method that should help trace generated content back to its source.

On the tmt side? Don't worry about it, I am gradually introducing explicit source links to the files for individual sections in #4288. The things I don't know are if having links to the local git repo is enough and how can we customize it and tweak it to our intricacies. E.g. we would want to make the edit button point differently for each section or object because the sources are quite scattered, similar to autodoc.

[abitrolly]

On the tmt side?

Ideally, it should be on Sphinx side, and then incorporated into theme. If the page is composed of multiple sources, I would list them all.

[LecrisUT]

On the tmt side?

Ideally, it should be on Sphinx side, and then incorporated into theme. If the page is composed of multiple sources, I would list them all.

I mean the missing web_link() is on tmt side? We have ways of constructing that link manually, but I would rather have some helpers on the sphinx side to translate local paths to urls instead. Dealing with the commit/branch ref to construct the url on our side would not be useful for an "edit on Github" link I believe, but sphinx should have all the tools to do it on their end if it has the file paths wouldn't it? Plus it would then work for editing locally from something like sphinx-autobuild as well

[abitrolly]

@LecrisUT the web_link(s)() should be on Sphinx side. Provided by HTML5 generator. Sphinx should preserve the build DAG for that to be possible. But, Spinx doesn't know that files on your filesystem are actually hosted on GitHub that have specific URLs for editing. This is what theme does, and its setting.

[LecrisUT]

Again sorry for the long silence here. The sphinx integration is still being worked on, and maybe with the introduction of the new way of handling stories, I hope things can work more easily out-of-the-box.

Regarding this change, I think we are actually good to go to have it in the current form. Eventually all generated files should go away, and it should work fine. If you can rebase this to keep only the conf.py changes, I think we are good to go. It would not work perfectly, but we can (and should) do more incremental improvements.

For the long-term vision, I have an idea on how to fix the linkage issue from the tinkering done over this. We already add some html objects like in

tmt/docs/ext/tmt_domain/story.py

Line 65 in 887a8ac

target_node = nodes.target(ids=[name, compat_target])

We can add one of them to be the edit button and point it to the correct source, because we can extract it (or rather a good enough approximation of it) when building these. If you know some references to check for a similar design, or if there are some extensions to work on top of, that would help. This has some strong overlap with how sphinx.ext.autodoc works, so I do wonder if there is no work being done in that area.