GitHub Docs Contribution

February 21, 2024

Understanding Quality Assurance Metrics #

Measurements and metrics are delicate, as they can be influenced by Goodhart’s law, which states that when a measure becomes a target, it ceases to be a good measure. When it comes to Quality Assurance (QA), it is crucial to pick the right metrics.

GitHub Docs: A Case Study on Doing Metrics Right #

In the case of documentation, some metrics to measure quality, like the number of pages or the frequency of changes can be misleading or lead to adverse incentives. Instead, focusing on user ratings and feedback can provide more valuable insights.

In this article, we will explore a case study of how GitHub collects metrics on its documentation. We will also discuss how you can contribute to this process by providing feedback or making direct contributions. This is important because you and other learners rely on this documentation every day, and any improvements made can have a significant impact.

What you will learn #

Through this guide, you will learn:

• How to approach improving documentation
• For practice, how to make contributions to Github Docs either via direct feedback on the page or making pull requests to the Github Docs repository

How to approach documentation feedback #

Here are some suggestions for things to report in a documentation feedback:

• Step-by-step instructions were unclear or failed at a certain step (provide specifics on which step)
• Needed more examples or visuals to understand a concept
• Encountered broken links or images
• Wanted an explanation for technical terms or jargon
• Sections were missing information or lacked necessary detail
• Code samples did not work as expected (provide specifics on errors)
• Difficult to find related or linked information
• Explanations were confusing or contradictory
• Information was outdated or no longer accurate
• Tutorial did not provide setup/prerequisites needed to follow along
• Instructions did not work for a specific use case or environment
• Unclear what audience or experience level the content was aimed at
• Hard to scan or quickly locate key pieces of info
• Design/formatting made it difficult to read or navigate

Now that you are ready to give some feedback, let’s get started viewing the options that we have to make Github Docs better!

Option 1: Proving feedback on the Github Docs Site #

On any page of documentation, such as the article “ Quickstart for GitHub Actions”, you can find a feedback section at the bottom of the page. This section includes a rating system and a text box to provide feedback.

The most straightforward option is to provide feedback directly on the page. You can rate the page and provide a comment. This feedback is then collected and analyzed by the GitHub team.

For example, for the “ Quickstart for GitHub Actions”, in the passage:

I think that attention should be taken to explain that some names are fixed (“magic names”), such as the folder .github/workflows, while other names are fully user-specified, such as the name of the workflow file. This is a common source of confusion for beginners:

For this reason, I can make this observation in the feedback box:

Once I click, the feedback is confirmed:

Option 2: Making direct contributions to the Github Docs Site repository #

The previous section was about providing feedback on the documentation. However, you can also contribute directly to the documentation by making a pull request (PR) to the GitHub repository where the documentation is hosted. This process can be initiated by clicking the “Make a contribution” button in the feedback section. (You can learn more about contributing to GitHub’s documentation here.)

Clicking the button will create a fork of the repository: This is a copy of the repository that you can modify without affecting the original. You can then make changes to the documentation, make the commits, and once you are done, you would create a pull request (PR) to propose your changes to the original repository. You can read about the process of creating a pull request from a forked repository.

When submitted a PR, you are provided with a template to complete. Any text contained in between <!-- and --> is a comment and will not be displayed in the PR, and these will often contain instructions or examples to guide you through the process. For instance, below, the template asks for a description of the changes made, and the motivation behind them, as well as invites you to self-complete a checklist of items to ensure the PR is complete and ready for review. Initially, it is recommended you submit the PR as a draft, so you can ensure it passes all the tests.

There is a battery of automated integration tests that will check your changes for quality control. It may require you to make a few changes (in which case you would simply go back to your forked repository, make the changes, and then commit them to the PR). Once all the checks are completed, you can submit the PR for review:

Once the PR is submitted for review, it will be opened:

At that point, the PR will be marked as needing review:

Eventually it will be triaged, which means that someone will look at it and decide what kind of modification it is, which will affect who will review it:

Finally, it might be accepted, see the actual pull request I made: