Feature Request: Wait until CI workflow has finished to post status comments to GitHub #905
Description
What product do you want to improve?
Uploader
Is your feature request related to a problem? Please describe.
My team has a monorepo that runs many different jobs across many different sub-modules during CI based on which files were modified in the feature branch. We use Flags to differentiate between test types (unit, contract, etc.) and Components to differentiate between sub-modules. One sub-module in particular has a contract test suite that can run in under a minute, but a unit test suite that takes over 20 minutes to run. This discrepancy is longer than the delay Codecov automatically waits before adding a status comment to pull requests. As such, engineers that work on that sub-module report that they'll initially get a comment saying coverage is very low due to how small their contract test suite is, but it is then updated later to be much higher when the unit test suite finishes running. Since each sub-module has one or more jobs that can upload coverage reports to Codecov, and since we only run each job when there was a change to that part of the code, it's impossible to know ahead of time how many uploads will happen for any given CI run. That means the after_n_builds configuration does not work for us.
Describe the solution you'd like
- An API endpoint I can hit at the end of my CI pipeline that tells Codecov it has received all reports and can now update the status comment.
- (Optionally) An initial status comment on the PR that says code coverage is still being generated for the current commit so we know if the status we're seeing is the latest information.
Describe alternatives you've considered
We use GitHub Actions for this repo, so if there's a Webhook you could tie into to automatically track run state and wait until it finishes that would be ideal. However, if that's not available, I would be fine just configuring a final job that hits this proposed API endpoint.
Additional context
N/A