[HOLD] Link baseline builds when creating AdHoc builds on a branch #79786
Description
HOLD for #79781
Background
It's an increasingly common practice to request Applause QA on AdHoc builds of E/App branches. This can increase confidence in large PRs before merge and reduce the number of deploy blockers we encounter on staging.
When Applause performs regression tests on AdHoc branches, if they encounter issues, they first check to see if the same bug is reproducible on staging before reporting it. They do this because when we request QA on AdHoc branches, we typically only want issues reported to us that are introduced by the changes on the branch. What can happen is:
- A bug exists on main, but isn't deployed to staging yet
- We create an AdHoc build of our branch, created off an up-to-date main
- We request QA of our branch
- The bug is reported, even though it exists on main and was not introduced in our branch.
Problem
When bugs exist on main, if you request AdHoc regression testing of a branch created off main, those bugs will be reported as having been introduced by your PR, which prevents us from efficiently identifying problems introduced in a branch.
Solution:
- Whenever code is merged into main, kick off an AdHoc build (this will happen in [Tracking] Use Adhoc Build Snapshots to Identify Deploy Blockers Faster #79781)
- Whenever such a build finishes, update the E/App README to include a link to the latest preview build of the main branch. (this will also happen in [Tracking] Use Adhoc Build Snapshots to Identify Deploy Blockers Faster #79781)
- (new) When kicking off AdHoc builds on PR branches, include not only the links to the AdHoc build on that branch, but also the AdHoc build from the commit where that branch forked from main. We'll call this the "baseline build"
- (new) When Applause finds a bug during AdHoc QA, they can check it against the baseline rather than staging. Therefore, we'll be confident that any issues they found were introduced on the branch, rather than being pre-existing/unrelated issues.