Blocking builds on regressions

This feature allows for your team to fail builds on regressions only. A regression is when a new dependency is introduced and the dependency cannot be auto-approved in the catalog. The regression is compared relative to a baseline alignment scan. Anything introduced after that baseline has the capability to fail builds, while everything before that baseline will not cause a build to fail, allowing you to block builds when Newly Introduced violations are added.

Information related to regressions includes:

• The total number of regressions.
• The type of violation (per package) that introduced a regression.
• A severity score (for security vulnerabilities) that allows you to create a script to configure a build stage to fail.

Information is provided in a JSON output format, but a human-readable format is also available.

Use cases and workflows

“PR check” A developer submits a pull request (PR) and they need to check if it has introduced any new standards violations. Rather than check the entire build, they can build against the defined baseline. Checking against the latest build has the potential to hold a developer responsible for sections of code they did not introduce - eg. A new vulnerability has been found in the existing code, but it was not introduced by the pull request.

1. tidelift alignment save (requests all now happen automatically, and will request unrequested packages identified in the SBOM) 
2. Check for violations brought in by the current request

The CI/CD build These are full builds in the default branch of the project. In this flow, the you can compare the total number of violations in the build against a baseline for the project and check that no new violations were introduced. The baseline for this flow is set automatically to be the using the latest build with no regressions.

1. tidelift alignment save (now automatically requests all unrequested packages identified in the SBOM)
2. Compare bill of materials with last passing regressions build (baseline)
3. Do the new dependencies have any violations?
   • If yes, flag this dependency as a violation
   • If no, set as baseline
   • If no new dependencies, set as baseline

The human readable format lists the alignment used as the baseline, the date of that baseline, the number of newly introduced package releases with issues, and a list of the releases and their issues.

To see the regression and alignment results, either add `--wait` or run `tidelift alignment status [alignment number]`.

Running a regression check against a one-time baseline (Optional)

To determine if any regressions have been introduced since a particular alignment, run `tidelift alignment save --baseline-alignment [number]`, where [number] is found on the alignment summary page. This can be used if maintaining two branches in the same project, and the branches use slightly different dependencies. This feature can also be used to determine how many regressions over time have been introduced.

You do not need to specify a baseline alignment number if you want to run against the pre-set baseline (see above section The CI/CD build).