Understanding package quality checks

At Tidelift, we're partnering directly with maintainers (and paying them!) to improve the quality of the open source software packages your team relies on.

For each package there's a quality checks page where you and your team can see how a package performs against Tidelift’s quality checks for security, development practices, and long-term outlook.

Quality-checks.png
Security checks

No known vulnerabilities on latest release
This check looks for any security vulnerabilities on the latest release of the package. This indicates that the maintainers of this package are resolving vulnerabilities as they are identified.

When the latest release is free from security vulnerabilities, this is an indicator that the package maintainers are working to resolve vulnerabilities that arise. This also means that a release is available to upgrade to that is free of vulnerabilities.

Discoverable security policy
This check looks for a publicly available security policy for the package. A security policy should define the process for how maintainers will handle security issues without exposing said issues publicly before a fix is available.

A security policy means that a process is in place to address and fix security issues as they are discovered.

2FA enabled at source repository
This check indicates that Tidelift has first-party attestation that two-factor authentication practices are being used for source repository access.

Multi-factor authentication provides extra protection from malicious code being added to a package.

2FA enabled for package manager
This check indicates that Tidelift has first-party attestation that two-factor authentication practices are being used for the package manager release process.

Multi-factor authentication provides extra protection from a malicious release being published for a package.

Release managers are reviewed
This check indicates that Tidelift has first-party confirmation that the allowed release managers for a package have been reviewed and verified. This helps ensure that only authorized users can release new versions of a package.

Reviewing the list of users who are allowed to create releases for a package ensures that releases are coming from a trusted user.

Security vulnerabilities have recommendations

This check confirms that recommendations have been provided for all security vulnerabilities on the package. These recommendations provide clear and actionable mitigation steps.

One of the most valuable things to know about a vulnerability is what to do to get rid of it. These recommendations provide a clear path to remove a given vulnerability.

Package cryptographically signs releases

This check looks for a cryptographic signature on package releases. Signing releases helps ensure that the package is coming from a trusted source and is non-malicious.

Package uses fuzzing tools

This check indicates if fuzzing tools can be automatically detected as part of the package's test process. There are limitations to this check as there are many ways to implement fuzzing that may not be automatically detectable.

Fuzzing is a method of testing that feeds random data into inputs. This is one way to automate edge testing and check for bugs and vulnerabilities before releasing a new version.

No executable artifacts in source repository

This check looks for any executable (binary) artifacts inside the package source repository.

When these executables are not reviewed, it increases the potential for malicious code to be introduced.

Releases are discoverable upstream

This check ensures there are releases available upstream for a given package.

When releases can be downloaded from a public package manager, the risk of getting a release from a malicious source is lowered.

Development practices checks

Package has a defined machine readable license

This check indicates that the package has a machine readable license.

Having a license that is easily detectable in the repository makes the criteria for use clear and indicates that the maintainers behind a package are looking to make usage easier.

Package has a clean release available
This check indicates that there is at least one release of the package with no known vulnerabilities, and that the latest compatible version of that release's dependencies are maintained and also have no known vulnerabilities.

A maintainer managing their dependencies and using packages that are free of vulnerabilities increases the likelihood that transitive issues will be taken care of for this package.

No known issues in dependencies for latest release
This check indicates that the dependencies of the latest release are maintained and have no known vulnerabilities.

A maintainer managing their dependencies and using packages that are free of vulnerabilities increases the likelihood that transitive issues will be taken care of for this package.

Package uses a code review process

This check looks for indicators that new code has been reviewed on recent pull requests in the package repository. These may include a user who committed the change who is different from the user that merged the change or other commonly used review tools.

Having multiple people look at code changes reduces the risk of bugs and malicious code being introduced to a package. This leads to a more stable and secure package overall.

Long-term outlook checks

Package is not deprecated
This check indicates if the package has been marked as deprecated. Deprecated packages will not receive updates if a vulnerability or other issue is identified.

Deprecated packages are unlikely to receive updates if a vulnerability or other issue is identified.

Package appears maintained
This check indicates if the package appears maintained based on pull request rates, issue close rates, the lifting status of the package, and Tidelift’s research. When a package appears maintained the likelihood that future vulnerabilities will be addressed increases.

A package with activity (responsiveness to PRs and issues) is more likely to have someone available when a vulnerability arises or when dependency management is required.

Package has a stable release greater than two years old
This check indicates that the package has a stable release that's more than two years old.

A package with an older stable release is more likely to be stable and have continued support than a brand new package that was just released.

Responsive to security issues

This check indicates if there is a maintainer for this package who has indicated that they will provide security fixes on the latest release stream.

When a maintainer says they will be available to support when a security issue arises, there is greater assurance that a fix will be made available.

Check statuses

Each Check shows a status of passed, not passed, no assertion, or pending. When a check has the status of no assertion, Tidelift doesn’t have the data necessary to pass or not pass the check. A pending status means that assessment is still ongoing.

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Article is closed for comments.

Articles in this section