How Tidelift evaluates packages

Tidelift has built a unique model of partnering with open source maintainers and paying them for the critical work they do in keeping their packages maintained inline with industry-leading secure software development practices. Together with open source maintainers, we’ve developed an informed and clear idea about what makes a package reliable and secure enough to be used by our customers.

When evaluating packages, we suggest that you use the Tidelift recommendation. The Tidelift recommendation is a holistic evaluation of the package, and whether it is developed and maintained in a way that would make it a good fit for use in an application.

The recommendation for the package is one of:

  • Not recommended: This package has flaws that make it risky to use in your environment
  • Caution advised: While the package may not have severe flaws, it either has current issues, or does not follow a number of secure development practices
  • Neutral: The package does not have obvious flaws, but also does not have additional criteria (funding, backing) that would make it a recommended long term bet
  • Recommended: The package is funded and adheres to secure development and enterprise standards, and is a good bet to build your software on top of.

If you want to make further determinations, you can look at raw data that Tidelift provides for the package.

What makes a package not recommended?

A package is not recommended if it falls into any of these categories:

It was removed from the upstream package manager

A package that was removed from the upstream package manager was removed for a reason. At best, the maintainer decided it wasn’t worth having it published, either due to bugs, or not wanting to maintain it. At worst, it was actively hostile and/or malware, and was pulled by the package manager itself. Packages that have been removed from the package manager should not be used.

Tidelift checks to ensure that any package that has previously existed upstream remains available. 

It is deprecated

A package that has been deprecated has been flagged by its maintainer as approaching its end of life. While the maintainer may still choose to make updates, they may also choose not to do so. Deprecated packages should not be brought into new applications, and work should be done to replace them in applications that might be currently using them.

Tidelift tracks packages that have been marked as deprecated by their maintainers directly in their ecosystem, and researches to find packages that have been marked deprecated by other means (such as notes in the documentation.) This data is also surfaced in the “Package is not deprecated” quality check.

It is unmaintained

Unmaintained packages will not be fixed for new bugs, features, or security vulnerabilities. Unmaintained packages are a latent risk within your organization. They should not be brought into new development, and work should be done to replace them in applications that might be currently using them.

Tidelift tracks the maintenance activity of packages, and when it reaches a risky point, does an evaluation of the package to see if it is still maintained.

It does not publish a security policy

A security policy is a documented procedure that a software project uses when handling security issues. It involves having a designated point of contact and a process for handling issues. Having a security policy demonstrates a willingness to fix issues as they arise.

Tidelift tracks whether a package has a defined security policy.

It is not responsive to security issues

The response to security issues, and providing fixes in a timely manner, is critical when building on top o f piece of software.

Tidelift tracks whether a project makes fixed releases available.

What makes a package marked as caution advised?

A package is marked as "caution advised" if it falls into any of these categories:

It does not use two-factor authentication when committing new code, or publishing new releases

Two-factor authentication is a key security practice that ensures that the person who is committing new code or publishing new releases is the correct, authenticated person. You've probably set up two-factor authentication for your email or your bank, as an example. It is important enough that some package managers have begun requiring it for everyone who publishes new software releases.

Tidelift tracks where maintainers have asserted that they are using two-factor authentication, or where they are required to by the systems they use.

It has executable artifacts in the source repository

A repository of source code should contain only source code. Having precompiled code in the source repository makes it easier to hide malicious code if it were to happen.

Tidelift tracks where binary artifacts have been detected in the source code repositories used by packages.

It has not reviewed who has access to push new releases

A key part of security is reviewing who has what permissions to do what, and keeping the attack surface small. Each new person with the ability to publish a release is an expansion of the attack surface for the package.

Tidelift tracks where the maintainer has validated that only the proper set of people have access.

It has at least one active vulnerability on its latest release

Having a vulnerability with no available fix is a problem - you're left without anything you can do in your environment.

Tidelift tracks whether vulnerabilities have a fixed release available.

It will bring in an issue due to its dependencies

When you add a software project to your environment, you're not just bringing in that piece of software; you're bringing in everything it depends on, and trusting the practices of the project's maintainers in managing those dependencies.

Tidelift tracks not just whether an individual package has issues, but also whether it will transitively bring an issue into your environment.

What makes a package recommended?

A package is recommended if its maintainers has made commitments to uphold secure development and maintenance standards, and has financial backing to do so.

Maintainers do this by partnering with Tidelift.

Was this article helpful?
0 out of 0 found this helpful



Article is closed for comments.

Articles in this section