Tidelift product overview

The Tidelift Subscription provides a proven approach for managing the health and safety of the open source software supply chain and also streamlines the development process by removing obstacles that slow down developers. This article provides a technical explanation of all the functionality included in the Tidelift Subscription as well as a deep dive into how to integrate Tidelift into your organization’s software development lifecycle.

The Tidelift Subscription

Before getting into deeper technical details, we’d like to introduce a few definitions that will make it easy to digest the content that follows. The Tidelift Subscription consists of three components as defined below:

  1. Open source management tools provide visibility into the open source components being used within a subscriber’s applications, including the ability to continuously inventory application components and to create up-to- date software bills of materials (SBOMs) for all applications.

    An SBOM includes insights such as:
    An example of a software bill of materials (SBOM) in the Tidelift user interface (UI)

  2. The Tidelift catalog is a repository of open source components evaluated by Tidelift’s team of security and licensing experts along with our network of partnered maintainers. The Tidelift catalog contains package release recommendations, security vulnerability recommendations, and verified license data for all evaluated packages and gives organizations the information they need to improve their application health.

    The Tidelift catalog page

     
  3. Custom catalogs provide a way for an organization to define and codify its own set of open source standards and then create, track, and manage repositories of pre-approved open source components. Using custom catalogs, organizations can track non-public, internally-developed (inner-source) packages included in applications as well.

    An example of a custom catalog in Tidelift


Out-of-box functionality

Tidelift is able to deliver immediate value and information on open source packages even before your organization begins defining open source usage and management standards or onboarding development teams. Using Tidelift’s package search functionality you can access package metadata for packages already in use or being considered for use. The metadata contains information such as:

  • Number of package contributors
  • Release history
  • License and license format
  • Known security issues and vulnerabilities

This information helps paint a holistic picture about package health, security history and practices, and it is crucial in evaluating whether a package should be approved for use or not.

The view of a package in Tidelift, in this case Pillow (pypi)



Onboarding applications and using the open source management tools

Software bill of materials (SBOMs)

The first step for organizations is to make sure they have a clear understanding of the open source components already in use, and then to make informed decisions on which components to use moving forward. The best approach for gathering the data required for this analysis is through creating an SBOM. Tidelift stores individual application SBOM data through its project functionality, where projects and applications have a one-to-one relationship.

An example of a SBOM in the Tidelift UI

With Tidelift’s CLI, you can create itemized lists for your applications direct and transitive package dependencies which can be exported in .CSV, SPDX, and CycloneDX formats. These exported formats make it easier to integrate into any internal processes that require an SBOM. Once you have created SBOMs for all of your applications, you can import the complete list of packages to your own custom catalog. This gives you a continuously updated view of the open source in use in your applications.

Building a custom catalog from these SBOMs can make onboarding more systematic and targeted across your application development teams. While bringing all your open source into Tidelift at once can sound appealing, this approach is, more often than not, overwhelming. We instead recommend bringing in sets of applications to make the process easier to manage which in turn allows you to become familiar with Tidelift workflows.

An example alignment in Tidelift’s command line interface (CLI)

Organizing applications

Quick wins and early successes are a crucial part of adopting new technologies. Generally, we find that the older an application is the more technical debt it will contain in the form of deprecated packages, security disclosure, and out-of-date packages. Categorizing the applications that you plan to bring into Tidelift and putting them into lifecycle buckets is a useful way to decide which order to onboard applications. New development, ongoing development, legacy development, maintenance, and sunset are examples of buckets you can start assigning to your applications.

New application development (default)

Starting with your newest development projects helps leverage Tidelift recommendations as early as possible and can help you stay current with the appropriate software versions available. Newer development paired with the introduction of DevOps practices results in faster interactions and quicker deployments. At the end of the day, it is easier to keep a newer package up-to-date than it is to bring a legacy application’s dependencies to the most current package versions.

With new applications we expect to see fewer, if any, maintenance standard violations. When reviewing standard violations for a new application, maintenance tasks can alert you to issues related to open source components being pulled into current development. This process helps your teams course correct before the application gets further into its development cycle.

After adding your most current and active development projects to Tidelift, move onto the next bucket, working your way towards the oldest applications. Even if you decide to bring in applications by the development team, use the same process of newest to oldest as you create your Tidelift projects.

Production applications

Not all organizations will have the same amount of new applications to use with Tidelift. The reality is open source packages have been in use for decades now and many of our customer applications are just as old. To address these older production applications, Tidelift can apply the same standard across an organization’s open source centrally— an advantage from a policy management point-of-view. With older applications, the maintenance standards can be used to provide insights into areas of improvement that need to be considered, in addition to the provided security and licensing standards.

Groups

Using Tidelift groups provides a way to categorize projects and users for enhanced sorting, filtering, and reporting purposes. A group can represent:

  • Development teams
  • Application tiers
  • Business units
  • Cloud development regions
  • Or a combination of the categories above

A group can have multiple projects and users assigned to it. Projects and users can also be assigned to multiple groups allowing for flexible group configurations. Groups have no hierarchy or inheritance at this time in Tidelift. It is still recommended to think about what groups need to be created in a logical way that fits your organization. Arbitrarily creating groups can become difficult to administer and maintain over time.

Developers

After an application has been added to a Tidelift project, it is important that developers begin to use the approved packages in a catalog and discontinue using packages that are denied. A catalog administrator will approve or deny packages based on the catalog’s policy which is composed of catalog standards.

The primary way that developers work with Tidelift is through the Tidelift command line interface (CLI). The following commands should be regularly used during a development:

  • tidelift alignment: Get a bill of materials and check if package releases are approved in your organization’s custom catalog
  • tidelift release list: Display releases for one of your catalogs
  • tidelift releases lookup: Display approved releases for a package in your catalog
  • tidelift request: Request a new package be approved and added to your organization’s catalog


Tidelift catalog

There are two key ways for users to interact with the Tidelift catalog:

  • Getting data from the Tidelift catalog
  • Requesting review of new packages

Getting data from the Tidelift catalog

Tidelift catalog releases and data can be browsed directly from tidelift.com/catalogs.

All custom catalogs receive security vulnerability recommendations and license data from the Tidelift catalog. You’ll automatically see these recommendations when resolving security or licensing issues.

Requesting new packages be reviewed and added to the Tidelift catalog

We are constantly growing the list of packages that are reviewed and managed as part of the Tidelift catalog. We aim to proactively add new packages that customers are using to the catalog so we can deliver better recommendations and data to our customers. If you would like to check with us on the progress of adding packages that you use to the catalog, send an email to support@tidelift.com.

Only publicly-available packages can be included in the Tidelift catalog. We have a screening mechanism in place to prevent any organization’s internal packages unintentionally ending up in this public catalog.



Custom catalogs

Custom catalogs give organizations the ability to track only the packages being used by its developers. They include all the packages that are approved-for-use at your organization, improving organizational alignment and developer experience when using open source. Custom catalogs also provide:

  • Version guidance so that you can ensure that only pre-approved packages are being used.
  • Centralized issue resolution workflows to streamline and automate updating the catalog (e.g. when there are new security vulnerabilities, licensing issues, or requests from your team to start using new packages).
  • Standardized open source release management, to reduce the complexity of managing your open source software supply chain.

Organizations also have the added flexibility to create custom catalogs based on an individual development team’s requirements, setting up unique standards and policies applicable to that team and its development initiatives.



Catalog Administration

Diagram of a development lifecycle without and with Tidelift. With Tidelift, development teams have a clear path on how to move forward

The core benefits of using the Tidelift catalog and custom catalogs:

  1. Version guidance so you can ensure only pre-vetted and approved packages are being used in production environments.
  2. Centralized issue resolution workflows to streamline and automate updating the catalog.
  3. Standardized open source release management to reduce the complexity of managing your open source software supply chain.

This helps your organization adopt a proactive approach for managing open source by establishing policies. Both the Tidelift catalog and custom catalogs act as the hub for your open source policy creation and enforcement.

A catalog represents an ideal recommended state of open source package usage amongst the packages that are approved for use within an organization. Catalogs are a top level construct which contain their own set of standards and each catalog sits at the same hierarchy level within Tidelift. Catalogs include information about packages and their versions that are approved for use within an organization. By enabling or disabling catalog standards, you can establish your policy for that catalog. The decision to use one catalog or multiple catalogs depends on the needs of your organization.

Single catalog strategy

Using a single catalog is the default recommendation for most organizations and comes with the advantage of having all applications compared to a central set of policies. The administrative overhead is lower in a single catalog, with all projects, reports, and task outputs reflecting decisions made on a singular policy. The tradeoff of using this strategy is that in some cases, an organization may require a different set of policies for their applications and a single catalog may not fulfill every application’s specific needs.

Multiple catalog strategy

An advanced catalog administrator can get creative with multiple catalogs. A catalog can slice an organization in different ways since it is a flat construct. Individual catalogs can represent:

  • Application tiers
  • Business units
  • Risk profiles
  • License distribution needs
  • Or a combination of the categories above

Typically a multiple catalog strategy is implemented when different applications within an organization require diverging policies. However, an individual development team’s needs may not justify the increase in administrative overhead that is incurred when using multiple catalogs. Each catalog will require its standards to be maintained independently of other catalogs, but the tradeoff of this increased overhead is more flexibility.

Catalog administrators and catalog users

Catalog roles can be generally viewed as either a catalog administrator or a catalog user. A catalog administrator creates policies and makes decisions based on those policies. A catalog user, here meaning a member (a user that benefits from the output of policy decisions) or a developer, takes the outputs of a catalog administrator’s decisions and applies them to their applications. At this stage, it is important to consider who plays the role of a catalog administrator and a catalog user. A catalog administrator could be one individual or a team based on the size of the organization. It is important that catalog administrators are armed with a set of standards and practices that help them decide which open source components to approve and which to reject. Furthermore, it is critical that these standards and policies are defined in a centralized manner, taking into account the needs of all the stakeholders who are impacted in the case of a vulnerability. Defining catalog standards and correctly assigning the decision maker(s) is an important early step in adopting the Tidelift Subscription.

Catalog administrators are also responsible for managing which standards are enabled and ensuring remediation tasks are addressed by appropriate individuals.



Tasks

At a high level, the primary goal of catalog administration is to answer the question, Do I want my developers to be using this package release?. To achieve this, a catalog requires curation by a catalog administrator in your organization. The primary method in which a catalog administrator curates one or more catalogs is through processing catalog tasks.

When an open source package violates a catalog standard, a resulting task is created to prompt for a decision to be made on that package’s usage. We notify you and create tasks when there are standards violations, taking the guesswork (and legwork) out of catalog management. In its ideal state, the catalog should become the curated source of truth for whether or not a package release should continue to be used or start being used by developers in your organization.

An example of tasks in a catalog in the Tidelift UI

Creating exceptions

There will be instances where it is allowable to use a package release that is not in line with your organization’s standards. By creating an exception, you are indicating that the standard violation in question does not apply to a particular catalog or applications. As such, exceptions are granted at the catalog level and apply to a specific package version and name.

Standards administration

Catalog standards view in the Tidelift UI

 

Standards are how a catalog’s policies are created and enforced. A catalog represents open source packages and package versions approved or denied for use in your organization’s applications. The Tidelift standards page is designed to be easy to understand and configure. Standards are simply enabled or disabled by catalog administrators. This binary mechanism makes setting up a catalog’s policy approachable from the start. For example, if you want to be alerted to security vulnerabilities, enable the ‘Releases have no vulnerabilities’ security standard. If you are not interested in being alerted about security issues in a catalog, you can easily disable the standard.



Enabling standards

There are a variety of security, maintenance, and licensing-related standards that are available out-of-the-box with the Tidelift Subscription. Additionally, organizations can also build their own standards as needed. When a standard is enabled on a catalog, any OSS package that violates that standard will result in the creation of a task.

An example of active catalog standards in the Tidelift UI

Licensing standard

The licensing standard provides more configuration options for setting up the licensing policies for a catalog. Tidelift has worked with its in-house licensing experts to create licensing templates to help users get started with a license policy ranging from least to most restrictive.

The default policy template applied is the ‘Mobile app/Hardware’ template, the most restrictive licensing policy. In the licensing configuration menu, a less-restrictive policy can be selected, or no template can be applied to create a licensing policy from scratch. Open source licensing is a specialized discipline in which Tidelift has significant knowledge and experience. It is recommended to start with a policy template and adjust as needed.

Licensing template selection in the Tidelift UI

If your organization already has an open source software license policy in place, that policy can be mirrored in Tidelift by setting licenses to be approved or denied for use based on your organization’s policy.

Enforcing license compliance

By choosing to use the license compliance standard, we ensure that all package releases in your catalog only use a license from your approved list of licenses. When enabling the standard, you can choose from a list of templates that fit your deployment scenario. You can also proceed without a template and begin classifying the licenses into the following three categories:

  • Approved: Licenses that are always approved for use in the catalog
  • Uncategorized: Licenses that will need additional review in the future
  • Denied: Licenses that are never approved for use in the catalog

The configuration page for the license compliance standard in the Tidelift UI


Security vulnerability standard

By enabling the ‘Releases have no vulnerabilities’ standard, you can ensure that all requested and approved packages are reviewed for any known vulnerabilities.

Tidelift maintains a security database of vulnerabilities and proactively reviews these vulnerabilities. For each vulnerability, we share the upgrade path recommended by Tidelift or our partnered maintainers. This recommended path lets your designated catalog administrator make an informed decision about whether to remove the vulnerable release, to create an exception, or to remediate the vulnerability.

Maintenance standard

There are two ways to enforce maintenance compliance within your catalog:

  • Releases are actively maintained
  • Releases are up-to-date

Releases are actively maintained

The configuration page for the releases are actively maintained standard exceptions in the Tidelift UI

‘Releases are actively maintained’ is a standard that will check to see if a particular package has been marked as deprecated. Packages that are explicitly marked as deprecated are less likely to receive fixes if a security vulnerability is discovered. Stale, deprecated packages can become a nuisance for upgrading an application as they may rely on older versions of other packages.

Tidelift is regularly monitoring for package deprecation from the following sources:

  • From the package manager, when a maintainer indicates that a package has been deprecated
  • Directly from our partnered maintainers for instances when deprecation information has not been shared publicly

Enabling this standard will notify you when your team is using or wants to use deprecated packages and help you uphold this standard. We will also display any additional information that a maintainer has provided about the deprecation, which may include recommendations for alternate packages.

Releases are up-to-date

The configuration page for the releases are up to date maintenance standard

The ‘Releases are up-to-date’ standard measures the risk of using outdated packages within your organization. Similar to deprecated packages, older releases are less likely to be patched. While the package itself may still be actively maintained, these older releases usually have security vulnerabilities and issues that are addressed in newer releases.

The longer your organization waits to update to a newer release, the harder it may be to use as more changes are made to the package and underlying APIs. With this standard, you can keep outdated package releases out of your catalog by enforcing a number of years that releases should be no older than.

Let’s use an example. Suppose you set a default that all releases should be no more than 1 year older than the latest release. 2.0.0 is the latest release, but your projects are still using releases 1.5.0 and 1.0.0. In the example, Tidelift will alert you to update where you’re using version 1.0.0, but not 1.5.0.

A table of examples for when a release would be allowed or denied based on release date



CI Flow

Starting your journey

The first step to effectively manage your organization’s open source software supply chain is to clearly understand what is in use today. Integrating Tidelift into your continuous integration (CI) workflow is a great way to start getting value from your subscription from day one.

Once integrated, Tidelift helps prevent developers from pulling denied packages into their builds. Having the most up-to-date SBOMs sent to Tidelift with each build gives you the visibility into all the open source that your organization is using at any given moment.

Tidelift works with most CI tools including: Jenkins, CircleCI, Github Actions, Teamcity, Gitlab Pipelines, Azure Devops Pipelines, and Atlassian Bitbucket.

Implementing the CLI

Customers can keep their SBOMs current and up-to-date by integrating the Tidelift CLI into their CI pipeline. Most commonly, customers include our CLI as part of their build process for their application. This ensures that Tidelift can generate an accurate SBOM of the actual artifact that is going to be deployed to the development and production environments.

The first step to using the Tidelift CLI is to install it into the CI pipeline where your application is being built. The exact steps vary depending on the operating system running the build server, you can find the additional instructions in our Installing the Tidelift CLI document. Once the permissions have been properly configured for the Tidelift CLI, you can execute tidelift alignment save --wait to upload the SBOM to the Tidelift servers.

Enforcing your standards

Once an organization becomes mature enough in their open source management practices, they can start using the Tidelift CLI to gatekeep their CI pipelines to enforce their standards. This allows an organization to move towards being proactive about preventing problematic open source from entering their production environment, rather than fixing these issues after the fact.

Using the CLI to enforce standards is considered an advanced action taken after standards selection, team onboarding, and catalog administration are fully adopted and the downstream effects of failing a build are fully understood. SBOMs will be submitted at build time and the data can be used to populate a catalog as well as to continuously compare packages to an approved catalog.

Understanding when a build will fail and under what circumstances based on catalog curation is an important step to adopting Tidelift into your software development lifecycle. Failing builds should be considered only after your organization becomes comfortable with Tidelift catalog administration, package approvals, package denials, and establishing your organization’s standards.


 

Blocking builds on regressions

Blocking builds on regressions fails a build if new open source issues are introduced into an SBOM. A regression is when a new dependency is introduced and the dependency cannot be auto-approved in your catalog. The regression is compared relative to a baseline alignment scan. Anything introduced after that baseline has the capability to fail scans, while everything before that baseline will not cause a build to fail, allowing for your team to block builds when new violations are added.

Information related to regressions includes:

  • The total number of regressions
  • The type of violation (per package) that introduced a regression
  • A severity score (for security vulnerabilities) that will allow your team to create a script to configure a build stage to fail

A flowchart of the software development lifecycle with Tidelift starting from developer to catalog maintenance



Integrations

Artifactory integration

Several organizations often use artifact managers to manage their use of open source. Tidelift provides integrations with JFrog Artifactory with the ability to import Artifactory repository metadata directly into a catalog, helping inventory the packages available to your developers. While Artifactory does a good job of providing an inventory of packages and package versions, Tidelift provides additional capabilities such as supplying package metadata as well as giving customers the ability to apply security, license, and maintenance standards. Additionally, Tidelift enables you to aggregate all or multiple Artifactory repository data into a single Tidelift catalog for a centralized view. Applying Tidelift standards in this workflow allows you to use Tidelift as a barometer to identify and prioritize the areas of improvement you need to plan as you start onboarding development teams.

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Article is closed for comments.

Articles in this section