Evaluating Software Supply Chain Security Solutions

A software supply chain security solution should produce and operationalize SBOMs to give you instant, accurate answers so you can secure your organization's software estate. We identified eleven key capabilities. Most solutions do some of these well. The gaps are where evaluations go wrong.

What is software supply chain security

Your software supply chain is everything that goes into building and shipping your software: the open source packages and vendor components you depend on, the build systems that assemble them, and the developer environments where the work happens. You control some of that. Most of it belongs to someone else.

Software supply chain security is the practice of knowing what is in your software, where each piece came from, and whether it can be trusted — and being able to demonstrate the same to whoever consumes what you ship. In practice it covers your dependencies down to the deepest transitive level, the known and emerging risks in them, evidence that your artifacts were built the way you expect, and control over what gets in before it arrives.

How to evaluate software supply chain security solutions

Software supply chain security isn’t a nice-to-have. Supply chain attacks have become a preferred method for bad actors. They no longer have to spend weeks or months finding a gap in their target’s defenses, slowly work through systems to avoid detection, and finally get access to the data or resources they’re after. Instead, they compromise the dependency with the weakest security and let the damage propagate. As a bonus for attackers, this scales very well. They don’t just compromise a single target — they compromise dozens or even hundreds of organizations at once.

With the increasing availability of frontier large language models (LLMs), the scale only gets better for attackers. Frontier models like Claude Mythos have now demonstrated this in the open. In August 2026 the UK AI Security Institute published an incident report describing AI agents that, during a cyber evaluation, researched an open source project's maintainers, created fake identities, and opened a malicious pull request without being instructed to.

Companies are not helpless against software supply chain attacks, but the problem is hard. Between 96% and 99% of commercial codebases contain open source components, and roughly 77% of the underlying code in those applications is pre-built open source libraries and frameworks. The mean commercial application carries around 1,180 open source components. Security leaders are at an information disadvantage by default.

Incident response often starts with a painful search to find all vulnerable versions. But if you have the right solution, you can jump straight to solving the problem. Even better: you can be proactive and address risks before they become incidents. With a rapidly evolving threat landscape and regulatory requirements, having the right solution can save countless hours and millions of dollars.

Selection criteria

With the increasing importance of securing your software supply chain from attack, you need to choose the right solution. You can’t decide just based on price or empty promises. The rest of this guide highlights key capabilities you need. Every solution vendor needs to be able to reasonably address the 11 areas highlighted here.

01. Source-built dependency graph

What it is

A dependency graph includes all of your direct and transitive dependencies with explicit relationships.

Why it’s important

A simple list of dependencies doesn’t tell you what really matters: how a dependency ended up in your software. Using a graph model tells you what pulled in a particular dependency, which informs how you address a vulnerability in that dependency. With a graph, additional data relationships — like vulnerabilities and source code repositories — are easily traced through your software estate.

Building the graph from your source code and build systems as software is created, rather than reconstructed from finished binaries afterwards gives you a more complete view. The difference is between recording what actually happened during a build and inferring it later from the output. Source-built evidence captures transitive dependencies, build provenance, and the identity of who produced each artifact, in formats a third party can independently check.

What to look for

Select a solution that:

• Assembles data into a queryable graph
• Builds the graph’s foundation from source code and build processes
• Augments the graph with additional information

02. SBOM generation

What it is

A software bill of materials (SBOM) is a document that lists the components that go into a particular piece of software.

Why it’s important

An accurate SBOM serves as the foundation for documenting and understanding your software supply chain. Without a quality SBOM, you cannot accurately identify vulnerable applications, respond to vulnerabilities and compromises, or identify risky dependencies. In addition, regulations such as Europe’s Cyber Resilience Act (CRA) and section 524B of the U.S. Federal Food, Drug, and Cosmetic (FD&C) Act mandate SBOMs as part of the compliance process. Beyond regulatory requirements, savvy organizations are increasingly demanding SBOMs from their software vendors; your customers may start asking for SBOMs for the software you provide.

What to look for

Various integrated and standalone solutions exist to generate SBOMs at all stages of software development from design to runtime. To produce SBOMs with the necessary quality levels, you’ll want to look for the following features:

• Multiple language support
• Produces SBOMs at all stages of the software development process
• Captures transitive dependencies in addition to direct dependencies
• Correctly identifies dependency relationships
• Includes software license identification
• Produces SBOMs in an open format like CycloneDX or SPDX
• States clearly where its dependency data is complete and where it is not

03. SBOM operationalization

What it is

SBOM operationalization turns a static document into a dynamic set of relationships that answer your practical questions.

Why it’s important

An SBOM that sits in storage may solve your compliance requirements, but it does nothing to help you secure your software. SBOM operationalization is what allows you to quickly answer questions like “where are we vulnerable to the latest Shai Hulud variant?” or “which of our dependencies are unmaintained?”

What to look for

The richer the data — and relationships — the better answers you get to your questions. Select a solution that:

• Builds a relationship graph for all SBOMs collectively, not just individual SBOMs
• Identifies where vulnerable dependencies are and how they arrived in the software
• Clearly identifies the scope of impact (or “blast radius”) for a vulnerability
• Provides context-aware prioritization of vulnerabilities
• Reports important metadata like
  • Software license
  • Support status (i.e. is a software version end-of-life?)
  • Known vulnerabilities, including whether a fix is available
  • How well-maintained a dependency is
• Supports alerts for newly-discovered issues
• Provides support for user-generated queries

Generator quality varies more than most teams expect, particularly on relationship data. That's why we created Waybill. Our annual "Best SBOM tools" review includes a comparison of how dependency relationship coverage differs between SBOM generators across ecosystems.

04. Threat intelligence and data collection

What it is

Ingesting up-to-date information on threat research and threat intelligence.

Why it’s important

The landscape shifts constantly. New vulnerabilities get discovered every day and attackers exploit them immediately. Roughly 80% of exploitation now occurs before or on the day of disclosure. If your security solutioning relies on static snapshots, you’re going to miss critical issues. Threat intelligence augments SBOM operationalization to shorten the time to remediation for vulnerabilities in your software.

What to look for

There’s no single feed that contains all of the possible threat information. You want a solution that supports ingesting:

• Public information on open source vulnerabilities from services like OSV
• Private data feeds for proprietary software or pre-disclosure vulnerabilities

Ingesting external data is not sufficient on its own. The solution you choose should also relate the threat data to the software packages in your dependency graph so that you can act on new threats.

05. Third-Party Software Risk Protection

What it is

Protection against business risk from software not written by your organization.

Why it’s important

Every company relies on software written by someone else, whether it be open source or vendor-supplied software. Third-party software dramatically shortens the software development process, allowing you to deliver value to the business much faster than if you had to write it all yourself.

But this reliance comes with a variety of risks that you must account for. Vulnerabilities can allow attackers to exfiltrate or modify sensitive company and customer data. License violations can open the organization to lawsuits or penalties. Unmaintained software won’t keep up with changes in technology or get fixes for newly-discovered vulnerabilities.

What to look for

Your software supply chain security solution should address all of the third-party risk types. It should:

• Identify the presence and location of existing and new vulnerabilities
• Detect malware and typosquatted package names
• Identify the software licenses used in dependencies
• Detect when a dependency is end-of-life (specific version) or unmaintained (whole project)
• Support policy rules that block the introduction of unacceptably-risky dependencies

06. Reachability analysis

What it is

Determining whether or not a vulnerability can be exploited in a specific instance

Why it’s important

The 2026 Verizon DBIR put vulnerability exploitation ahead of stolen credentials as the top initial access vector for the first time in 19 years, at 31% of breaches. Volume is not the problem. Knowing which of them can actually be reached is.

Vulnerabilities are reported against software versions, but don’t always apply to every possible use of the software. If you don’t use the particular function that contains the vulnerability, your use is not exploitable. Vulnerabilities in a test function may not affect production uses. You may have compensating controls in place that protect you from exploit. Reachability analysis focuses effort on the vulnerabilities that actually matter in your environment. It’s the difference between an emergency response and a backlog item.

What to look for

Your solution should:

• Analyze code to determine if the affected code paths are traveled
• Separate test cases from code that goes into the production application
• Produce evidence for why a vulnerability is not reachable

07. Generation of machine-readable documents (e.g., VEX)

What it is

Vulnerability exploitability exchange (VEX) documents are machine-readable statements of known vulnerabilities and how the software is vulnerable — or not.

Why it’s important

VEX documents provide crucial information from your vendors to you and from you to your customers. Vendor-provided VEX documents tell you which vulnerabilities don’t affect your software and how to mitigate those that do. In addition, VEX documents can be part of fulfilling vulnerability management requirements in regulations like the CRA.

What to look for

Pick a solution that can:

• Ingest VEX documents for your dependencies and incorporate that information into the dependency graph
• Produce VEX documents from reachability analysis that you can provide to your customers

08. Security posture analysis

What it is

Evaluating software development practices and build pipelines for conformance to security standards. Security posture is focused on the methods used to develop software, not the security of the software produced.

Why it’s important

Secure software starts with secure development practices, both for your first-party code repositories and your dependencies. The better a project’s security posture, the more protected it will be from common attack vectors.

What to look for

The ideal solution will:

• Evaluate the security posture of your open source dependencies
• Flag poorly-maintained dependencies
• Detect secrets and credentials that appear in code commits
• Detect and fix misconfigurations in DevOps tools
• Audit your internal repositories against standard practices
• Remediate gaps in your internal security posture

09. Data protection

What it is

Ensuring the integrity and privacy of your internal code and data

Why it’s important

Your company’s software code represents your unique value to customers. Exfiltration or modification of that code represents a real business risk. And a breach of customer data can have significant financial and reputational impacts. Your software supply chain security solution can’t be the cause of your security problems.

What to look for

Select a solution that:

• Keeps human approval in the loop for automated fixes
• Does not retain code used for security analysis
• Has SOC2 or similar certifications

10. Timeline support

What it is

Views of historical data and trends.

Why it’s important

“Live in the present” is good life advice but makes cybersecurity leadership considerably more difficult. Trends are often more informative than point-in-time views of data. This is especially true when you first get serious about securing your software supply chain: many organizations are surprised at the number of vulnerabilities that exist in their software estate.

What to look for

Select a solution that:

• Provides trend graphs
• Makes historical data available — including SBOMs and VEX documents

11. Integrations

What it is

Connections to existing tooling

Why it’s important

Despite the marketing messaging, no one solution can solve all of your problems. You already have systems in place for managing source code, storing software artifacts, work planning, logging, and so on. If your software supply chain security solution can’t present information where your employees are already looking, they’ll miss that information. Good developer ergonomics means software engineers work with your solution instead of routing around it.

What to look for

Select a solution that integrates into your:

• Source code management platform
• Developer IDEs and coding agents
• Artifact management system
• Build and deployment pipelines
• Alerting system
• Ticket tracking system

And, of course, the solution you select should work with the language ecosystems and frameworks that you use. Full coverage isn’t always possible, but you should target 90% coverage or better.

How to use this guide

No solution covers all 11 of these equally well, and priority for your organization may differ from your peers’.

Rank the 11 by what your organization needs. A medical device manufacturer facing FDA 524B has different priorities than a bank preparing for an FFIEC exam, and a weighted scorecard beats a checklist every time.

Then ask for evidence instead of a demo. A vendor showing you their own environment has proved the product works in their environment. Ask each one to run against a repository you own, and to show you what it found, what it missed, and where its data is incomplete. The solutions that can do that comfortably are the ones worth shortlisting.

If you’d like to work through this list against your own estate, the Kusari team is happy to walk through it with you.

Talk to us

Frequently asked questions