Big vulnerabilities in upstream dependencies can linger in deployed software long past the point when a patch is available. Maven estimates that 35% of Log4J downloads continue to pull the version with the world-famous vulnerability.

What's the cause? Why aren't developers applying patches?

National Ecosystems

If you follow the 35% link above you'll see that countries have characteristic exposure profiles. Taiwan is far and away the worst, and China next.

Taiwan and China share a language but not a government. Maybe the problem is security resources that fit common practices. Is Mandarin well supported by Dependabot and similar tooling? Are technology news sources for developers not covering Log4J? Is there cultural skepticism? Are different development platforms (e.g. Gitea instead of GitHub) popular there, and is there a difference in security resources?

The market share of unpatched L4J in a given country is not the same as the market share at a global scale. Taiwan is tiny - even 80% unpatched downloads would have less impact on the global numbers than 20% of a huge country like China.

The follow-up work here is a country-specific study of China and Taiwan. What's holding back patches may be obvious to developers in these places.

Maturity Levels

When a codebase is mature, there is more resistance to change.

From Beyond Metadata: Code-Centric and Usage-Based Analysis of Known Vulnerabilities in Open-Source Software:

In the early phases of development, updating a library to a more recent release is relatively unproblematic, because the necessary adaptations in the application code can be performed as part of the normal development activities. On the other hand, as soon as a project gets closer to the date of release to customers, and during the entire operational lifetime, all updates need to be carefully pondered, because they can impact the release schedule, require additional effort, cause system downtime, or introduce new defects.

How can patches specifically target mature codebases?

Mature software will have older code and will tend to use older library versions. The biggest issue is simply providing non-breaking patches for older library versions. The older a library the less its developers want to work on it, and the greater the chance that an upgrade will only be available with a major version upgrade.

What can the security community do? Encourage library developers to support old versions. Discourage breaking changes of any kind. Encourage application developers to give preference to libraries with a record of support for older versions.

Trustworthiness

There may exist a patch but it may not be well vetted. Every upgrade is a chance for something to go wrong. There may be new bugs and vulnerabilities.

Ways to ameliorate the problem:

• Encourage and help with automated tests
• Have a trusted third party certify updates
• Discourage library providers who lack the resources to make trustworthy upgrades

Lack of Auto-Upgrade

The reach of automated vulnerability scanning and patching is probably still pretty low, at a guess.

Vendored (copied and pasted) code is hard to scan or upgrade. Not all languages have high-quality scanning and upgrading. The CI/CD infrastructure for automated scanning and patching is relatively new. Package repositories like Maven lack facilities to force upgrades.

Follow-ups:

1. Study improvements to vendored code detection and upgrade
2. Identify needless gaps in tooling. For example, improve Dependabot availability in Gitea.
3. Collaborate with package repositories on forced upgrades (or discouragement of known-bad versions)

Vulnerability Fatigue

Developers may be skeptical of vulnerability reports. There is a never-ending stream of announcements, but the daily impact is low.

Vulnerabilities may be in libraries that aren't used in production. Vulnerable open source dependencies: counting those that matter found that "about 20% of the dependencies affected by a known vulnerability are not deployed, and therefore, they do not represent a danger to the analyzed library because they cannot be exploited in practice." Identifying whether a dependency is one of these can take a considerable amount of work.

To ameliorate this problem, security tooling can improve detection of which risks are not a factor in production.

Another source of fatigue is inflation. Developers get cynical about new vulnerability reports when they have ignored old ones without suffering harm.

To ameliorate this problem, there could be checks and limits on new reports. A report and patch should be accompanied by a score from the Common Vulnerability Scoring System Version 3.1 Calculator.