How often we flag a package people install on purpose

Catch rates are easy to advertise and hard to check. Noise is the number that decides whether a scanner stays switched on, and it is the one almost nobody publishes about themselves. These are ours, measured at rules version 2026.09.21c and rebuilt whenever that changes.

Read every number with the sentence next to it.

We chose all of these corpora. None of them is a random sample of the registry and none of them is a measurement of what happens in the field, so none of these numbers is a rate you can carry over to your own dependencies. What they do tell you is how this scanner behaves on lists we have published in full, which you can re-run yourself.

Attack samples we wrote

21 of 22

What that counts: attack samples our rules detected, out of the ones in our corpus. We wrote every sample in it, to match what an attack does rather than to match our rules.

What the corpus is made of, counted from the samples themselves rather than described: technique family we wrote: 17, published incident report: 3, evasion variant we wrote: 2. Only the ones written from a published incident report follow a specific named attack. The rest reconstruct a class of attack, so they are our idea of what that class looks like and not evidence that any particular attack was caught.

And they are not all npm packages. What each sample actually is: npm package: 16, Python source file: 3, CI workflow: 2, agent skill: 1. That matters for reading the number, because a corpus holding CI workflows and an agent skill is not measuring npm packages alone.

This is not field recall. It cannot be. Field recall would need the set of all attacks published in a period, including the ones nobody has found yet, and nobody has that list. Anyone quoting a field recall figure for a package scanner, including us, would be making it up.

The corpus includes cases we expected to fail, on purpose. A harness containing only things you catch tells you nothing.

What we do not catch

Not detected: payload hidden inside a minified bundle technique family we wrotenpm package

Written from: attacker ships the stealer inside a published dist bundle

Minified and bundled files are left out of the source-level rules on purpose. A single dense line of machine-written code produces so many spurious matches on ordinary packages that including it would flag half the registry, and a scanner that flags everything is one people turn off. The trade is real and we take it knowingly: a stealer buried in a published dist bundle can get past this, which is why a report that could not read part of an artifact says it was only partly analyzed instead of returning a confident clear.

Noise on packages people install on purpose

Popular npm packagesrules 2026.09.21cmeasured 2026-09-22

8 of 154

What that counts: packages flagged by our rules out of the whole list, with no indicator feed in play.

The corpus, and who chose it: The list we publish package pages for, chosen by us for search traffic, not sampled from the registry at random. It is weighted to packages a working developer installs on purpose, which is the population a noisy scanner annoys.

Full breakdown: 140 came back clear, 8 came back flagged, and 6 could not be read at all in this run and are counted separately rather than folded into either side.

Package	Verdict	Risk	Rule categories
`figlet`	flagged	high	prompt_injection
`jose`	flagged	high	credential_access
`jsdom`	flagged	high	credential_access, obfuscation
`mongodb`	flagged	high	credential_access
`pm2`	flagged	high	obfuscation, persistence
`ts-node`	flagged	high	prompt_injection
`vite`	flagged	high	credential_access, obfuscation
`xstate`	flagged	high	credential_access

Naming them is not the same as conceding them. We are not claiming here that each of these is wrong, only that our rules flagged it and that you can go and read the evidence yourself. Where we work one out to have been wrong, it is corrected and listed in the public corrections log rather than edited out of this table.

Could not be read in this run:

`autoprefixer`	fetch failed: unreachable (npm: packument-fetch)
`better-sqlite3`	fetch failed: too_large (fetch: content-length 11402131)
`handlebars`	analysis exceeded the time budget
`highlight.js`	fetch failed: internal (unexpected: other side closed)
`next`	fetch failed: too_large (fetch: content-length 41747455)
`postcss`	fetch failed: unreachable (npm: packument-fetch)

Adversarial npm packagesrules 2026.09.21cmeasured 2026-09-22

9 of 53

What that counts: packages whose result disagreed with the label we wrote down before running them.

The corpus, and who chose it: Chosen by us because each one legitimately does something our rules look for: reads cloud credentials, posts to a chat webhook, spawns shells, compiles code at run time, or runs an install step that downloads a binary. Every label was written before the run.

Full breakdown: 46 came back clear, 4 came back flagged, and 3 could not be read at all in this run and are counted separately rather than folded into either side.

The two directions, separately, because collapsing them into one number would hide which way we were wrong: 4 we had labelled clear came back flagged, and 5 we had labelled flagged came back clear. The second group is the more uncomfortable one: those are packages whose install-time behaviour we predicted we would report and did not.

Labelled clear, came back flagged: ethers, node-gyp, pm2, serverless.
Labelled flagged, came back clear: bcrypt, esbuild, puppeteer, sharp, sqlite3.

Package	Verdict	Risk	Rule categories	Our label
`ethers`	flagged	high	exfiltration	we labelled it clear
`node-gyp`	flagged	medium	misdirection, obfuscation	we labelled it clear
`pm2`	flagged	high	obfuscation, persistence	we labelled it clear
`serverless`	flagged	high	credential_access	we labelled it clear

Could not be read in this run:

`googleapis`	fetch failed: too_large (fetch: content-length 17382056)
`handlebars`	analysis exceeded the time budget
`node-pty`	fetch failed: too_large (fetch: content-length 15468777)

Well-known MCP serversrules 2026.09.21cmeasured 2026-09-22

1 of 20

What that counts: server packages flagged by our rules out of the whole list.

The corpus, and who chose it: The MCP servers we publish pages for, chosen by us from the widely used ones.

Full breakdown: 19 came back clear, 1 came back flagged, and 0 could not be read at all in this run and are counted separately rather than folded into either side.

Package	Verdict	Risk	Rule categories
`@upstash/context7-mcp`	flagged	high	prompt_injection

Every sample, and how it went

Sample	What it is	How it was written	Written from	Result	Label when written
postinstall dropper	npm package	published incident report	ua-parser-js / coa / rc (2021): postinstall fetches and runs a remote binary	detected	expected to detect
ssh + cloud credential stealer	npm package	technique family we wrote	typosquat crypto/credential stealers: read key material, POST to hardcoded host	detected	expected to detect
npm token + env harvest	npm package	published incident report	Shai-Hulud (2025): harvest npm/GitHub tokens and env, exfil to attacker endpoint	detected	expected to detect
obfuscated decode-then-eval loader	npm package	published incident report	event-stream / flatmap-stream (2018): decrypt a blob at runtime, then execute it	detected	expected to detect
decoded into a variable, then evaluated	npm package	evasion variant we wrote	the same loader with the decode and the eval split by a variable, the first thing an author tries to dodge a one-expression pattern	detected	expected to detect
crypto wallet stealer	npm package	technique family we wrote	wallet-draining typosquats: read wallet files, exfil	detected	expected to detect
self-updating beacon	npm package	technique family we wrote	persistent loaders: periodically pull and execute remote code	detected	expected to detect
payload hidden inside a minified bundle	npm package	technique family we wrote	attacker ships the stealer inside a published dist bundle	not detected	written as a case we expected to miss
DNS-based exfiltration	npm package	technique family we wrote	covert channel: encode secrets into DNS lookups (no HTTP sink)	detected	written as a case we expected to miss
reverse shell in a postinstall	npm package	technique family we wrote	bash /dev/tcp reverse shell dropped by an install script (common CTF/malware idiom)	detected	expected to detect
Discord-webhook token stealer	npm package	technique family we wrote	many 2023-2025 npm token stealers: read npm/env secrets, POST to a Discord webhook (a "normal" domain)	detected	expected to detect
clipboard crypto clipper	npm package	technique family we wrote	clipper malware: poll the clipboard and swap a copied wallet address for the attacker's	detected	expected to detect
multi-provider env token harvest	npm package	technique family we wrote	Shai-Hulud-style worm / token stealers: collect NPM/GitHub/AWS secrets from env and POST them	detected	expected to detect
remote code loader to a domain	Python source file	technique family we wrote	dropper that fetches a stage-2 payload from a DOMAIN (no hardcoded IP) and execs it	detected	expected to detect
bin shadows a system command	npm package	technique family we wrote	metadata attack: a package whose bin shadows git/npm so a global install hijacks the command	detected	expected to detect
CI workflow secret exfiltration	CI workflow	technique family we wrote	poisoned GitHub Actions workflow that POSTs a build secret to an attacker host	detected	expected to detect
Windows LOLBin dropper via postinstall	npm package	technique family we wrote	Windows-targeted npm malware: postinstall uses certutil to fetch a remote exe	detected	expected to detect
in-memory shellcode injection	Python source file	technique family we wrote	loader that allocates executable memory and runs a raw shellcode buffer	detected	expected to detect
obfuscated dynamic import (anti-analysis)	Python source file	evasion variant we wrote	evasion: assemble the module name from fragments so a static scan never sees "os"	detected	expected to detect
credential exfil to a throwaway sink	npm package	technique family we wrote	reads env/npm secrets and POSTs them to a webhook.site capture URL	detected	expected to detect
CI workflow dropper from a raw IP	CI workflow	technique family we wrote	poisoned GitHub Actions step fetches a payload from a bare IP and pipes it to a shell	detected	expected to detect
agent skill exfiltrates via a rendered image	agent skill	technique family we wrote	prompt-injection in an agent SKILL.md: a markdown image whose URL templates the conversation to an attacker host	detected	expected to detect

How these were measured

The indicator feeds were switched off for the package surveys, so what is measured is our own rules and nothing else. Leaving them on would mix matches against somebody else's data into the count and stop it being a measurement of the part we wrote.
Every package was fetched and scanned as published, at whatever version the registry served on the measurement date shown on each corpus. The corpus records package names, not versions. We did not write down which release each result came from, so a re-run today resolves whatever is current and may be scanning different code than the run behind these counts. Read each number as measured on the named packages on the named date, not as a result you can reproduce version for version.
Each corpus is the list as it stood on its measurement date, and every package that was in it is named in the machine-readable version of this page. We add to these lists over time, so a list can be longer today than it was when it was measured. Where that has happened these counts still describe the packages named here and not the later additions, and the measurement date on each card is how you tell which is which.
Labels for the adversarial corpus were written down before the run, not after. That is why we are able to report disagreements at all, and it is why the number is not zero.
Packages the fetcher could not read are reported as their own count. They are not quietly dropped from the denominator, which would make every other number look better than it is.
The attack-sample run is deterministic and needs no network, so it re-runs on every build. If its result changes, this page changes with it.
The attack samples are ours, and the table above says of each sample whether a published incident report is behind it or we wrote the class of attack it belongs to. Detecting a sample we wrote is weaker evidence than detecting an incident somebody else published, and the corpus is mostly the weaker kind.

What none of this tells you

A clear result means no indicator matched and no rule fired. It is the absence of a match, not a clean bill of health, and we would rather you read it that way.
Nothing is executed. Behaviour that only appears at run time, against a live server, or on one operating system, is outside what reading source code can see.
These numbers describe rules version 2026.09.21c. The build fails rather than publish a measurement recorded under a different rules version, so a stale survey cannot be served. That check reads the version string, which is set by hand, so it catches a version bump and not every possible rule edit.
This page is a list of our own misses, found by going looking. Treat it as a floor rather than a ceiling: it holds what these corpora happened to surface, not everything there is to find. Verdicts somebody disputed and we upheld are listed separately at /corrections, and where the underlying data comes from, with its gaps, is at /sources.

Check it yourself

The machine-readable version of this page, including every package in every corpus and its result, is at /benchmark.json. The rule catalog is at /v1/rules. Any package here can be re-scanned against the live service, bearing in mind that it names the package and not the release, so what you scan may be a later one:

curl -s https://lazaretto.dev/benchmark.json | jq '.surveys[].members[] | select(.verdict=="flagged")'