Closure — evidence census

The census

Bounded by a method, not by one paper's bibliography.

A per-paper roster moves when you change the seed or the data provider — ours reordered when the forward-edge source changed. A census is bounded by camera-trap animal density and abundance estimation, so it holds still, and corrections accumulate into it instead of evaporating on the next run.

1,003

works in the census

912

found by definition query

added by citation closure — foundations no query names

version-controlled definition queries

The boundary is reviewable. The queries that define this census live in the repository, so a change of scope is a diff someone can argue with — not a silent drift in what "complete" meant.

The denominator

Every number below is a share of something stated.

33,545 candidates screened → 368 in the roster → 172 estimate animal density

Scope. English-language, DOI-bearing works reachable through Crossref reference lists and Semantic Scholar forward citations. Non-DOI and unindexed literature is out of scope and is NOT counted in the denominator.

Evidence grid

What has been done, by method and by contribution type.

A · is a genuine gap — no work of that kind was found inside the denominator. The reliability column is not decoration: it is the measured precision of the classifier for that row, so you know which counts to trust before you use them.

Method	method	validation	application	review	total	automated	reliability
REM	6	15	14	1	36	1	P 100% · R 32%
not-stated	9	4	13	4	32	0	P 17% · R 82%
CT-DS	12	6	9	2	29	5	P 100% · R 56%
SCR	3	4	7	1	15	0	unmeasured
index-RAI	4	6	3	2	15	0	P 100% · R 33%
N-mixture	7	4	2	·	13	0	P 100% · R 25%
REST	3	3	4	1	11	0	P 100% · R 40%
TTE	2	6	3	·	11	0	P 100% · R 10%
spatial-count	5	4	1	·	10	0	unmeasured

3 empty cells 9 method families 4 replication candidates

The not-stated row is a feature. Those works estimate density but never name the model, even in their abstract. An earlier version of this vocabulary forced them into whichever class shared a word with the title, which is how its largest row reached 31% precision. Declaring them is the honest alternative.

Precision before recall. Every estimator class scores 100% precision at the title tier — a named method is a correct one. Recall is low there by design, because the classifier abstains instead of guessing, and the abstract tier recovers what it left.

Reliability

The classifier was measured before its output was published.

Classification runs in two tiers. Titles first, then works the title cannot decide are escalated to their abstract. To measure the title tier, two independent labellers were given strictly more information — title plus abstract — and ground truth is where those two agree. Asking a second model whether the first was right would have been the same correlated-error trap that already produced one false result here.

The category surgery worked. An earlier vocabulary put identification status ("unmarked") on the estimator axis, producing a sink class that absorbed anything with the word in its title — 31% precision on the largest row in the table. Splitting the axes and permitting the classifier to answer not-stated moved inter-labeller agreement from κ 0.73 to κ 0.95, and every estimator class to 100% precision: when a method is named here, it is right.

What the percentages below do and do not cover. They measure the title tier only. Its recall is deliberately low because it abstains rather than guesses; the abstract tier then resolved 84 of the 101 works it could not decide. The two-tier pipeline as a whole has not been measured end to end — that is the next evaluation, not a claim being made here.

0.95

Cohen's κ between independent labellers — is the task well defined?

60%

title-tier accuracy — abstentions count as misses

129

works with solid ground truth

134

abstracts read — none stored, none served

κ = 0.95 is moderate, not strong. Two labellers holding the abstract disagreed on 4% of works. That is a finding about the categories, not the classifier: some method families overlap in practice and need tightening before their counts can be sharpened.

Rights

Every document, and every source we read them from.

398

documents classified

34%

servable — open licence or public domain

104

publisher TDM terms declared

policies to read to clear them all

explicit_commercial 134silent 110tdm_declared 104explicit_noncommercial 50

Our own supply chain

A rights ledger that exempts its own sources is decoration. Each data dependency carries a verdict, and the build refuses to publish an artifact derived from a non-commercial source.

crossref CC0 / PDopenalex CC0 / PDs2 restrictedunpaywall CC0 / PDanthropic-claude CC0 / PD

Reading is not serving. A bibliographic reference is a fact and may always be named; reproducing expression from a restricted source may not. Abstracts were read to build the reliability measurement above and are stored nowhere.

What this does not claim

The limits are the product too.

The ordering carries no demonstrated signal. Citation frequency scored r = +0.10 against expert relevance. Semantic reranking scored p = 0.66 against real author citation decisions — no effect — and gained on pre-2022 papers while losing on recent ones, the signature of pretraining recall rather than judgement. Rows are grouped and measured, never ranked.

What is validated. Across 10 seed papers and 140 held-out real citations, 79% of works the authors actually cited were recoverable in the pool. Same-field and off-field controls scored 0.00 with three independent expert graders.

Classification is from titles. Method family and contribution type are measured above. Sample sizes, effective sampled area and validation targets require full text and are not asserted anywhere on this page.