How Lineage works
Written for someone deciding whether to rely on this. It sets out what a map claims, how each claim is produced, which parts are machine judgment rather than fact, and what the method does not cover. If you only read one section, read what you can and cannot rely on.
What a map is
A map answers one question: how did this term get here, and which papers moved it?It is not a literature search and not a ranking of important papers. It is an account of a term’s development, assembled from what the papers themselves say the term means.
Three things are on it. Papers, drawn as dots, positioned by year and sized by how much later work built on them for this term. Threads, the parallel lines of development the idea split into. And links, typed relationships between two specific papers, each carrying a sentence explaining what the later paper did to the earlier one’s idea.
How a map is built
Choose the papers. Starting from your term and its field, a list of candidate papers (the corpus) comes from OpenAlex, scoped to the discipline you chose. Candidates are ranked by how close each paper is to the term in meaning, and a fixed number of the closest are kept. A cut-off score is deliberately not used, because it deletes foundational papers: “Attention Is All You Need” scores only 0.567 for closeness to the word “attention”, and a cut-off of 0.62 once removed it from the map entirely. The list is then widened by one step, to the papers those papers cite and the papers that cite them. That step is deliberately not filtered for closeness to the term, because ancestors do not use their descendants’ vocabulary. Bahdanau’s 2014 paper, the origin of neural attention, never says “attention”; it says “align”. It can only enter through citations. At this stage only titles, authors, dates and citations are used. No paper is downloaded and nothing is paid for until the list of papers is settled.
Fetch and extract full text. Papers are fetched from arXiv, LaTeX source preferred over PDF because LaTeX extracts cleanly and PDFs do not. Extraction is checked against the paper’s claimed title, because metadata sources sometimes attach the wrong identifier to a work: two well-known models in our own corpus carried titles belonging to different papers. If the text we get back does not match the title we expected closely enough, the paper is dropped rather than quietly filed under the wrong name.
Read every paper for its own definition. A language model reads each paper’s full text and answers one question: does this paper give the term a meaning of its own? If yes it must return the defining passage verbatim, plus the section it came from and what the paper claims it contributed. Papers that merely use the term are recorded as such and excluded from the map. A negative answer is a finding, not a failure, and is cached like any other.
Find the threads. The corpus is read as a whole to identify the distinct lines of development the term followed, grouped by what the papers say rather than by who cites whom. Two papers can define a term identically without ever citing each other, and a citation can cross between threads. Grouping by who cites whom would draw a picture of the citation network, not a picture of the idea.
Type the links between papers. Where one corpus paper cites another, the sentence containing that citation is retrieved from Semantic Scholar and a model classifies the relationship into one of six types, with a confidence score and a one-sentence justification. Direction is chronological: the cited paper is the ancestor, the citing paper is the descendant, and the type describes what the descendant did. Links that point backwards in time are discarded as data errors.
Verify every quote. Each quote is string-matched against our own extraction of that paper’s text. Found: marked ✓ verified. Not found: marked ○ inferred and shown as such. A quote is never upgraded, never quietly rewritten to fit, and never hidden because it failed.
How papers are ranked
Two independent measures decide what appears on a map and how it is drawn. Neither is displayed as a number, because a spurious precision would be worse than none.
Uptake sets a paper’s size: how much later work in the corpus built on it for this term. Not raw citation count, which measures fame and puts famous-but-incidental papers at the centre of every map. Each citation is weighted by whether the citing sentence actually engages that paper’s treatment of the term. On our “embedding” map this is the difference between VGG, cited fourteen times but never once for its embedding contribution, and word2vec, cited for exactly that.
Displacement marks turning points: how far a paper’s definition sits from the average of every definition that came before it. The two measures are close to uncorrelated in practice, which is the point. The most-built-on paper usually consolidates a settled meaning; the paper that moved a term furthest is often lightly cited within the corpus, because the field it moved into is a different literature. Ranking on uptake alone would systematically hide exactly the papers a researcher is looking for.
The six link types
The type is the model’s classification. The citing sentence beside it is not, so you can check the reading against the evidence in one glance.
What you can and cannot rely on
A map mixes three kinds of statement, and they are marked differently everywhere they appear because conflating them is the failure mode this tool exists to avoid.
Quoted passages carrying the ✓ stamp. These were string-matched against our own extraction of the paper's text, so the words are the paper's and not the tool's. The guarantee stops exactly there: converting a paper into plain text can drop a symbol or a phrase, and where we know that happened the paper's page says so.
Citation sentences from Semantic Scholar, and passages marked ○ inferred. Real text from a real source, but drawn from a different extraction than ours, so we cannot mechanically confirm it against the paper.
Link types, confidence scores, the thread groupings, and any sentence labelled “the tool’s reading”. Generated from the evidence shown beside it. Treat as a well-informed starting point, never as a citable claim.
The practical rule: quote what is verified, verify what is sourced, and check anything that is judgment against the evidence sitting next to it.
Limitations and scope
What a map covers, and what it does not. These are properties of the method rather than defects to be patched, so they are worth knowing before you rely on one.
- Coverage is arXiv-only. Fields that publish through journals, conferences or books are underrepresented or absent, and a term’s pre-arXiv history is invisible. If your field does not preprint, this is not the right tool yet.
- A corpus is bounded, and the bound is arbitrary. Up to 150 papers, ranked by relevance and citation weight. That is a defensible sample, not the literature. Papers outside it are not judged unimportant; they were not examined.
- Verification is exact string matching. When extraction mangles a passage, common in maths-heavy LaTeX and scanned PDFs, a genuine quote fails the check and is marked inferred. Some inferred quotes are perfectly real. The check is conservative on purpose: it fails towards disclosure.
- Links need a citation. Parallel discovery, where two groups reach the same idea without citing each other, produces no link. The map shows unconnected work as unconnected rather than inventing a relationship.
- The reading is a model’s. Link types, threads and contributions are generated. They are grounded in quoted evidence and shown beside it, but they are not peer review, and a confidence score is the model’s own estimate of how sure it is, not a measurement of anything.
- It is a draft until a person checks it. Anything nobody has been through by hand is exactly what the machine produced, unedited. The audit trail on each map records what a person changed and when, so you can see how much human judgment a given map has had.
Draft, then judgment
What the pipeline produces is a draft. A curator can rename a thread, merge over-split ones, delete weak groupings, and reclassify or remove links. Every such decision is appended to the map’s audit trail, so a curated map always carries the record of what a human changed and when. No edit rewrites a quote or changes a verification result: curation reorganises the reading, never the evidence.
Reproducibility
Every map records the corpus it examined, the extraction prompt version behind each definition, and the model that produced it. Re-running a term with the same prompt version reuses the cached per-paper extractions, so a map can be rebuilt without re-reading the corpus, and a changed prompt is a new version rather than a silent overwrite. If you want to audit a claim, each paper links to its arXiv source and the passage is quoted, so the check takes seconds.
The whole thing is open: gitlab.com/rgok001/lineage. The pipeline that chooses the corpus, reads the papers and checks the quotes is there to be read, alongside an as-built record of what the system does, where it has failed, and what those failures changed. A method you cannot inspect is one you have to take on trust.
Requesting a map
Anyone signed in can request a term. Reading a corpus costs real money in model usage, so requests are reviewed before they run. Approved runs report progress as they go: which paper is being read, what each step found, and what the run has cost so far. A 150-paper corpus takes about fifteen minutes.