citing
citing is a python script that lists every work citing a paper, from three open citation indexes merged into one list, and checks that list against my BibTeX files and against a textâa manuscript, a wiki pageâthat already cites some of them. The paper is named by its key in my library, like everything else here: citing sanchezmunoz14a lists the works citing the bundler.
Written with the help of Claude Opus 5.5 on 15 October (2026). Version 2.0.2. It is public, on GitHub, under the MIT license. Its sibling vip uses it to grow the tree of the works citing a paper.
The idea
No index has every citation. OpenAlex, OpenCitations and Semantic Scholar each miss some that the others have, and each lists a paper in its own way: one by its journal DOI, another by its arXiv number, a third by a record parsed from a PDF with no identifier at all. For the bundler they give 208, 201 and 199 citing works, and 232 once merged: none of the three reaches nine tenths of the whole.
citing asks all three, joins what belongs togetherâby DOI, arXiv number, PubMed id, the journal DOI that arXiv records for a preprint and, for the records with no identifier, by titleâand then says of each work whether I already have it: under a key in the text I am keeping, in my bibliography but not yet in that text, or nowhere.
Usage
citing sanchezmunoz14a # every work citing it, with its key when it has one
citing sanchezmunoz14a -m # only those not in my bib files
citing sanchezmunoz14a -k notes.tex -m # those that notes.tex does not cite yet
citing sanchezmunoz14a -w PAGE -m # those that a page of this wiki does not list yet
citing sanchezmunoz14a -m -f ids # their DOIs or arXiv numbers, for doi2bib and arXiv2bib
citing sanchezmunoz14a -s 2024 -f csv # a spreadsheet of the recent ones
citing 10.1038/nphoton.2014.114 # a DOI, an arXiv number or an OpenAlex id work too
The keys of a text are read from \cite{...}, {{key}}, {{cite|a,b}} and the like, so that the same command keeps a manuscript's bibliography or a list on this wiki up to date. -o "Laussy,del Valle" marks our own papers with a star and counts them apart. The ids format feeds doi2bib and arXiv2bib, which turn the new ones into entries of my bases.
citing -- every work that cites a paper, from three open citation indexes,
merged into one list and checked against your BibTeX files and your notes.
Usage:
citing PAPER [options]
PAPER is a DOI, an arXiv id (arXiv:2604.24204, cond-mat/0601234), an OpenAlex
id (W2067552737), or a key of the --bib files: its DOI is then read from the
entry's doi or url field (url = {doi:...} works), its arXiv id from eprint,
journal or url, and an entry with neither is looked up by title in OpenAlex.
Sources (all free, no account needed):
OpenAlex filter=cites:<work>
OpenCitations index v2 (its arXiv-only and PubMed-only rows are named
through the arXiv API and PubMed)
Semantic Scholar citations, with the sentences that cite (--contexts)
Preprints are joined to their published version through the doi arXiv records
for them; works are also joined by DOI, PubMed id and normalised title.
Crossref's own count of citing works is printed as a check on coverage.
Options:
-b, --bib FILES BibTeX file(s), comma-separated: citing works are
given the key they have there (DOI, arXiv id, title)
-k, --known FILE a text holding the keys already accounted for, e.g.
a LaTeX file or wiki source ("-" = stdin). Keys are
read from \cite{..}, {{key}}, {{cite|a,b}} and the
like; with --bib only real bib keys count
-w, --wiki-page TITLE read the known keys from a MediaWiki page instead
--wiki-api URL that wiki's api.php (or env CITING_WIKI_API)
-o, --own NAMES surnames, comma-separated, marking self-citations
("Laussy,del Valle"); shown with a * and counted apart
-s, --since YEAR only citing works from YEAR on
-m, --missing only works not among the known keys (or, without
--known, not in the bib files)
-f, --format FMT text (default), csv, json, keys (bib keys, one per
line), ids (DOI or arXiv:ID of works with no key --
ready for a doi-to-bibtex tool)
--contexts under each work, the sentences that cite (Semantic
Scholar; often taken from the arXiv version)
--hunt QUERY also look for citations the indexes failed to link:
works matching QUERY in title/abstract (OpenAlex)
are kept if their Crossref reference list cites the
paper. Repeatable
--hunt-since YEAR start of the hunt (default: the paper's year)
--hunt-max N candidates checked per query (default 300)
--mailto EMAIL contact for the OpenAlex/Crossref polite pools
(or env CITING_MAILTO); nothing is sent without it
--s2-key KEY Semantic Scholar API key (or env S2_API_KEY);
without one, rate limits are met by waiting
--openalex-key K OpenAlex API key (or env OPENALEX_API_KEY); without
one, OpenAlex allows a daily budget per address
-q, --quiet no progress or summary on stderr
-V, --version the version
-h, --help this message
Status column: K known, b in a bib file but not known, N new (no key).
Sources column: A OpenAlex, C OpenCitations, S Semantic Scholar,
X found by --hunt in a Crossref reference list.
Defaults: ~/.citingrc (or the file named by env CITING_RC) holds options, as
on the command line, one or more per line; # starts a comment. An option
given on the command line replaces the one from the file, e.g.
--bib ~/bib/refs.bib,~/bib/preprints.bib
--mailto [email protected]
What comes out
On the bundler, against the wiki page where I keep the papers citing it, citing sanchezmunoz14a -w PAGE -o "Laussy,del Valle" -m gives the works the page does not list yet:
N 2013 arXiv:1310.2677 Carreño Dynamics of a three-photon state interacting with a quantum C
b* 2014 arXiv_silva14a Silva Measuring photon correlations simultaneously in time and fre A
N 2015 Bakker Cavity quantum electrodynamics with quantum dots in microcav S
N* 2018 arXiv:1802.04540 Carreño Photon correlations in both time and frequency AS
N 2018 10.6084/m9.figshare.c.43 Cortese Strong coupling of ionising transitions S
N 2018 Fischer Fischer, Kevin A. and Kelaita, Yousif A. and Sapra, Neil V. S
b 2021 arXiv_cosacchi21a Cosacchi Suitability of solid-state platforms as sources of $N$-photo AS
N 2022 Rivera n -photon bound states in the continuum for strong intensity S
N 2023 arXiv:2310.02309 Rinaldi Parameter estimation by learning quantum correlations in con S
N 2025 arXiv:2503.12384 Song On-demand manipulation of superbunching emission from colloi S
N 2026 arXiv:2604.15605 Tang Deterministic multiphoton bundle emission via interference-i S
and, on the error output so that the list itself can be piped:
citing 2.0.2: works citing sanchezmunoz14a (doi:10.1038/nphoton.2014.114 W2067552737)
OpenAlex: 208
OpenCitations: 201
Semantic Scholar: 199
arXiv: 33 preprint ids
Crossref: naming 1
232 citing works (Crossref counts 200 published ones); 20 self-citations
221 known, 2 in the bib files but not known, 9 with no key
The first column is the status: K known (its key is in the text or the page), b in a bib file but not in the text, N new, with no key anywhere; a star marks our own papers. The last column says who found the work: A OpenAlex, C OpenCitations, S Semantic Scholar, X the hunt (below). What is left once a page is up to date is a residue to be looked at by hand: preprints, a thesis, our own preprints, a record whose "title" is a list of authors. Crossref's own count of the citing works is printed as a check on the coverage.
The paper by its key
With my bibliography known (see Defaults), the paper is named by its key. Its DOI is read from the entry, where I keep it as url = {doi:10.1038/nphoton.2014.114}; an arXiv_ entry is followed through its arXiv number. Some 360 entries of sci.bib, mostly old papers, have neither: these are looked up in OpenAlex by title, year and first author, and citing says which work it took:
$ citing hanle24a
hanle24a has no DOI in the bib; OpenAlex W1970887935 by title: 10.1007/bf01331827
citing 2.0.2: works citing hanle24a (doi:10.1007/bf01331827 W1970887935)
OpenAlex: 658
OpenCitations: 616
Semantic Scholar: 469
arXiv: 9 preprint ids
Crossref: naming 5
662 citing works (Crossref counts 631 published ones)
2 in the bib files, 660 with no key
N 1925 10.1007/bf02980594 Born Quantentheorie und Molekelbildung ACS
N 1925 10.1007/978-3-642-61659- Heisenberg Ăber eine Anwendung des Korrespondenzprinzips auf die Frage ACS
N 1925 10.1002/andp.19253821104 Kossel PrĂŒfung von DâLeuchten, das von einem nahezu parallelen Elek ACS
N 1926 10.1002/andp.19263840504 Kerschbaum Ăber Messungen der Leuchtdauer der Atome an Alkalimetallen, ACS
N 1926 10.1007/bf01451773 Ladenburg Die quantentheoretische Dispersionsformel und ihre experimen ACS
...
OpenAlex's search drops a word that has lost its accentâHanle's Ăber becomes Uber once the TeX is strippedâso the search is tried again with the first author's name and without the accented words. A key that does not exist gets the nearest ones:
$ citing sanchezmunoz14z
citing: no key "sanchezmunoz14z" in the bib files (close: sanchezmunoz14b, sanchezmunoz14a, sanchezmunoz18a, sanchezmunoz15a, sanchezmunoz20b)
Hunting
An index knows a citation only if it matched the reference, and a garbled author name or a missing DOI in a reference list is enough to lose it in all three at once. --hunt "photon bundle" searches OpenAlex for works on the subject that are not in the list, reads each one's reference list as deposited with Crossref, and keeps those that cite the paperâby its DOI, its title, or its volume, page, year and first author. On the bundler, 787 candidates gave nothing new: beyond the three indexes, only Google Scholar is left.
Defaults
Options used every time go in ~/.citingrc, written as on the command line; one given on the command line replaces the one from the file. Mine names my three bases and this wiki:
# defaults for citing (https://github.com/laussy/citing); the command line overrides
--bib ~/bib/sci.bib,~/bib/arXiv.bib,~/bib/Books.bib
--wiki-api http://localhost/laussywiki/api.php
Limitations
- Google Scholar is not used: it has no interface for scripts and refuses them. In physics it finds a third to a half more than Crossrefâtheses, conference abstracts, book chapters, journals outside Crossref, preprints never published.
- A preprint whose title changed on publication, and whose arXiv record does not give the journal DOI, stays a separate line, to be checked by hand.
- Semantic Scholar limits the rate of anonymous calls, and citing then waits rather than fails. Otherwise a run is quick: a quarter of a minute for the bundler, half a minute for the 662 works citing Hanle.
- OpenAlex gives each address a daily budget of 1000 credits; a citation query costs one, a title search about ten.
Version history
- 1.0.0 (15 October (2026)): first version, as citers, for a DOI.
- 2.0.0 (15 October (2026)): renamed citing; the paper by its key in the bib files (found by title when the entry has no DOI), by its arXiv number or by its OpenAlex id; defaults in
~/.citingrc; its functions importable, for vip. - 2.0.1 (15 October (2026)): an answer cut off in transfer is asked again instead of stopping the run, as it had stopped Hanle's.
- 2.0.2 (15 October (2026)): Crossref is asked only for the DOIs that no index has named, instead of one call per row of OpenCitations, which took most of a run's time: the bundler's now takes 15 seconds instead of five and a half minutes.
The file
~/bin/citing
#!/usr/bin/env python3
# _ _ _
# ___(_) |_(_)_ __ __ _
# / __| | __| | '_ \ / _` |
# | (__| | |_| | | | | (_| |
# \___|_|\__|_|_| |_|\__, |
# |___/
#
# F. P. Laussy -- https://github.com/laussy/citing
# v1.0.0 Thu 15 Oct 2026 - first version (as "citers"): every work citing a
# DOI, from OpenAlex, OpenCitations and Semantic
# Scholar, merged, matched to BibTeX keys and compared
# with the keys a text (or a MediaWiki page) already
# has.
# v2.0.0 Thu 15 Oct 2026 - renamed citing (env CITING_*); the paper can be a
# BibTeX key of the --bib files, an arXiv id or an
# OpenAlex id as well as a DOI (a key with no DOI is
# found by title); defaults in ~/.citingrc; the
# functions are importable (vip builds on them).
# v2.0.1 Thu 15 Oct 2026 - an answer cut off in transfer (IncompleteRead) or a
# dropped connection is retried instead of fatal.
# v2.0.2 Thu 15 Oct 2026 - Crossref is asked only for the DOIs no index has
# named (it was asked for every OpenCitations row).
"""
citing -- every work that cites a paper, from three open citation indexes,
merged into one list and checked against your BibTeX files and your notes.
Usage:
citing PAPER [options]
PAPER is a DOI, an arXiv id (arXiv:2604.24204, cond-mat/0601234), an OpenAlex
id (W2067552737), or a key of the --bib files: its DOI is then read from the
entry's doi or url field (url = {doi:...} works), its arXiv id from eprint,
journal or url, and an entry with neither is looked up by title in OpenAlex.
Sources (all free, no account needed):
OpenAlex filter=cites:<work>
OpenCitations index v2 (its arXiv-only and PubMed-only rows are named
through the arXiv API and PubMed)
Semantic Scholar citations, with the sentences that cite (--contexts)
Preprints are joined to their published version through the doi arXiv records
for them; works are also joined by DOI, PubMed id and normalised title.
Crossref's own count of citing works is printed as a check on coverage.
Options:
-b, --bib FILES BibTeX file(s), comma-separated: citing works are
given the key they have there (DOI, arXiv id, title)
-k, --known FILE a text holding the keys already accounted for, e.g.
a LaTeX file or wiki source ("-" = stdin). Keys are
read from \\cite{..}, {{key}}, {{cite|a,b}} and the
like; with --bib only real bib keys count
-w, --wiki-page TITLE read the known keys from a MediaWiki page instead
--wiki-api URL that wiki's api.php (or env CITING_WIKI_API)
-o, --own NAMES surnames, comma-separated, marking self-citations
("Laussy,del Valle"); shown with a * and counted apart
-s, --since YEAR only citing works from YEAR on
-m, --missing only works not among the known keys (or, without
--known, not in the bib files)
-f, --format FMT text (default), csv, json, keys (bib keys, one per
line), ids (DOI or arXiv:ID of works with no key --
ready for a doi-to-bibtex tool)
--contexts under each work, the sentences that cite (Semantic
Scholar; often taken from the arXiv version)
--hunt QUERY also look for citations the indexes failed to link:
works matching QUERY in title/abstract (OpenAlex)
are kept if their Crossref reference list cites the
paper. Repeatable
--hunt-since YEAR start of the hunt (default: the paper's year)
--hunt-max N candidates checked per query (default 300)
--mailto EMAIL contact for the OpenAlex/Crossref polite pools
(or env CITING_MAILTO); nothing is sent without it
--s2-key KEY Semantic Scholar API key (or env S2_API_KEY);
without one, rate limits are met by waiting
--openalex-key K OpenAlex API key (or env OPENALEX_API_KEY); without
one, OpenAlex allows a daily budget per address
-q, --quiet no progress or summary on stderr
-V, --version the version
-h, --help this message
Status column: K known, b in a bib file but not known, N new (no key).
Sources column: A OpenAlex, C OpenCitations, S Semantic Scholar,
X found by --hunt in a Crossref reference list.
Defaults: ~/.citingrc (or the file named by env CITING_RC) holds options, as
on the command line, one or more per line; # starts a comment. An option
given on the command line replaces the one from the file, e.g.
--bib ~/bib/refs.bib,~/bib/preprints.bib
--mailto [email protected]
"""
import http.client
import json
import os
import re
import sys
import time
import unicodedata
import urllib.error
import urllib.parse
import urllib.request
import xml.etree.ElementTree as ET
from difflib import SequenceMatcher
__version__ = '2.0.2'
UA = 'citing/%s (+https://github.com/laussy/citing)' % __version__
QUIET = False
MAILTO = None
OPENALEX_KEY = None
CALLS = {} # host -> requests made
BUDGET = {} # host -> rate-limit headers of its last answer
def say(*a):
if not QUIET:
print(*a, file=sys.stderr, flush=True)
# --------------------------------------------------------------------------
# HTTP
# --------------------------------------------------------------------------
_direct = urllib.request.build_opener(urllib.request.ProxyHandler({}))
_default = urllib.request.build_opener()
def get(url, params=None, headers=None, tries=5, wait=5, as_json=True):
"""GET with retries on rate limits and server hiccups. A local host is
reached without the environment's proxy (a proxy set for the outside world
usually cannot see localhost)."""
host = urllib.parse.urlsplit(url).hostname or ''
if host == 'api.openalex.org' and OPENALEX_KEY:
params = dict(params or {}, api_key=OPENALEX_KEY)
if params:
url += ('&' if '?' in url else '?') + urllib.parse.urlencode(params)
opener = _direct if host in ('localhost', '127.0.0.1', '::1') else _default
ua = UA + (' mailto:%s' % MAILTO if MAILTO else '')
for attempt in range(tries):
try:
req = urllib.request.Request(url, headers={'User-Agent': ua, **(headers or {})})
CALLS[host] = CALLS.get(host, 0) + 1
with opener.open(req, timeout=90) as r:
data = r.read()
rl = {k.lower(): v for k, v in r.headers.items() if k.lower().startswith('x-ratelimit')}
if rl:
BUDGET[host] = rl
return json.loads(data) if as_json else data.decode('utf-8', 'replace')
except urllib.error.HTTPError as e:
if e.code == 404:
return None
if e.code in (429, 500, 502, 503, 504) and attempt < tries - 1:
time.sleep(wait)
wait = min(wait * 2, 60)
continue
raise
except (urllib.error.URLError, TimeoutError, http.client.HTTPException, ConnectionError):
# also an answer cut off on the way (IncompleteRead) or a dropped connection
if attempt < tries - 1:
time.sleep(wait)
wait = min(wait * 2, 60)
continue
raise
# --------------------------------------------------------------------------
# Normalisation
# --------------------------------------------------------------------------
def strip_accents(s):
return ''.join(c for c in unicodedata.normalize('NFKD', s) if not unicodedata.combining(c))
TEX_ACCENT = re.compile(r'\\[`\'^"~=.uvHtcdbk]\s*\{?\\?([a-zA-Z])\}?')
def detex(s):
"""TeX accents and braces off: {S\\'anchez Mu\\~noz} -> Sanchez Munoz."""
s = TEX_ACCENT.sub(r'\1', s or '')
s = re.sub(r'\\(ss|o|O|l|L|aa|AA|ae|AE|i)\b', lambda m: {'ss': 'ss', 'i': 'i'}.get(m.group(1), m.group(1)[0]), s)
return s.replace('{', '').replace('}', '').replace('~', ' ')
def ntitle(t):
"""A title reduced to lowercase letters and digits, markup removed."""
t = re.sub(r'<[^>]+>', '', t or '') # HTML / MathML tags
t = re.sub(r'\$|\\[a-zA-Z]+', '', detex(t)) # TeX maths and commands
return re.sub(r'[^a-z0-9]', '', strip_accents(t).lower())
def ndoi(d):
d = (d or '').strip().lower()
d = re.sub(r'^(https?://)?(dx\.)?doi\.org/', '', d)
d = re.sub(r'^doi:\s*', '', d)
return d.rstrip('.,;') or None
def narxiv(a):
a = (a or '').strip()
a = re.sub(r'^(arxiv:|https?://arxiv\.org/abs/)', '', a, flags=re.I)
return re.sub(r'v\d+$', '', a) or None
def arxiv_of_doi(d):
m = re.match(r'10\.48550/arxiv\.(.+)$', d or '')
return narxiv(m.group(1)) if m else None
def surname(name):
m = re.search(r'\{([^{}]*\s[^{}]*)\}\s*$', name or '') # BibTeX {Lopez Carreno}
if m:
return detex(m.group(1)).strip()
name = detex(name).strip()
if ',' in name:
return name.split(',')[0].strip()
parts = name.split()
return parts[-1] if parts else ''
# --------------------------------------------------------------------------
# Records and their merging
# --------------------------------------------------------------------------
class Rec:
def __init__(self, src=None):
self.src = {src} if src else set() # no source: metadata lookup only
self.dois = set()
self.arxiv = set()
self.pmid = set()
self.title = None
self.authors = []
self.year = None
self.venue = None
self.type = None
self.contexts = []
self.published = False # metadata comes from the journal, not a preprint
def ids(self):
out = ['doi:' + d for d in self.dois] + ['arxiv:' + a for a in self.arxiv] + ['pmid:' + p for p in self.pmid]
nt = ntitle(self.title)
if len(nt) >= 20:
out.append('title:' + nt)
return out
def add_doi(rec, d):
d = ndoi(d)
if not d:
return
a = arxiv_of_doi(d)
if a:
rec.arxiv.add(a)
else:
rec.dois.add(d)
def merge_all(recs):
"""Union-find over shared identifiers; returns merged records."""
parent = list(range(len(recs)))
def find(i):
while parent[i] != i:
parent[i] = parent[parent[i]]
i = parent[i]
return i
owner = {}
for i, r in enumerate(recs):
for k in r.ids():
if k in owner:
parent[find(i)] = find(owner[k])
else:
owner[k] = i
groups = {}
for i in range(len(recs)):
groups.setdefault(find(i), []).append(recs[i])
out = []
for members in groups.values():
# published metadata first, then the richest
members.sort(key=lambda r: (not r.published, not r.title, -len(r.authors)))
m = Rec()
for r in members:
m.src |= r.src
m.dois |= r.dois
m.arxiv |= r.arxiv
m.pmid |= r.pmid
m.contexts += [c for c in r.contexts if c not in m.contexts]
for f in ('title', 'year', 'venue', 'type'):
if getattr(m, f) is None and getattr(r, f):
setattr(m, f, getattr(r, f))
if not m.authors and r.authors:
m.authors = r.authors
m.published = m.published or r.published
out.append(m)
return out
def destamp(title):
"""Drop the arXiv watermark a PDF parser can leave in front of a title,
spaced out into crumbs: "qu an tph ] 1 4 Se p 20 17 1 Nonlinear ...".
Only a run of four or more short crumbs, two of them with digits, is taken."""
toks = (title or '').split()
n = 0
while n < len(toks) and (len(toks[n]) <= 3 or re.search(r'[\d\]\[]', toks[n])):
n += 1
if n >= 4 and n < len(toks) and sum(bool(re.search(r'\d', t)) for t in toks[:n]) >= 2:
return ' '.join(toks[n:])
return title
def similar(a, b):
"""How alike two normalised titles are, 0..1. A long shared opening counts
as a match: index titles often carry junk in front (a PDF's arXiv stamp,
"N ov 2 01 8 ...") or lose their end."""
if not a or not b:
return 0.0
if min(len(a), len(b)) >= 50 and (a[:40] in b or b[:40] in a):
return 1.0
sm = SequenceMatcher(None, a, b, autojunk=False)
return sm.ratio() if sm.real_quick_ratio() > 0.8 and sm.quick_ratio() > 0.8 else 0.0
def fuzzy_join(works, cut=0.9):
"""Fold records with no identifier at all (Semantic Scholar has some, from
papers it parsed badly) into the work their title is closest to."""
anchored = [w for w in works if w.dois or w.arxiv or w.pmid]
keep = list(anchored)
for w in works:
if w in anchored:
continue
nt = ntitle(destamp(w.title))
best, score = None, 0.0
for a in anchored:
sc = similar(nt, ntitle(a.title))
if sc > score:
best, score = a, sc
if best is not None and score >= cut:
best.src |= w.src
best.contexts += [c for c in w.contexts if c not in best.contexts]
else:
keep.append(w)
return keep
# --------------------------------------------------------------------------
# Sources
# --------------------------------------------------------------------------
def crossref(doi):
p = {'mailto': MAILTO} if MAILTO else None
r = get('https://api.crossref.org/works/' + urllib.parse.quote(doi), p)
return r['message'] if r else None
def from_crossref(m, src):
r = Rec(src)
add_doi(r, m.get('DOI'))
r.title = ' '.join(m.get('title') or []) or None
r.authors = [(a.get('given', '') + ' ' + a.get('family', '')).strip() or a.get('name', '')
for a in m.get('author', [])]
dp = (m.get('issued') or m.get('published') or {}).get('date-parts') or [[None]]
r.year = dp[0][0]
r.venue = ' '.join(m.get('container-title') or []) or None
r.type = m.get('type')
r.published = m.get('type') != 'posted-content'
return r
def openalex_work(ident, select='id,doi,title,publication_year,cited_by_count,authorships'):
"""One OpenAlex work, by W-id or 'doi:...'; None if OpenAlex has none."""
p = {'mailto': MAILTO} if MAILTO else {}
return get('https://api.openalex.org/works/' + urllib.parse.quote(ident, safe=':/'), dict(p, select=select))
def oa_rec(x, src='A'):
"""An OpenAlex work as a Rec."""
r = Rec(src)
add_doi(r, x.get('doi'))
ids = x.get('ids') or {}
if ids.get('pmid'):
r.pmid.add(ids['pmid'].rsplit('/', 1)[-1])
loc = x.get('primary_location') or {}
lp = loc.get('landing_page_url') or ''
if 'arxiv.org/abs/' in lp:
r.arxiv.add(narxiv(lp.split('/abs/')[-1]))
r.title = x.get('title')
r.authors = [a['author']['display_name'] for a in x.get('authorships', [])]
r.year = x.get('publication_year')
r.venue = (loc.get('source') or {}).get('display_name')
r.type = x.get('type')
r.published = r.type not in ('preprint',) and bool(r.dois)
return r
def openalex(wid):
"""The works citing OpenAlex work wid."""
p = {'mailto': MAILTO} if MAILTO else {}
# page numbers, not cursors: without an API key OpenAlex answers a
# next-page cursor with 401 (2026); plain paging reaches 10,000 works
out, page = [], 1
sel = 'id,doi,title,publication_year,authorships,primary_location,type,ids'
while page:
d = get('https://api.openalex.org/works',
dict(p, filter='cites:' + wid, page=page, select=sel, **{'per-page': 200}))
out += [oa_rec(x) for x in d['results']]
page = page + 1 if len(d['results']) == 200 and page < 50 else None
return out
def opencitations(doi):
d = get('https://api.opencitations.net/index/v2/citations/doi:' + doi) or []
out = []
for c in d:
r = Rec('C')
for tok in c.get('citing', '').split():
kind, _, val = tok.partition(':')
if kind == 'doi':
add_doi(r, val)
elif kind == 'arxiv':
r.arxiv.add(narxiv(val))
elif kind == 'pmid':
r.pmid.add(val)
if c.get('creation'):
r.year = int(c['creation'][:4])
out.append(r)
return out
def semanticscholar(pid, key=None):
"""Citations of a paper, pid = 'DOI:...' or 'ARXIV:...'."""
h = {'x-api-key': key} if key else None
out, offset = [], 0
while offset is not None:
d = get('https://api.semanticscholar.org/graph/v1/paper/%s/citations' % pid,
{'fields': 'contexts,title,year,externalIds,authors,venue',
'limit': 1000, 'offset': offset}, headers=h, tries=7, wait=10)
if not d:
break
for c in d.get('data', []):
p = c['citingPaper']
e = p.get('externalIds') or {}
r = Rec('S')
add_doi(r, e.get('DOI'))
if e.get('ArXiv'):
r.arxiv.add(narxiv(e['ArXiv']))
if e.get('PubMed'):
r.pmid.add(str(e['PubMed']))
r.title = p.get('title')
r.authors = [a['name'] for a in p.get('authors', [])]
r.year = p.get('year')
r.venue = p.get('venue') or None
r.contexts = c.get('contexts') or []
out.append(r)
offset = d.get('next')
return out
def arxiv_meta(ids):
"""arXiv id -> Rec (title, authors, year, and the journal doi if any)."""
ns = {'a': 'http://www.w3.org/2005/Atom', 'x': 'http://arxiv.org/schemas/atom'}
out = {}
ids = sorted(set(ids))
for k in range(0, len(ids), 50):
if k:
time.sleep(3) # arXiv asks for a pause between calls
xml = get('https://export.arxiv.org/api/query',
{'id_list': ','.join(ids[k:k + 50]), 'max_results': 50}, as_json=False, tries=6, wait=10)
for e in ET.fromstring(xml).findall('a:entry', ns):
i = narxiv(e.find('a:id', ns).text.split('/abs/')[-1])
r = Rec()
r.arxiv.add(i)
r.title = ' '.join((e.find('a:title', ns).text or '').split())
r.authors = [a.find('a:name', ns).text for a in e.findall('a:author', ns)]
r.year = int(e.find('a:published', ns).text[:4])
d = e.find('x:doi', ns)
if d is not None:
add_doi(r, d.text)
out[i] = r
return out
def pubmed_meta(pmids):
if not pmids:
return {}
d = get('https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi',
{'db': 'pubmed', 'id': ','.join(sorted(pmids)), 'retmode': 'json'})
out = {}
for p in sorted(pmids):
s = (d or {}).get('result', {}).get(p)
if not s:
continue
r = Rec()
r.pmid.add(p)
r.title = s.get('title')
r.authors = [a['name'] for a in s.get('authors', [])]
try:
r.year = int(s.get('pubdate', '')[:4])
except ValueError:
pass
for a in s.get('articleids', []):
if a.get('idtype') == 'doi':
add_doi(r, a['value'])
out[p] = r
return out
def enrich(recs):
"""Name what the indexes only give as identifiers, and tie preprints to
their journal version."""
# only preprint-only records need arXiv: the others are named already
ax = {a for r in recs if not r.dois for a in r.arxiv}
if ax:
say(' arXiv: %d preprint ids' % len(ax))
try:
recs += list(arxiv_meta(ax).values())
except urllib.error.HTTPError as e:
say(' arXiv API refused (%s): preprints stay unnamed and unjoined; try again later' % e.code)
# a DOI another index has already named needs no Crossref call: the
# records join on it anyway (Hanle 1924: 616 calls -> 5, 27 s in all)
named = {d for r in recs if r.title for d in r.dois}
bare = [r for r in recs if not r.title and r.dois and not r.dois & named]
if bare:
say(' Crossref: naming %d' % len(bare))
for r in bare:
best = None
for d in sorted(r.dois):
m = crossref(d)
if m and (best is None or m.get('type') == 'journal-article'):
best = m
if best:
recs.append(from_crossref(best, None))
pm = {p for r in recs if not r.title and not r.dois for p in r.pmid}
recs += list(pubmed_meta(pm).values())
return recs
# --------------------------------------------------------------------------
# --hunt: citations no index has linked
# --------------------------------------------------------------------------
def target_meta(seed):
doi = seed['doi'] if seed['doi'] and not seed['arxiv'] else None
m = (crossref(doi) if doi else None) or {}
au = m.get('author') or [{}]
dp = (m.get('issued') or {}).get('date-parts') or [[None]]
title = ' '.join(m.get('title') or []) or seed['title'] or ''
return {'doi': doi, 'nt': ntitle(title),
'volume': str(m.get('volume') or ''), 'page': str(m.get('page') or '').split('-')[0],
'year': str(dp[0][0] or seed['year'] or ''), 'surname': ntitle(au[0].get('family', '')),
'count': m.get('is-referenced-by-count'), 'title': title}
def cites_target(ref, t):
if t['doi'] and ndoi(ref.get('DOI')) == t['doi']:
return True
txt = ' '.join(str(ref.get(k, '')) for k in ('unstructured', 'article-title', 'journal-title', 'author'))
if len(t['nt']) >= 20 and t['nt'][:30] in ntitle(txt):
return True
if t['volume'] and t['page']:
if (str(ref.get('volume', '')) == t['volume'] and str(ref.get('first-page', '')) == t['page']
and str(ref.get('year', '')) == t['year']):
return True
u = ref.get('unstructured', '')
if (re.search(r'\b%s\b\D{1,8}\b%s\b' % (re.escape(t['volume']), re.escape(t['page'])), u)
and t['year'] in u and t['surname'] and t['surname'] in ntitle(u)):
return True
return False
def hunt(query, since, cap, known_dois, t):
p = {'mailto': MAILTO} if MAILTO else {}
q = query.replace(',', ' ')
cands, page = [], 1
while page and len(cands) < cap:
d = get('https://api.openalex.org/works',
dict(p, filter='title_and_abstract.search:%s,from_publication_date:%s-01-01' % (q, since),
page=page, select='doi', **{'per-page': 200}))
cands += [ndoi(x['doi']) for x in d['results'] if x.get('doi')]
page = page + 1 if len(d['results']) == 200 and page < 50 else None
cands = [c for c in dict.fromkeys(cands[:cap]) if c not in known_dois and not arxiv_of_doi(c)]
say(' hunt "%s": %d candidates not already listed' % (query, len(cands)))
found = []
for i, c in enumerate(cands, 1):
m = crossref(c)
if m and any(cites_target(ref, t) for ref in m.get('reference', [])):
r = from_crossref(m, 'X')
found.append(r)
say(' + %s %s' % (c, (r.title or '')[:70]))
if i % 50 == 0:
say(' %d/%d checked' % (i, len(cands)))
return found
# --------------------------------------------------------------------------
# BibTeX
# --------------------------------------------------------------------------
def bib_entries(text):
"""(key, {field: value}) for every @entry, braces matched by hand (an entry
may close on its last field line)."""
for m in re.finditer(r'@(\w+)\s*\{\s*([^,\s]+)\s*,', text):
if m.group(1).lower() in ('string', 'comment', 'preamble'):
continue
i, depth = m.end(), 1
while i < len(text) and depth:
depth += {'{': 1, '}': -1}.get(text[i], 0)
i += 1
body, fields = text[m.end():i - 1], {}
for f in re.finditer(r'(\w+)\s*=\s*', body):
j = f.end()
if j < len(body) and body[j] in '{"':
close = '}' if body[j] == '{' else '"'
k, d = j + 1, 1
while k < len(body) and d:
if body[k] == '{':
d += 1
elif body[k] == '}':
d -= 1
elif close == '"' and body[k] == '"' and d == 1:
d = 0
k += 1
break
k += 1
val = body[j + 1:k - 1]
else:
val = re.match(r'[^,\s}]*', body[j:]).group(0)
fields.setdefault(f.group(1).lower(), val)
yield m.group(2), fields
def entry_ids(f):
"""The doi:/arxiv: identifiers a BibTeX entry's fields give."""
ids = []
for v in (f.get('doi'), f.get('url')):
if v and ('10.' in v):
d = ndoi(re.sub(r'^.*?(10\.\d{4,9}/)', r'\1', v))
a = arxiv_of_doi(d)
ids.append('arxiv:' + a if a else 'doi:' + d)
for v in (f.get('eprint'), f.get('journal'), f.get('url')):
m = re.search(r'(?:arXiv:|arxiv\.org/abs/)\s*([\w.\-/]+\d)', v or '', re.I)
if m:
ids.append('arxiv:' + narxiv(m.group(1)))
if (f.get('archiveprefix') or '').lower() == 'arxiv' and f.get('eprint'):
ids.append('arxiv:' + narxiv(f['eprint']))
return list(dict.fromkeys(ids))
class Bib:
def __init__(self, paths):
self.by = {}
self.authors = {}
self.fields = {}
self.keys = set()
self.titles = [] # (normalised title, key, year), for records with no id
for p in paths:
text = open(os.path.expanduser(p), encoding='utf-8', errors='replace').read()
for key, f in bib_entries(text):
self.keys.add(key)
self.fields.setdefault(key, f)
self.authors[key] = f.get('author', '')
ids = entry_ids(f)
nt = ntitle(f.get('title'))
if len(nt) >= 20:
ids.append('title:' + nt)
y = re.search(r'\d{4}', f.get('year', ''))
self.titles.append((nt, key, int(y.group(0)) if y else None))
for i in ids:
self.by.setdefault(i, [])
if key not in self.by[i]:
self.by[i].append(key)
def match(self, rec, since=None):
out = []
for i in rec.ids():
for k in self.by.get(i, []):
if k not in out:
out.append(k)
if not out and not (rec.dois or rec.arxiv or rec.pmid):
nt = ntitle(destamp(rec.title))
# a work older than the cited paper cannot be citing it
best = max(((similar(nt, t), k) for t, k, y in self.titles if not (since and y and y < since)),
default=(0, None))
if best[0] >= 0.9:
out.append(best[1])
return out
# --------------------------------------------------------------------------
# Known keys
# --------------------------------------------------------------------------
KEY = re.compile(r'(?<![\w/.])((?:arXiv_)?[A-Za-z]+\d{2}[a-z]{1,3})(?![\w])')
def lcfirst(k):
return k if k.startswith('arXiv_') else k[:1].lower() + k[1:]
def known_keys(text, bib):
toks = {lcfirst(t) for t in KEY.findall(text)}
if bib:
real = {lcfirst(k): k for k in bib.keys}
return {real[t] for t in toks if t in real}
return toks
def wiki_text(api, title):
d = get(api, {'action': 'query', 'prop': 'revisions', 'rvprop': 'content', 'rvslots': 'main',
'titles': title, 'format': 'json', 'formatversion': 2})
page = d['query']['pages'][0]
if page.get('missing'):
sys.exit('citing: no page "%s" on %s' % (title, api))
return page['revisions'][0]['slots']['main']['content']
# --------------------------------------------------------------------------
# The paper: DOI, arXiv id, OpenAlex id or BibTeX key
# --------------------------------------------------------------------------
ARXIV_ID = re.compile(r'^(?:arxiv:\s*|https?://arxiv\.org/abs/)?(\d{4}\.\d{4,5}|[a-z][a-z.\-]*(?:\.[a-z]{2})?/\d{7})(v\d+)?$', re.I)
def oa_title_search(title, year=None, surname_=None):
"""The OpenAlex work with this title (TeX allowed; and year, give or take
one), or None. OpenAlex's search drops a word whose accent is missing
("Uber" for "Ăber"), so the search is retried with the first author's
surname and without the accented words."""
p = {'mailto': MAILTO} if MAILTO else {}
def clean(t):
t = re.sub(r'[^\w\s-]', ' ', strip_accents(re.sub(r'\$|\\[a-zA-Z]+', '', detex(t))))
return ' '.join(t.split())
plain = ' '.join(w for w in (title or '').split() if '\\' not in w and w.isascii())
tries = [(clean(title), None)]
if surname_:
tries.append((clean(title), clean(surname_)))
if plain != title and len(clean(plain).split()) >= 3:
tries.append((clean(plain), clean(surname_) if surname_ else None))
nt = ntitle(title)
for q, who in tries:
if not q:
continue
flt = 'title.search:' + q + (',raw_author_name.search:' + who if who else '')
d = get('https://api.openalex.org/works',
dict(p, filter=flt, select='id,doi,title,publication_year,authorships',
**{'per-page': 25})) or {'results': []}
best, score = None, 0.0
for x in d['results']:
sc = similar(nt, ntitle(x.get('title')))
if year and x.get('publication_year') and abs(x['publication_year'] - year) > 1:
sc -= 0.2
if surname_ and sc >= 0.9:
names = ' '.join(ntitle(a['author']['display_name']) for a in x.get('authorships', []))
if ntitle(surname_) not in names:
sc -= 0.05
if sc > score:
best, score = x, sc
if score >= 0.9:
return best
return None
def resolve(arg, bib=None, prog='citing'):
"""What the command line names -> {doi, arxiv, wid, key, title, year}.
A BibTeX key is looked up in bib; a key whose entry has neither DOI nor
arXiv id is found in OpenAlex by title."""
s = {'doi': None, 'arxiv': None, 'wid': None, 'key': None, 'title': None, 'year': None}
a = arg.strip()
m = re.match(r'^(?:https?://openalex\.org/)?(W\d{4,})$', a, re.I)
if m:
s['wid'] = m.group(1).upper()
return s
if re.match(r'^(?:(?:https?://)?(?:dx\.)?doi\.org/|doi:\s*)?10\.\d{4,9}/\S+$', a, re.I):
s['doi'] = ndoi(a)
s['arxiv'] = arxiv_of_doi(s['doi'])
if bib:
r = Rec()
add_doi(r, s['doi'])
s['key'] = (bib.match(r) or [None])[0]
return s
m = ARXIV_ID.match(a)
if m:
s['arxiv'] = narxiv(m.group(1))
s['doi'] = '10.48550/arxiv.' + s['arxiv'].lower()
if bib:
r = Rec()
r.arxiv.add(s['arxiv'])
s['key'] = (bib.match(r) or [None])[0]
return s
# a BibTeX key
if not bib:
sys.exit('%s: "%s" is no DOI, arXiv or OpenAlex id; a BibTeX key needs --bib (or a ~/.%src)'
% (prog, a, prog))
key = a if a in bib.fields else next((k for k in bib.fields if k.lower() == a.lower()), None)
if key is None:
import difflib
near = difflib.get_close_matches(a, list(bib.fields), n=5, cutoff=0.75)
sys.exit('%s: no key "%s" in the bib files%s' % (prog, a, ' (close: %s)' % ', '.join(near) if near else ''))
f = bib.fields[key]
s['key'], s['title'] = key, detex(f.get('title', '')).strip() or None
y = re.search(r'\d{4}', f.get('year', ''))
s['year'] = int(y.group(0)) if y else None
ids = entry_ids(f)
dois = [i[4:] for i in ids if i.startswith('doi:')]
axs = [i[6:] for i in ids if i.startswith('arxiv:')]
if dois:
s['doi'] = dois[0]
elif axs:
s['arxiv'] = axs[0]
s['doi'] = '10.48550/arxiv.' + axs[0].lower()
elif s['title']:
names = re.split(r'\s+and\s+', f.get('author', ''))
x = oa_title_search(f.get('title'), s['year'], surname(names[0]) if names[0] else None)
if not x:
sys.exit('%s: %s has no DOI or arXiv id, and OpenAlex has no work titled "%s"'
% (prog, key, s['title']))
s['wid'] = x['id'].rsplit('/', 1)[-1]
if x.get('doi'):
s['doi'] = ndoi(x['doi'])
s['arxiv'] = arxiv_of_doi(s['doi'])
say(' %s has no DOI in the bib; OpenAlex %s by title%s' % (key, s['wid'], ': ' + s['doi'] if s['doi'] else ''))
else:
sys.exit('%s: %s has no DOI, arXiv id or title' % (prog, key))
return s
def label(seed):
if seed['key']:
return seed['key']
if seed['arxiv']:
return 'arXiv:' + seed['arxiv']
return seed['doi'] or seed['wid']
# --------------------------------------------------------------------------
# Options and defaults
# --------------------------------------------------------------------------
def read_rc(env, name):
"""The options written in ~/.<name>rc (or the file env names), as tokens."""
import shlex
path = os.environ.get(env) or os.path.expanduser('~/.%src' % name)
try:
text = open(path, encoding='utf-8').read()
except OSError:
return []
out = []
for line in text.splitlines():
out += shlex.split(line, comments=True)
return out
def parse_options(argv, o, takes, flags, lists, ints, prog, positional, given=None):
"""Read argv into o. lists: options that may repeat (comma-separated values
are split, except for 'raw' ones); ints: options holding a number.
Returns the set of options given."""
given = set() if given is None else given
seen_lists = set()
i = 0
argv = list(argv)
while i < len(argv):
a = argv[i]
if '=' in a and a.split('=', 1)[0] in takes:
a, val = a.split('=', 1)
argv[i:i + 1] = [a, val]
if a in takes:
if i + 1 >= len(argv):
sys.exit('%s: %s needs a value' % (prog, a))
k, v = takes[a], argv[i + 1]
if k in lists:
if k not in seen_lists: # the first use replaces the default
o[k], seen_lists = [], seen_lists | {k}
o[k] += [v] if lists[k] == 'raw' else [x.strip() for x in v.split(',') if x.strip()]
elif k in ints:
try:
o[k] = int(v)
except ValueError:
sys.exit('%s: %s takes a number' % (prog, a))
else:
o[k] = v
given.add(k)
i += 2
continue
if a in flags:
o[flags[a]] = True
given.add(flags[a])
elif a.startswith('-') and a != '-':
sys.exit('%s: unknown option %s (%s -h)' % (prog, a, prog))
elif positional is None:
sys.exit('%s: no argument expected here: %s' % (prog, a))
else:
positional.append(a)
i += 1
return given
TAKES = {'-b': 'bib', '--bib': 'bib', '-k': 'known', '--known': 'known', '-w': 'wiki_page',
'--wiki-page': 'wiki_page', '--wiki-api': 'wiki_api', '-o': 'own', '--own': 'own',
'-s': 'since', '--since': 'since', '-f': 'format', '--format': 'format',
'--hunt': 'hunt', '--hunt-since': 'hunt_since', '--hunt-max': 'hunt_max',
'--mailto': 'mailto', '--s2-key': 's2_key', '--openalex-key': 'openalex_key'}
FLAGS = {'-m': 'missing', '--missing': 'missing', '--contexts': 'contexts', '-q': 'quiet', '--quiet': 'quiet'}
def parse_args(argv):
o = {'bib': [], 'known': None, 'wiki_page': None, 'wiki_api': os.environ.get('CITING_WIKI_API'),
'own': [], 'since': None, 'missing': False, 'format': 'text', 'contexts': False,
'hunt': [], 'hunt_since': None, 'hunt_max': 300, 'mailto': os.environ.get('CITING_MAILTO'),
's2_key': os.environ.get('S2_API_KEY'), 'openalex_key': os.environ.get('OPENALEX_API_KEY'),
'quiet': False}
for a in argv:
if a in ('-h', '--help'):
print(__doc__.strip())
sys.exit(0)
if a in ('-V', '--version'):
print('citing', __version__)
sys.exit(0)
lists = {'bib': 'split', 'own': 'split', 'hunt': 'raw'}
ints = ('since', 'hunt_since', 'hunt_max')
pos = []
given = parse_options(argv, o, TAKES, FLAGS, lists, ints, 'citing', pos)
rc = dict(o)
for k in lists:
rc[k] = []
rc_given = parse_options(read_rc('CITING_RC', 'citing'), rc, TAKES, FLAGS, lists, ints, 'citing (~/.citingrc)', None)
for k in rc_given - given:
o[k] = rc[k]
if not pos:
print(__doc__.strip())
sys.exit(1)
if len(pos) > 1:
sys.exit('citing: one paper at a time')
o['paper'] = pos[0]
if o['format'] not in ('text', 'csv', 'json', 'keys', 'ids'):
sys.exit('citing: --format is text, csv, json, keys or ids')
if o['wiki_page'] and not o['wiki_api']:
sys.exit('citing: --wiki-page needs --wiki-api (or env CITING_WIKI_API)')
return o
def main(argv):
global QUIET, MAILTO, OPENALEX_KEY
o = parse_args(argv)
QUIET, MAILTO, OPENALEX_KEY = o['quiet'], o['mailto'], o['openalex_key']
bib = Bib(o['bib']) if o['bib'] else None
known = None
if o['wiki_page']:
known = known_keys(wiki_text(o['wiki_api'], o['wiki_page']), bib)
elif o['known']:
txt = sys.stdin.read() if o['known'] == '-' else open(os.path.expanduser(o['known']), encoding='utf-8', errors='replace').read()
known = known_keys(txt, bib)
seed = resolve(o['paper'], bib)
if not seed['wid'] and seed['doi']:
w = openalex_work('doi:' + seed['doi'])
if w:
seed['wid'] = w['id'].rsplit('/', 1)[-1]
seed['title'] = seed['title'] or w.get('title')
seed['year'] = seed['year'] or w.get('publication_year')
ids = ' '.join(x for x in (seed['doi'] and 'doi:' + seed['doi'], seed['arxiv'] and 'arXiv:' + seed['arxiv'],
seed['wid']) if x and x != label(seed))
say('citing %s: works citing %s%s' % (__version__, label(seed), ' (%s)' % ids if ids else ''))
oa = openalex(seed['wid']) if seed['wid'] else []
say(' OpenAlex: %d%s' % (len(oa), '' if seed['wid'] else ' (paper unknown to OpenAlex)'))
plain_doi = seed['doi'] if seed['doi'] and not seed['arxiv'] else None
oc = opencitations(plain_doi or seed['doi']) if seed['doi'] else []
say(' OpenCitations: %d' % len(oc))
s2id = 'DOI:' + plain_doi if plain_doi else ('ARXIV:' + seed['arxiv'] if seed['arxiv'] else None)
s2 = semanticscholar(s2id, o['s2_key']) if s2id else []
say(' Semantic Scholar: %d' % len(s2))
t = target_meta(seed)
recs = enrich(oa + oc + s2)
works = fuzzy_join(merge_all(recs))
# the paper itself can come back through a preprint alias; drop it
works = [w for w in works if not (seed['doi'] and seed['doi'] in w.dois)
and not (seed['arxiv'] and seed['arxiv'] in w.arxiv)]
if o['hunt']:
listed = {d for w in works for d in w.dois}
found = []
for q in o['hunt']:
found += hunt(q, o['hunt_since'] or t['year'] or 1900, o['hunt_max'], listed, t)
listed |= {d for r in found for d in r.dois}
works = merge_all([w for w in works] + found)
words = lambda s: ' %s ' % re.sub(r'[^a-z]+', ' ', strip_accents(detex(s)).lower()).strip()
own = [words(n) for n in o['own']]
for w in works:
w.keys = bib.match(w, int(t['year']) - 1 if t['year'] else None) if bib else []
if known is not None and any(k in known for k in w.keys):
w.status = 'K'
w.key = next(k for k in w.keys if k in known)
elif w.keys:
w.status = 'b'
w.key = sorted(w.keys, key=lambda k: k.startswith('arXiv_'))[0]
else:
w.status, w.key = 'N', None
names = [detex(a) for a in re.split(r'\s+and\s+', bib.authors.get(w.key, ''))] if (bib and w.key) else w.authors
w.own = any(n.strip() and n in words(a) for n in own for a in names)
w.first = surname(names[0]) if names else '?'
sel = works
if o['since']:
sel = [w for w in sel if (w.year or 0) >= o['since']]
if o['missing']:
sel = [w for w in sel if w.status != 'K'] if known is not None else [w for w in sel if w.status == 'N']
sel.sort(key=lambda w: (w.year or 0, (w.key or w.first or '').lower()))
def ident(w):
d = sorted(w.dois)
return d[0] if d else ('arXiv:' + sorted(w.arxiv)[0] if w.arxiv else '')
srcs = lambda w: ''.join(c for c in 'ACSX' if c in w.src)
f = o['format']
if f == 'text':
for w in sel:
print('%s%s %s %-24s %-14s %-60s %s' % (w.status, '*' if w.own else ' ', w.year or '????',
(w.key or ident(w))[:24], w.first[:14], (w.title or '')[:60], srcs(w)))
if o['contexts']:
for c in w.contexts:
print(' > ' + ' '.join(c.split()))
elif f == 'csv':
import csv
cw = csv.writer(sys.stdout)
cw.writerow(['status', 'own', 'year', 'key', 'doi', 'arxiv', 'first_author', 'title', 'venue', 'sources'])
for w in sel:
cw.writerow([w.status, int(w.own), w.year or '', w.key or '', ' '.join(sorted(w.dois)),
' '.join(sorted(w.arxiv)), w.first, w.title or '', w.venue or '', srcs(w)])
elif f == 'json':
json.dump([{'status': w.status, 'own': w.own, 'year': w.year, 'key': w.key, 'keys': w.keys,
'dois': sorted(w.dois), 'arxiv': sorted(w.arxiv), 'pmid': sorted(w.pmid),
'title': w.title, 'authors': w.authors, 'venue': w.venue, 'type': w.type,
'sources': srcs(w), 'contexts': w.contexts} for w in sel],
sys.stdout, indent=1, ensure_ascii=False)
print()
elif f == 'keys':
for w in sel:
if w.key:
print(w.key)
elif f == 'ids':
for w in sel:
if w.status == 'N' and ident(w):
print(ident(w))
n = {s: sum(w.status == s for w in works) for s in 'KbN'}
say('%d citing works (Crossref counts %s published ones)%s' % (
len(works), t['count'] if t['count'] is not None else '?',
'; %d self-citations' % sum(w.own for w in works) if own else ''))
if bib and known is not None:
say(' %d known, %d in the bib files but not known, %d with no key' % (n['K'], n['b'], n['N']))
elif bib:
say(' %d in the bib files, %d with no key' % (n['b'], n['N']))
if __name__ == '__main__':
try:
main(sys.argv[1:])
except KeyboardInterrupt:
sys.exit(130)
except BrokenPipeError:
pass