syncloss
syncloss is a python script that reads the online history of laussy.org and reports the edits the sync has eaten: words that were on the page once and are on neither wiki now.
Written with Claude Opus 5 on 24 August (2026), the day fourteen edits came back. Version 1.5.1.
Why
I went looking for a sentence about a golden number in an enneadecaeteris that I had written into a 𝕐 post at laussy.org, and it was not there. Nor were ten more, the oldest of them from 6 July (2026). Two faults were stacked: llw2lw reconciled a month page by comparing sets of post ids, so a post that exists on both sides and merely says something different matched no test and the push went straight over it; and syncy, on the rare occasions it did run, ranked the two copies by the gadget's <!--lastedited:--> marker, which the ordinary MediaWiki editor never touches, so an edit made online looked exactly as old as the local copy it disagreed with. Both are fixed, and the stories belong on llw2lw and syncy, where they are.
A fix, though, is a promise about the future. This tool asks the other question, the one about the past: what has already been lost, and is anything else missing that I have not noticed? It knows nothing about either bug, and that is the point: it would have found the damage before anybody knew what caused it.
How it finds a loss
It works at all only because llw2lw imports into the online history rather than overwriting it. Every version a post ever had online is therefore still on laussy.org, sitting under the revision that destroyed it: the evidence of the crime survived the crime.
So the detector needs no theory. Walk the online history oldest-first, follow each post's body, and flag every point where the body goes back to a state it already had—A → B → A. Prose does not revert by itself. B is an edit somebody made and something else undid.
Each flagged B is then cross-examined. Against the local history: if that text was never on the local wiki, it was written online and destroyed by a push, which is the sync bug's own signature. And against both pages as they stand today: an edit I simply made again afterwards, or one I have since put back by hand, is reported as recovered rather than as a loss. A post I went on to rewrite locally is not a loss either—the words are gone because I replaced them, and the local copy's marker standing later says so.
The two modes
A month page and an ordinary page are not the same question, so there are two walks.
On the 𝕐 and ℤ month pages the page is not the unit: twenty posts share one revision, and any one of them can change while the other nineteen do not. The walk therefore parses each revision into posts and follows every post's body separately, which is what makes it possible to say which sentence vanished and on which day.
--pages asks the same question of the whole wiki, back to 1 May 2026 unless --since moves the start, and with no parsing at all. An ordinary page is the unit, and MediaWiki hands the whole answer over for free: list=allrevisions gives every online revision's sha1 in one pass. A push imports the bytes unchanged, so text that has ever been on the local wiki hashes the same on both sides; an online sha1 that appears nowhere in that page's local history is text local has never held, and on a wiki that only ever flows local→online that can only have been written online. Current means about to be destroyed, superseded means destroyed already and still readable in the history. File description pages are left out unless --files: their one bookkeeping line is worded differently by the gadget and by syncy, so all of them differ and none of it is prose.
That mode is why this page exists at all rather than a bug report. Until the afternoon it was written, llw2lw guarded only the month pages and imported over every other one unconditionally, on an assumption its own header stated as fact and nothing enforced—which is how the Poetry and Fabrice edits came to be sitting there, one push from gone. llw2lw 3.10.0 closed that by borrowing this very test, and 3.11.0 stopped hanging it on the condition that had let Poetry and Fabrice through, so every ordinary page is now asked before it is pushed. That leaves this tool as the backstop rather than the guard—which is the right place for it. A guard can only refuse what it thought to ask about; this walks the history and asks nothing in advance.
laussy.org has list=recentchanges switched off, which is why the sweep asks allrevisions instead—and gets the sha1s for free.
The two lists
The findings are printed apart, because they are answers to different questions and mixing them makes both useless.
Words that were written online and never on the local wiki can only have been destroyed by a push. There is nothing to weigh: the words existed, they were mine, they are gone, put them back.
Words that were on both sides were taken out by something local—a hand cleanup, a later rewrite, or a repair putting the earlier words back. The casualties of my own de-duplication of 15 August (2026) are in that list. They need an eye, not a rule. --pages splits its findings the same way, current against superseded, and its second list has its own kinds of innocent: a page move, an old import, an action=edit whose pre-save transform moved a byte.
~/.syncloss-seen
A repair looks exactly like the damage. Putting my words back returns a post to a text it already had, which is precisely the pattern this hunts, so the restoration of 24 August (2026) made every repaired post report itself as a fresh crime. --dismiss files everything currently shown into ~/.syncloss-seen as settled, and the next run prints what is new. Without it the second list only grows and stops being read, which is a slower way of not having the tool.
What it found
Twelve findings on the month pages. Nine were mine, written online and destroyed: Y-01on2 (the golden-number sentences that started this), Y-00AVI (contant put right to constant), Y-01rzp (Brian put right to Bryan Ferry, wiped twice), Y-01kti (the __Z__ cross-post cut, whose loss is why the same list stood duplicated on both 𝕐 and ℤ), Y-00hMp, Y-00uMx, Y-20260705-021218, Y-20260803-015434 (firstname, written twice and lost twice) and Z-00W0b. Three more were casualties of my own de-duplication and not sync losses at all.
Then --pages went over the rest of the wiki and found two live ones that no guard covered: Poetry, where I had added Mayakovsky online on 3 August (2026), and Fabrice, where a video animator had become a YouTube link on 21 August (2026). Neither had ever been on the local wiki and the next push would have taken both. They went back with silent-lwedit --at, at the timestamps they were really written (see silent-lwedit).
Usage
syncloss the current month of 𝕐 and ℤ
syncloss --all every month page, 𝕐 from May 2026, ℤ from August
syncloss --page "July (2026)" one page (repeatable)
syncloss --since 2026-08-01 only losses on or after a date; also moves --pages' start
syncloss --pages EVERY page: any online text local has never held
syncloss --pages --files include the File: description pages (noisy)
syncloss --all --dismiss settle everything shown; next run reports what is new
-v prints the surviving text beside the lost text, --all-hits shows every overwrite ever seen including the dismissed and the recovered, --json F writes the findings out for a script to read, and --no-cache refetches every revision instead of trusting ~/.syncloss-cache. Exit status is 1 when something is still missing and 0 when nothing is.
Versions
All seven of these are of 24 August (2026): the tool was written to find the damage, and then several times corrected by being run against the wiki it had just repaired.
- v1.0 (24 August (2026)) — the A → B → A walk over the 𝕐 and ℤ month pages.
- v1.1 — two ways of crying wolf. The
<!--lastedited:-->marker is part of a post's text, so a restored paragraph whose marker still held its old value read as missing; the marker is now stripped before asking is this back?. And a post I kept editing settles its own history—Y-20260803-015434 went first name → firstname → first name → firstname over three weeks, every leg of it an overwrite and only the last one a loss—so a finding with a later finding of its own that is back on the wiki is marked superseded. - v1.2 — the split between the two lists, and
--dismiss. Also the walk had been sendingrvstarttogether withrvcontinueon every page but the first; the two disagree about where the next batch begins, revisions were skipped, and a history with holes in it invents reverts that never happened—25 reported where there were 15. - v1.3 — a post rewritten locally after the online edit stops being reported: syncy's own marker comparison, used here to keep quiet rather than to pick a winner.
- v1.4 —
--pages, which is what turned up Poetry and Fabrice. - v1.5 —
--pageshonours--dismisstoo. - v1.5.1 — a count, not code: eleven losses from 6 July (2026), not thirteen from June. The June post was superseded by a local rewrite the next day, which this tool's own v1.3 rule says is not a loss.
Code
The script lives in ~/bin/syncloss (python, no credentials: it reads both wikis anonymously, with a browser user agent because Cloudflare answers 406 to anything else, and caches the histories it has already fetched in ~/.syncloss-cache).
It is read-only by construction and writes to neither wiki, ever. What it prints is the wikitext to put back and the revision ids to read it in context; putting it back is a separate decision, taken by hand.
~/bin/syncloss
#!/usr/bin/env python3
# ___ _
# / __|_ _ _ _ __ _| |___ ______
# \__ \ || | ' \/ _| | / _ (_-<_-<
# |___/\_, |_||_\__|_|_\___/__/__/
# |__/ SyncLoss V1.5.1 Laussy & Claude
#
# Find the edits the local/online sync has eaten.
#
# A 𝕐 or ℤ month page lives on two wikis at once and is written on both: the
# local one is the base of record, but Fabrice also microblogs from his phone
# and — the case this tool exists for — sometimes just fixes a word in the
# ordinary MediaWiki editor at laussy.org. llw2lw then pushes local over
# online, and until 2026-08-24 nothing in the chain could see that an existing
# post's TEXT had changed online: llw2lw compared SETS OF POST IDS (a post
# edited online is present on both sides, so it is neither "extra" nor
# "dead"), and syncy, when it did run, compared the two copies by their
# <!--lastedited:--> marker — which the ordinary editor does not touch, so the
# two copies read as the same age, syncy called it a CLASH, and local won.
# Eleven of his edits went that way between 6 July and 24 August 2026, among
# them the golden-number-in-an-enneadecaeteris sentences of Y-01on2; three more
# were undone by his own de-duplication of 15 August, which is a different
# accident with the same look.
#
# The detector needs no knowledge of any of that. Because llw2lw imports into
# the online history rather than overwriting it, every version a post ever had
# online is still on laussy.org. So: walk that history oldest-first, follow
# each post's body, and flag every point where the body goes BACK to a state
# it already had — A → B → A. Prose does not revert by itself. B is an edit
# somebody made and something else undid.
#
# Each flagged B is then cross-examined against the LOCAL history:
# * if B was never on the local wiki, it was written online and destroyed;
# * if B was on both, it is a local edit undone by a later push or by a
# hand cleanup (the 15 August de-duplication undid four that way).
# and against the two pages as they stand TODAY: an edit he simply made again
# afterwards is reported as recovered, not as a loss.
#
# Read-only. It never writes to either wiki. What it prints is the wikitext
# to put back, and the revision ids to read the whole thing in context.
"""
syncloss the current month of 𝕐 and ℤ
syncloss --all every month page from 𝕐 May 2026 / ℤ August 2026
syncloss --page "July (2026)" one page (repeatable)
syncloss --since 2026-08-01 only losses whose online edit is on/after a date
syncloss --pages EVERY page, not just the microblog month pages:
any online revision whose text the local wiki has
never held. Add --files to include the File:
description pages (noisy, never prose).
-v, --verbose print the surviving text beside the lost text
--dismiss file everything shown into ~/.syncloss-seen as settled,
so the next run reports only what is new
--all-hits every overwrite ever seen, dismissed and recovered included
--json F write the findings to F as JSON
--no-cache refetch every revision (the cache is ~/.syncloss-cache)
-V, --version
Findings come in two kinds and are reported apart. Words WRITTEN ONLINE and
never on the local wiki can only have been destroyed by a push — that is this
bug. Words that were on BOTH sides were taken out by something local: a hand
cleanup, a rewrite, or a repair putting the earlier text back. Read those,
then --dismiss them.
Exit status is 1 when something is still missing, 0 when nothing is.
"""
import argparse
import difflib
import gzip
import hashlib
import json
import os
import re
import sys
import time
import urllib.parse
import urllib.request
from datetime import datetime
# V1.5.1: the count at the top said thirteen from 13 June. The 13 June post was
# superseded, not lost — this very tool's V1.3 rule says so — so it is
# eleven, from 6 July.
#
# V1.5: --pages honours --dismiss too. Its second list is the noisy one by
# nature — a page move, an import, an old action=edit whose pre-save
# transform moved a byte, all land there beside a real loss — so without
# a way to say "looked at it, it is nothing" the list only ever grows and
# stops being read.
#
# V1.4: --pages, the same question asked of the WHOLE wiki. The post-level
# walk above only knows the 𝕐 and ℤ month pages, and llw2lw guards only
# those too — on the theory, written into its header, that nothing else
# is ever edited online. It is not true: on 3 August 2026 he added
# Mayakovsky to [[Poetry]] at laussy.org and on 21 August turned "a video
# animator" into a YouTube link on [[Fabrice]], neither of which the
# local wiki had ever seen, both one push away from being gone.
#
# An ordinary page needs no post parsing, and MediaWiki hands over the
# whole answer: every revision's sha1. Text that has ever been on the
# local wiki has a local revision with that sha1 — a push imports the
# bytes unchanged, so it hashes the same on both sides. An ONLINE
# revision whose sha1 appears nowhere in the local history is therefore
# text local has never held, which on a wiki that only ever flows
# local→online can only have been written online. Current → about to be
# destroyed; already superseded → destroyed, and still in the history to
# be read back.
#
# File description pages are excluded unless --files: their text is one
# bookkeeping line that syncy and the gadget word differently, so all
# 151 of them differ and none of it is prose.
#
# V1.3: a post he went on to rewrite LOCALLY after the online edit is not a
# loss either — the words are gone because he replaced them, and the
# local copy's <!--lastedited:--> marker says so by standing later than
# the online one. Y-20260613-011637 read "It almost works one. It fails
# forever." online on 13 June and "It almost works once. And then it
# fails forever." locally on the 14th; only the second is his last word
# on it. This is syncy's own comparison, used here to keep quiet rather
# than to choose a winner.
#
# V1.2: a repair looks exactly like the damage. Putting Fabrice's words back
# returns a post to a text it already had, which is the very pattern
# this tool hunts, so the 24 August restoration made every repaired post
# report itself. Two answers, both needed. The report now SPLITS on
# where the overwritten version came from — one written online and never
# on the local wiki is the sync bug's own signature, one that was on both
# sides is a hand cleanup or a later rewrite and wants a human eye — and
# a finding can be DISMISSED (--dismiss) into ~/.syncloss-seen, so a run
# shows what is new rather than the whole history of the wiki every time.
# Also: the walk sent rvstart together with rvcontinue on every page but
# the first, which skipped revisions, and a history with holes in it
# invents reverts — 25 reported where there were 15.
#
# V1.1: two ways of crying wolf, both found by running V1.0 over the repaired
# wiki. (a) The <!--lastedited:--> marker is part of a post's text, so a
# restored paragraph whose marker was left at its old value still read
# as missing; the marker is now stripped before asking "is this back?".
# (b) A post he went on editing settles its own history: Y-20260803-015434
# went "first name" → "firstname" → "first name" → "firstname" over three
# weeks, and every leg of that is an overwrite, but only the last one is
# a loss. A finding with a LATER finding of its own that is back on the
# wiki is now marked superseded and reported only under --all-hits.
VERSION = '1.5.1'
# The gadget's own bookkeeping, not prose: two copies of a post that differ
# only here say the same thing.
MARKER_RE = re.compile(r'<!--lastedited:\d{8}-\d{6}-->')
LOCAL_API = 'http://localhost/laussywiki/api.php'
ONLINE_API = 'https://laussy.org/api.php'
# laussy.org is behind Cloudflare: a bare curl/urllib User-Agent gets a 406.
USER_AGENT = ('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36')
CACHE_DIR = os.path.expanduser('~/.syncloss-cache')
# Findings looked at and settled: their fingerprints, one per line.
SEEN_FILE = os.path.expanduser('~/.syncloss-seen')
# Same first month per branch as syncy's BRANCHES, and the same two spellings
# of a post id: the timestamp written out, and the Anno Fabri tag posts have
# worn since 18 August 2026.
BRANCHES = [('', 2026, 5), ('Z/', 2026, 8)]
MONTHS = ['January', 'February', 'March', 'April', 'May', 'June', 'July',
'August', 'September', 'October', 'November', 'December']
ID_RE = r'[YZ]-(?:\d{8}-\d{6}|[0-9A-Za-z]{5,6})'
POST_OPEN = re.compile(r'<blockquote id="(%s)">' % ID_RE)
BQ_OPEN = re.compile(r'<blockquote\b', re.I)
BQ_CLOSE = re.compile(r'</blockquote\s*>', re.I)
# ---- the wikis ------------------------------------------------------------
def api(base, params, tries=4):
# format/formatversion are defaulted here rather than at every call site:
# forgetting them gets an HTML error page back and a JSON decode error
# thirty frames away from the mistake.
p = {'format': 'json', 'formatversion': '2'}
p.update(params)
req = urllib.request.Request(base + '?' + urllib.parse.urlencode(p),
headers={'User-Agent': USER_AGENT})
for attempt in range(tries):
try:
with urllib.request.urlopen(req, timeout=180) as r:
return json.loads(r.read().decode('utf-8'))
except Exception:
if attempt == tries - 1:
raise
time.sleep(3 * (attempt + 1))
def history(base, title, cache_key, use_cache=True):
"""Every revision of one page, oldest first, with its wikitext.
Cached on disk: a history is append-only, so a second run only has to ask
for what is newer than the last revision already held.
"""
os.makedirs(CACHE_DIR, exist_ok=True)
slug = re.sub(r'[^\w.()-]', '_', f'{cache_key}.{title}')
path = os.path.join(CACHE_DIR, slug + '.jsonl.gz')
revs = []
if use_cache and os.path.exists(path):
with gzip.open(path, 'rt', encoding='utf-8') as fh:
revs = [json.loads(l) for l in fh]
have = {r['revid'] for r in revs}
# Where to resume from is fixed BEFORE the walk. Recomputing it from the
# growing list each time round meant sending rvstart together with
# rvcontinue, and the two disagree about where the next page begins: the
# walk then skipped revisions, and a history with holes in it invents
# reverts that never happened (V1.0 reported 25 overwrites where there
# were 15).
start = revs[-1]['ts'] if revs else None
cont = None
while True:
p = {'action': 'query', 'prop': 'revisions', 'titles': title,
'rvprop': 'content|ids|timestamp|user|comment|size',
'rvslots': 'main', 'rvlimit': '20', 'rvdir': 'newer',
'format': 'json', 'formatversion': '2'}
if start:
p['rvstart'] = start
if cont:
p['rvcontinue'] = cont
d = api(base, p)
pg = d['query']['pages'][0]
if pg.get('missing'):
return []
for r in pg.get('revisions', []):
if r['revid'] in have:
continue
revs.append({'revid': r['revid'], 'ts': r['timestamp'],
'comment': r.get('comment', ''),
'text': r['slots']['main'].get('content', '')})
have.add(r['revid'])
cont = d.get('continue', {}).get('rvcontinue')
if not cont:
break
revs.sort(key=lambda r: (r['ts'], r['revid']))
with gzip.open(path, 'wt', encoding='utf-8') as fh:
for r in revs:
fh.write(json.dumps(r, ensure_ascii=False) + '\n')
return revs
# ---- posts ----------------------------------------------------------------
def find_posts(text):
"""(pid, body) for every post. A post may quote something in a
<blockquote> of its own, so count the nesting rather than take the first
closing tag — the mistake that used to truncate such posts in syncy."""
for m in POST_OPEN.finditer(text):
depth, pos = 1, m.end()
while depth:
close = BQ_CLOSE.search(text, pos)
if not close:
break
opn = BQ_OPEN.search(text, pos)
if opn and opn.start() < close.start():
depth, pos = depth + 1, opn.end()
else:
depth, pos = depth - 1, close.end()
if depth == 0:
yield m.group(1), text[m.end():close.start()]
def norm(s):
return re.sub(r'\s+', ' ', s).strip()
def digest(s):
return hashlib.sha1(norm(s).encode()).hexdigest()[:12]
def sentences(s):
return [x for x in re.split(r'(?<=[.!?])\s+|\n', norm(s)) if x.strip()]
def marker(body):
"""When the gadget last stamped this copy of a post, '' if never."""
m = re.findall(r'<!--lastedited:(\d{8}-\d{6})-->', body)
return max(m) if m else ''
def demark(s):
"""The text without the gadget's last-edit marker, for asking whether a
paragraph is back on the wiki: putting the words back is the repair, and
the marker is a date, which is not ours to move."""
return norm(MARKER_RE.sub('', s))
# ---- the detector ---------------------------------------------------------
def scan(title, use_cache=True):
online = history(ONLINE_API, title, 'online', use_cache)
if not online:
return []
local = history(LOCAL_API, title, 'local', use_cache)
# every body each post ever wore locally, so a flagged version can be told
# apart: written online and destroyed, or a local edit undone later
local_bodies = set()
for r in local:
for pid, body in find_posts(r['text']):
local_bodies.add((pid, digest(body)))
local_ts = {r['ts'] for r in local} # imports keep the local time
demark_local = demark(local[-1]['text']) if local else ''
demark_online = demark(online[-1]['text'])
local_now = {pid: body for pid, body
in find_posts(local[-1]['text'] if local else '')}
states, bodies = {}, {}
for i, r in enumerate(online):
for pid, body in find_posts(r['text']):
h = digest(body)
bodies.setdefault(h, body)
seq = states.setdefault(pid, [])
if not seq or seq[-1][1] != h:
seq.append((i, h))
out = []
for pid, seq in states.items():
seen = {}
for k, (i, h) in enumerate(seq):
if h in seen and k > 0:
j, lost_h = seq[k - 1]
lost, kept = bodies[lost_h], bodies[h]
# the words that were in the lost version and are not in the
# one that replaced it — the thing to look for on the wiki now
sm = difflib.SequenceMatcher(None, sentences(kept), sentences(lost))
gone = []
for tag, _a1, _a2, b1, b2 in sm.get_opcodes():
if tag in ('replace', 'insert'):
gone += sentences(lost)[b1:b2]
gone = [g for g in gone if demark(g)]
if not gone:
seen[h] = i
continue
out.append({
'page': title, 'pid': pid,
'edit_rev': online[j]['revid'], 'edit_ts': online[j]['ts'],
'edit_comment': online[j]['comment'],
'wiped_rev': online[i]['revid'], 'wiped_ts': online[i]['ts'],
'wiped_comment': online[i]['comment'],
'origin': ('local' if (pid, lost_h) in local_bodies else 'online'),
'wiped_by': ('a push of the local page'
if online[i]['ts'] in local_ts
else 'an edit made online'),
'lost': lost, 'kept': kept, 'gone': gone,
'recovered': all(demark(g) in demark_local for g in gone),
'recovered_online': all(demark(g) in demark_online
for g in gone),
# He rewrote the post locally after writing this online:
# the words did not go missing, they were replaced.
'superseded': marker(local_now.get(pid, '')) > marker(lost),
})
seen[h] = i
# A post he kept editing settles its own history: if a LATER version of it
# is the one on the wiki today, the earlier version was not lost, it was
# replaced — by him. Only the last unrecovered leg is a loss.
for pid in {f['pid'] for f in out}:
legs = sorted([f for f in out if f['pid'] == pid],
key=lambda f: f['edit_ts'])
for k, f in enumerate(legs):
if any(later['recovered'] for later in legs[k + 1:]):
f['superseded'] = True
return out
def fingerprint(f):
"""What identifies one overwrite for good: the post, the revision that
held the version that went, and that version's words. A whole-page finding
(--pages) has no post and names its online revision instead."""
if 'rev' in f:
return hashlib.sha1(f"page|{f['title']}|{f['rev']['revid']}"
.encode()).hexdigest()[:16]
return hashlib.sha1(
f"{f['page']}|{f['pid']}|{f['edit_rev']}|{digest(f['lost'])}"
.encode()).hexdigest()[:16]
def load_seen():
if not os.path.exists(SEEN_FILE):
return set()
with open(SEEN_FILE) as fh:
return {l.split('#')[0].strip() for l in fh if l.split('#')[0].strip()}
def dismiss(findings):
seen = load_seen()
with open(SEEN_FILE, 'a') as fh:
for f in findings:
fp = fingerprint(f)
if fp in seen:
continue
if 'rev' in f:
fh.write(f"{fp} # {f['title']} rev {f['rev']['revid']} "
f"{f['rev']['timestamp']}\n")
else:
fh.write(f"{fp} # {f['page']} {f['pid']} rev {f['edit_rev']} "
f"{f['edit_ts']}\n")
seen.add(fp)
def sweep_pages(since, with_files=False):
"""Every page whose ONLINE history holds text the LOCAL wiki never had.
No post parsing and no page list: list=allrevisions walks the online wiki's
revisions in one pass with their sha1s, and a page is only looked up
locally if one of its online revisions is unaccounted for.
"""
pages, cont = {}, None
while True:
p = {'action': 'query', 'list': 'allrevisions', 'arvlimit': '500',
'arvdir': 'newer', 'arvstart': since + 'T00:00:00Z',
'arvnamespace': '*',
'arvprop': 'ids|timestamp|sha1|comment|user|size'}
if cont:
p.update(cont)
d = api(ONLINE_API, p)
for pg in d['query']['allrevisions']:
pages.setdefault(pg['title'], []).extend(pg['revisions'])
cont = d.get('continue')
if not cont:
break
if not with_files:
pages = {t: r for t, r in pages.items() if not t.startswith('File:')}
print(f'\033[36m· {len(pages)} page(s) edited online since {since}\033[0m',
file=sys.stderr)
out = []
for i, (title, revs) in enumerate(sorted(pages.items()), 1):
if i % 50 == 0:
print(f'\033[36m {i}/{len(pages)}\033[0m', file=sys.stderr)
local, lcont = set(), None
while True:
p = {'action': 'query', 'prop': 'revisions', 'titles': title,
'rvprop': 'sha1', 'rvlimit': 'max'}
if lcont:
p.update(lcont)
d = api(LOCAL_API, p)
pg = d['query']['pages'][0]
if pg.get('missing'):
local = None
break
local |= {r.get('sha1') for r in pg.get('revisions', [])}
lcont = d.get('continue')
if not lcont:
break
if local is None:
continue # online-only page: not a lost edit
revs.sort(key=lambda r: (r['timestamp'], r['revid']))
orphans = [r for r in revs if r['sha1'] not in local]
if not orphans:
continue
last, top = orphans[-1], revs[-1]
out.append({'title': title, 'rev': last, 'count': len(orphans),
'live': top['sha1'] not in local, 'top': top})
return out
def report_pages(found):
live = [f for f in found if f['live']]
past = [f for f in found if not f['live']]
if live:
print('\n\033[1;31mONLINE TEXT THE LOCAL WIKI HAS NEVER HELD, AND IT IS '
'STILL THE ONLINE TEXT\033[0m — one push away from being gone. '
'Bring it into local (silent-lwedit --at <the date he wrote it> '
'keeps the date honest), then push.')
for f in live:
r = f['rev']
print(f"\n \033[1m{f['title']}\033[0m")
print(f" laussy.org rev {r['revid']} {r['timestamp']} "
f"{r.get('size')} bytes '{r.get('comment', '')[:52]}'")
if past:
print('\n\033[1;33mONLINE TEXT THE LOCAL WIKI NEVER HELD, SINCE '
'OVERWRITTEN\033[0m — read the revision to see what it said; a '
'page move or an old import lands here too.')
for f in past:
r = f['rev']
print(f" {f['title']:38} rev {r['revid']} {r['timestamp']} "
f"'{r.get('comment', '')[:34]}' → now rev {f['top']['revid']}")
if not found:
print('\n\033[32mNo page holds online text the local wiki has never '
'had.\033[0m')
# ---- report ---------------------------------------------------------------
def report(findings, verbose=False, first=1):
for n, f in enumerate(findings, first):
origin = ('written ONLINE and never on the local wiki'
if f['origin'] == 'online' else
'a local edit, undone later')
tag = ''
if f['recovered']:
tag = ' \033[32m(since put back)\033[0m'
elif f['superseded']:
tag = ' \033[33m(superseded by a later edit of his)\033[0m'
print(f"\n\033[1m[{n}] {f['page']} / {f['pid']}\033[0m{tag}")
print(f" edit {f['edit_ts']} rev {f['edit_rev']} "
f"'{f['edit_comment'][:52]}' ({origin})")
print(f" wiped {f['wiped_ts']} rev {f['wiped_rev']} "
f"'{f['wiped_comment'][:52]}' (by {f['wiped_by']})")
if verbose:
for line in difflib.unified_diff(
sentences(f['kept']), sentences(f['lost']),
'on the wiki now', 'what he wrote', lineterm='', n=0):
if line.startswith(('---', '+++', '@@')):
continue
print(' ', line[:400])
else:
for g in f['gone']:
print(f" + {g[:400]}")
def main():
ap = argparse.ArgumentParser(add_help=False)
ap.add_argument('--page', action='append', default=[])
ap.add_argument('--all', action='store_true')
ap.add_argument('--since')
ap.add_argument('--all-hits', action='store_true')
ap.add_argument('--dismiss', action='store_true')
ap.add_argument('--pages', action='store_true')
ap.add_argument('--files', action='store_true')
ap.add_argument('-v', '--verbose', action='store_true')
ap.add_argument('--json')
ap.add_argument('--no-cache', action='store_true')
ap.add_argument('-V', '--version', action='store_true')
ap.add_argument('-h', '--help', action='store_true')
a = ap.parse_args()
if a.help:
print(__doc__.strip())
return 0
if a.version:
print(f'syncloss {VERSION}')
return 0
if a.pages:
found = sweep_pages(a.since or '2026-05-01', a.files)
if not a.all_hits:
seen = load_seen()
found = [f for f in found if fingerprint(f) not in seen]
report_pages(found)
if a.json:
with open(a.json, 'w') as fh:
json.dump(found, fh, ensure_ascii=False, indent=1)
if a.dismiss:
dismiss(found)
print(f'\n{len(found)} finding(s) written to {SEEN_FILE}; they '
f'will not be reported again.')
return 0
return 1 if any(f['live'] for f in found) else 0
titles = list(a.page)
if not titles:
now = datetime.now()
if a.all:
for prefix, y0, m0 in BRANCHES:
y, m = y0, m0
while (y, m) <= (now.year, now.month):
titles.append(f'{prefix}{MONTHS[m - 1]} ({y})')
y, m = (y + 1, 1) if m == 12 else (y, m + 1)
else:
titles = [f'{p}{MONTHS[now.month - 1]} ({now.year})'
for p, _y, _m in BRANCHES]
findings = []
for t in titles:
print(f'\033[36m· {t}\033[0m', file=sys.stderr)
try:
findings += scan(t, use_cache=not a.no_cache)
except Exception as e: # a page may not be online
print(f' could not scan {t}: {e}', file=sys.stderr)
if a.since:
findings = [f for f in findings if f['edit_ts'][:10] >= a.since]
findings.sort(key=lambda f: f['edit_ts'])
seen = set() if a.all_hits else load_seen()
missing = [f for f in findings
if not f['recovered'] and not f['superseded']
and fingerprint(f) not in seen]
if a.all_hits:
report(findings, a.verbose)
else:
online_born = [f for f in missing if f['origin'] == 'online']
both_sides = [f for f in missing if f['origin'] != 'online']
if online_born:
print('\n\033[1;31mWRITTEN ONLINE AND DESTROYED\033[0m — the sync '
'bug\'s own signature: these words were never on the local '
'wiki, so a push could only overwrite them.')
report(online_born, a.verbose)
if both_sides:
print('\n\033[1;33mUNDONE, BUT THE TEXT WAS ON BOTH SIDES\033[0m — '
'a hand cleanup or a later rewrite took these out. Some are '
'losses, some are you changing your mind, and some are a '
'repair putting the earlier words back: read before acting, '
'then --dismiss.')
report(both_sides, a.verbose, first=len(online_born) + 1)
if a.json:
with open(a.json, 'w') as fh:
json.dump(findings, fh, ensure_ascii=False, indent=1)
if a.dismiss:
dismiss(missing)
print(f"\n{len(missing)} finding(s) written to {SEEN_FILE}; "
f"they will not be reported again.")
return 0
print()
if missing:
print(f'\033[31m{len(missing)} edit(s) still missing from the local '
f'wiki\033[0m, out of {len(findings)} overwritten in all.')
print('Put the text back on the LOCAL wiki only, then run llw2lw. '
'On a page whose last-modified date must not move, write it '
'with silent-lwedit.')
else:
print(f'\033[32mNothing missing\033[0m ({len(findings)} '
f'overwrite(s) in the history, all since put back, superseded '
f'or dismissed).')
return 1 if missing else 0
if __name__ == '__main__':
sys.exit(main())