mdtable2wiki
mdtable2wiki is a python script that turns a Markdown table—typically one Claude has just handed me—into a clean, full-width MediaWiki table, ready to paste straight into my wiki.
Two things can silently destroy such a table if one converts it by hand: the pipes, and the maths. Both are dealt with by this script.
Written with the help of Claude Opus 5 on 26 July (2026). Version 1.4.0.
The idea
A Markdown table and a MediaWiki table carry exactly the same information, so the conversion ought to be a five-line sed. It is not, for one reason: MediaWiki cuts cells on |, and it does so before anything else happens. Any pipe living inside the content—in a piece of code, in a URL, and above all in a ket—is read as a cell separator, and the row quietly loses half of itself. Nothing warns you; the table simply comes out wrong, and it is not obvious afterwards where the damage was done.
So the script's real work is to know, for every character of the input, whether it is structure or content, and to neutralise every content pipe according to where it sits: | becomes | in text, and \vert inside maths. Everything else—alignment, emphasis, links, captions—is bookkeeping on top of that.
Usage
mdtable2wiki file.md # convert every table found in a file
... | mdtable2wiki # convert from stdin
mdtable2wiki # paste the table, then Ctrl-D
mdtable2wiki -x -c # clipboard in, clipboard out
mdtable2wiki -x -c --ai # console copy in, {{AI}}-ready table out
mdtable2wiki -x -c -n # the same with a toast: this is Ctrl+Alt+W
The last one is the way I actually use it: select the table wherever it is, copy, run mdtable2wiki -x -c, paste into the edit box. Text around the tables is passed through untouched, so a whole answer can be piped through the script and only its tables will change.
The pipe problem
Consider this innocent row, which is the sort of thing I ask for a dozen times a week:
| even | symmetric | $\ket{\psi^+},\ket{\phi^\pm}$ | yes |
Written out by hand as | $|\psi^+\rangle$ the cell is already lost: MathJax runs in the browser, long after MediaWiki has finished parsing, so MediaWiki sees a bare pipe in the middle of a table row and splits there. The script protects the four contexts separately.
| Context | A literal | becomes |
Why |
|---|---|---|
| ordinary text | | |
the entity is invisible to the table parser |
inside `code` |
| |
same, inside <code> tags
|
inside $…$ |
\vert |
an entity would not survive a <math> tag; \vert is correct $\mathrm{\TeX}$ in either case
|
\| inside $…$ |
\Vert |
the double bar of a norm |
Because every cell is written on its own line rather than with || separators, the result stays legible however brutal the content is. A cell containing x||y survives, which is the case that convinced me the entity route was the right one.
Maths
$…$ is left exactly as it is, since SimpleMathJax here declares $ an inline delimiter ($wgSmjExtraInlineMath); --math tag wraps it in <math> instead, for a wiki that wants that.
The interesting question was which macros actually render. MathJax 3 ships an autoload map, and braket is in it:
braket:["bra","ket","braket","set","Bra","Ket","Braket","Set","ketbra","Ketbra"]
so \ket and its relatives load themselves on first use and need no help—and, being brace-delimited, they carry no pipe to endanger the row. They are therefore left verbatim, which keeps the wikitext readable. The physics package is not in that map, so its macros are expanded by the script instead:
| Written | Becomes | |
|---|---|---|
\expval{H} |
\langle H\rangle |
expectation value |
\abs{z} |
\left\vert z\right\vert |
|
\norm{A} |
\left\Vert A\right\Vert |
|
\dyad{0} |
\vert 0\rangle\!\langle 0\vert |
|
\braket{\phi|\psi} |
\langle \phi\vert \psi\rangle |
expanded only because of that pipe |
\braket{\psi}, having no pipe, is left alone. The flag -e expands the kets as well, for pasting into a wiki whose MathJax lacks the braket package.
Alignment
The :---: row is honoured wherever it says something. Where it is silent—and Claude's tables are usually all ---—the script guesses, per column: all-numeric goes right, everything whose cells read ten characters or fewer goes centre, the rest stays left. The width is measured on what the cell will look like, not on its source, so $\ket{\psi^+},\ket{\phi^\pm}$ counts as nine characters and not as twenty-nine.
The majority alignment is then written once on the table tag and only the minority columns carry a per-cell style, which keeps the source from filling up with repeated style= attributes.
Copying out of a browser
The obvious annoyance is that one reads Claude's answer rendered, where the Markdown table is a real HTML table and the maths is a pile of KaTeX spans. Exporting the Markdown each time to get at the source would be tedious.
It turns out to be unnecessary. Selecting a rendered table and copying it puts the actual markup on the X11 clipboard under the text/html target, next to the plain text:
xclip -selection clipboard -t TARGETS -o
-x prefers that target, and any input containing <table is detected and routed through an HTML extractor. The maths comes back verbatim, because KaTeX keeps the $\mathrm{\TeX}$ it was given in its MathML annotation:
<annotation encoding="application/x-tex">\ket{\psi^+},\ket{\phi^\pm}</annotation>
The extractor reads that annotation and suppresses the rendered glyph spans, which would otherwise come through as duplicated rubbish. <strong>, <em>, <code> and links are recovered too, and text-align on the cells becomes the :---: row, so the alignment I was shown is the alignment I get rather than a fresh guess. No userscript, no extension, no export.
Copying out of the terminal
A table copied from Claude Code's console is a different beast: the terminal has already rendered the Markdown away, and what lands on the clipboard is a grid of box-drawing glyphs—┌─┬─┐ borders, │ separators, long cells wrapped over several physical lines—with the bold and the code spans silently stripped. Version 1.3.0 recognises the grid and rebuilds the table: content lines are split on │, the physical lines between two borders are rejoined into one logical row, and a mangled copy—a short bottom border, a lost trailing │—is taken in stride. The first row is the header, as in Markdown; -H if it is not.
Such a table is usually destined for the a= of Template:AI, where a literal pipe would end the parameter. --ai writes every structural pipe of the result as {{!}}, which expands back to | before Module:AI reads the text, so the table survives being a template argument—the recipe of Template:AI's documentation, automated. The console round trip is thus: copy the answer in the terminal, mdtable2wiki -x -c --ai, paste into a=. The markup the terminal stripped from the prose is not recoverable from the clipboard at all; for a faithful quote, the raw Markdown lives in the session transcripts under ~/.claude/projects/.
Options
| Flag | Effect |
|---|---|
-x, --from-clipboard |
read the input from the clipboard (HTML flavour preferred) |
-c, --copy |
copy the result back to the clipboard |
-t, --caption |
table caption, as |+
|
-w, --width |
table width, default 100%; auto omits it
|
-a, --align |
auto (default), left, center, right
|
--threshold |
the ten characters above which auto stops centring
|
--header-align |
headers centred (default) or following their column |
-r, --row-headers |
first column as ! scope="row"
|
-s, --sortable |
add the sortable class
|
--class, --style |
override the class, append CSS to the table tag |
--math |
dollar (default) or tag for <math>
|
-e, --expand-kets |
expand \ket and friends as well
|
-H, --no-header |
treat the first row as data |
--ai |
structural pipes written as {{!}}, for a table inside the a= of Template:AI
|
-q, --quiet |
no report on stderr |
-n, --notify |
report with a desktop toast as well, there being no terminal when the script is run from a key |
On width: a table narrower than its column is centred rather than left-hugging, by margin: 1em auto in MediaWiki:Common.css. The tables this script writes are width:100% and so are unaffected; only those written by hand, or with -w auto, actually move.
On a key
Since 1.4.0 the whole round trip is one keystroke, on the keymapper side rather than KDE's: Ctrl+Alt+W converts whatever table is on the clipboard and puts the wikitable back on it, Ctrl+Alt+ShiftRight+W does the same in the {{!}}-escaped form for the a= of Template:AI. W for wikitable; T, which would have been the obvious letter, has been <tt></tt> for years, and a plain rule in keymapper.conf also fires with extra modifiers, so Ctrl+Alt+Shift+T is not free either—it is <tt></tt> as well. In the file the two rules read {Z}, since that is the scan code of the key engraved W on an AZERTY keyboard; the Keystrokes page lists everything by the engraved letters.
Nothing is pasted for me: the clipboard is filled and there it stops, deliberately—xdotool typing into a French layout is a good way to corrupt the keyboard. The toast (--notify, exactly as html2md does it) is what tells me the conversion happened, and how many tables it found.
Limitations
- No
colspanorrowspan. Neither Markdown source nor the HTML extractor handles merged cells; a table with them will come out as a grid. - HTML input keeps only the tables. Prose around them is dropped, unlike the Markdown path which passes it through.
\Set{x | x>0}loses its scaled bar. The pipe has to go, and\vertis a fixed-height substitute.- The
text/htmlclipboard target is an X11 notion; on Waylandwl-paste -t text/htmlis tried first and should do the same, but I have not tested it. - Nothing is done about Markdown outside tables—headings, lists, block quotes. That is LaTeX2wiki's business, and a whole other job.
Version history
- 1.0.0 (26 July (2026)) — first version. Pipe protection in all four contexts, physics macros, automatic alignment, captions, row headers, clipboard in and out.
- 1.2.0 — HTML input: a table selected in a browser is taken off the
text/htmlclipboard target, with the $\mathrm{\TeX}$ recovered from KaTeX's MathML annotation. Two subtleties: a pipe recovered inside maths must not be escaped as text (it became\Vert), and an empty<strong></strong>pair cannot be removed with a regular expression, since the**and*alternatives overlap and it ate the emphasis it was meant to keep. It is done by index at parse time. - 1.3.0 (9 August (2026)) — box-drawing input and
--ai: a table copied from the Claude Code terminal—glyph borders, cells wrapped over several physical lines, no Markdown left in it—is recognised and rebuilt, and--aiwrites the structural pipes as{{!}}so the table can sit inside thea=of Template:AI. One trap for the record: the box-content test readline.strip()[:1] in BOX_V, andinon Python strings is substring containment, so the empty slice of a blank line was in everything—each empty line of the input became table content until the emptiness was tested first. - 1.4.0 (16 August (2026)) —
--notify, a desktop toast for the hotkey runs, and the two keymapper bindings on Ctrl+Alt+W. With nothing converted the clipboard is now left alone instead of being written back over with the input.
The file
~/bin/mdtable2wiki
#!/usr/bin/env python3
# _ _ _ _ ___ _ _ _
# _ __ __| | |_ __ _| |__| |___|_ )_ __ _(_) |_(_)
# | ' \/ _` | _/ _` | '_ \ / -_)/ /\ V V / | / / |
# |_|_|_\__,_|\__\__,_|_.__/_\___/___|\_/\_/|_|_\_\_|
#
# mdtable2wiki - Version 1.4.0
# Claude & F.P. Laussy
# https://laussy.org/wiki/mdtable2wiki
#
# Turn a Markdown table (typically one Claude just wrote) into a clean,
# full-width MediaWiki table ready to copy/paste into the wiki.
#
# Usage:
# mdtable2wiki file.md convert tables found in a file
# ... | mdtable2wiki convert from stdin
# mdtable2wiki paste the table, then Ctrl-D
# mdtable2wiki -x -c clipboard in, clipboard out
# mdtable2wiki -x -c -n the same, with a toast: Ctrl+Alt+W
#
# What it does
# ------------
# * takes Markdown OR HTML OR a box-drawing table: select a rendered
# table in a browser, copy, and -x picks it up off the clipboard's
# text/html target - there is no need to get hold of the Markdown
# source at all. Maths comes back verbatim because KaTeX stores the
# LaTeX in its MathML annotation. A table copied from the Claude Code
# TERMINAL arrives as box-drawing glyphs (the console renders the
# markdown away); those are recognised too and converted like any
# other table, wrapped cell lines rejoined;
# * --ai emits every structural pipe of the wikitable as {{!}}, so the
# result can sit inside a template parameter - i.e. the a= of
# Template:AI. mdtable2wiki -x -c --ai is the console round trip;
# * finds every Markdown table in the input (leading/trailing pipes
# optional, ragged rows padded, text around tables passed through);
# * emits {| class="wikitable" style="width:100%;..." with one cell per
# line, |+ caption, ! header row - the robust MediaWiki layout;
# * honours the :---: alignment row; where the Markdown says nothing,
# --align auto centres short columns, right-aligns numeric ones and
# leaves prose left;
# * PIPES: any literal | in text becomes |, and any | inside math
# becomes \vert - otherwise MediaWiki eats the cell at that point.
# This is the single most common way md->wiki tables break;
# * MATH: $...$ / $$...$$ are left as-is (this wiki's SimpleMathJax has
# $ as an inline delimiter); --math tag wraps them in <math> instead.
# \ket \bra \ketbra \Ket \Set ... are left alone: MathJax autoloads
# its braket package for those. Only the physics-package macros it
# does NOT know are expanded: \expval \abs \norm \dyad. \braket{a|b}
# is expanded too, but only because of the pipe (see above); with
# -e / --expand-kets everything is expanded, for wikis without the
# braket package;
# * inline markup: **b** -> ''' , *i* -> '' , `c` -> <code>, ~~s~~ ->
# <s>, [t](url) -> [url t], <br> -> <br/>.
#
# Everything inside code spans and math is protected from the markup
# rewriting, so LaTeX goes through untouched apart from the pipes.
import argparse
import re
import shutil
import subprocess
import sys
from html.parser import HTMLParser
VERSION = '1.4.0'
# ------------------------------------------------------------------ html
# A table selected with the mouse in a browser and copied lands on the
# clipboard as real <table> markup under the text/html target - no need to
# hunt down the Markdown source. KaTeX (which is what renders maths in a
# chat window) keeps the original LaTeX in its MathML annotation, so the
# maths survives the round trip exactly as it was written.
VOID = {'area', 'base', 'br', 'col', 'embed', 'hr', 'img', 'input', 'link',
'meta', 'param', 'source', 'track', 'wbr'}
INLINE_OPEN = {'b': '**', 'strong': '**', 'i': '*', 'em': '*',
'code': '`', 'tt': '`', 's': '~~', 'del': '~~'}
class TableExtractor(HTMLParser):
"""Pull <table>s out of a fragment of HTML as plain cell text."""
def __init__(self):
HTMLParser.__init__(self, convert_charrefs=True)
self.depth = 0
self.tables = []
self.stack = [] # open tables, innermost last
self.katex = None # depth at which a KaTeX span opened
self.katex_display = False
self.math = []
self.in_annotation = False
self.hrefs = []
self.inline = [] # (marker, index of its opening in parts)
# -- helpers
def _cell(self):
if not self.stack:
return None
row = self.stack[-1]['row']
return row[-1] if row else None
def _emit(self, s, kind='text'):
"""kind 'math' is exempt from pipe-escaping: its | is LaTeX, and
fix_math() will turn it into \\vert later."""
c = self._cell()
if c is not None:
c['parts'].append((kind, s))
# -- parsing
def handle_starttag(self, tag, attrs):
a = dict(attrs)
if tag not in VOID:
self.depth += 1
cls = a.get('class', '') or ''
if self.katex is None and 'katex' in cls:
self.katex = self.depth
self.katex_display = 'katex-display' in cls
self.math = []
return
if tag == 'annotation' and a.get('encoding') == 'application/x-tex':
self.in_annotation = True
return
if self.katex is not None:
return
if tag == 'table':
self.stack.append({'rows': [], 'row': []})
elif tag == 'tr' and self.stack:
self.stack[-1]['row'] = []
self.stack[-1]['rows'].append(self.stack[-1]['row'])
elif tag in ('td', 'th') and self.stack:
style = a.get('style', '') or ''
m = re.search(r'text-align\s*:\s*(left|right|center)', style, re.I)
align = m.group(1).lower() if m else None
if not align and a.get('align'):
align = a['align'].lower()
row = self.stack[-1]['row']
if row is None:
row = self.stack[-1]['row'] = []
self.stack[-1]['rows'].append(row)
row.append({'parts': [], 'header': tag == 'th', 'align': align})
elif tag == 'br':
self._emit('<br>')
elif tag == 'a':
self.hrefs.append(a.get('href', ''))
self._emit('[')
elif tag in INLINE_OPEN:
c = self._cell()
self.inline.append((INLINE_OPEN[tag],
len(c['parts']) if c else 0))
self._emit(INLINE_OPEN[tag])
def _close_inline(self):
"""Close an emphasis, dropping the pair outright if it wrapped
nothing (a regex cannot do this: ** and * overlap)."""
if not self.inline:
return
marker, at = self.inline.pop()
c = self._cell()
if c is None:
return
inside = ''.join(s for _k, s in c['parts'][at + 1:])
if not inside.strip():
del c['parts'][at:]
if inside:
self._emit(inside)
else:
self._emit(marker)
def handle_endtag(self, tag):
if self.katex is not None and self.depth == self.katex:
body = re.sub(r'\s+', ' ', ''.join(self.math)).strip()
if body:
d = '$$' if self.katex_display else '$'
self._emit(d + body + d, 'math')
self.katex = None
elif tag == 'annotation':
self.in_annotation = False
elif self.katex is None:
if tag == 'table' and self.stack:
self.tables.append(self.stack.pop())
elif tag == 'a' and self.hrefs:
self._emit('](%s)' % self.hrefs.pop())
elif tag in INLINE_OPEN:
self._close_inline()
if tag not in VOID:
self.depth = max(0, self.depth - 1)
def handle_data(self, data):
if self.in_annotation:
self.math.append(data)
elif self.katex is None:
self._emit(data)
def _cell_md(cell):
out = []
for kind, s in cell['parts']:
s = re.sub(r'\s+', ' ', s.replace('\xa0', ' '))
if kind != 'math':
s = s.replace('|', r'\|')
out.append(s)
return ''.join(out).strip()
def html_tables_to_md(html):
"""Every <table> in the HTML, rewritten as a Markdown table."""
p = TableExtractor()
p.feed(html)
p.close()
out = []
for t in p.tables:
rows = [r for r in t['rows'] if r]
if not rows:
continue
ncol = max(len(r) for r in rows)
aligns = []
for i in range(ncol):
a = None
for r in rows:
if i < len(r) and r[i]['align']:
a = r[i]['align']
break
aligns.append({'center': ':---:', 'right': '---:',
'left': ':---'}.get(a, '---'))
def line(cells):
v = [_cell_md(c) for c in cells] + [''] * (ncol - len(cells))
return '| ' + ' | '.join(v) + ' |'
out.append(line(rows[0]))
out.append('|' + '|'.join(aligns) + '|')
out.extend(line(r) for r in rows[1:])
out.append('')
return '\n'.join(out)
def looks_like_html(text):
return re.search(r'<table[\s>]', text, re.I) is not None
# ------------------------------------------------------------- box tables
# A table copied straight from a terminal (Claude Code renders Markdown
# tables as box-drawing glyphs) has no pipes and no Markdown left in it.
# Recognise the glyph grid and rebuild the Markdown table: content lines
# split on │, physical lines between two border lines rejoined into one
# logical row (the console wraps long cells). The copy is often mangled
# (short bottom border, lost trailing │) - the parser only needs each
# content line to START with │, the rest is best effort.
BOX_BORDER = set('─┌┬┐├┼┤└┴┘━┏┳┓┣╋┫┗┻┛═╔╦╗╠╬╣╚╩╝╭╮╰╯')
BOX_V = '│┃║'
def _is_box_border(line):
st = line.strip()
return len(st) >= 2 and all(c in BOX_BORDER for c in st)
def _is_box_content(line):
st = line.strip()
return bool(st) and st[0] in BOX_V
def _box_block_to_md(block):
"""One run of border/content lines -> Markdown table lines, or None."""
phys_rows, cur = [], []
for ln in block:
if _is_box_border(ln):
if cur:
phys_rows.append(cur)
cur = []
else:
cur.append(ln)
if cur:
phys_rows.append(cur)
if not phys_rows:
return None
logical = []
for phys in phys_rows:
split = []
for ln in phys:
st = ln.strip()
parts = re.split('[%s]' % BOX_V, st)
parts = parts[1:] # before the leading │
if st[-1:] in BOX_V and parts:
parts = parts[:-1] # after the trailing │
split.append(parts)
ncol = max(len(p) for p in split)
cells = []
for c in range(ncol):
frags = [p[c].strip() for p in split
if c < len(p) and p[c].strip()]
cells.append(' '.join(frags))
logical.append(cells)
ncol = max(len(r) for r in logical)
def line(cells):
v = [c.replace('|', r'\|') for c in cells] + [''] * (ncol - len(cells))
return '| ' + ' | '.join(v) + ' |'
md = [line(logical[0]), '|' + '---|' * ncol]
md.extend(line(r) for r in logical[1:])
return md
def box_tables_to_md(text):
"""Rewrite every box-drawing table in the text as Markdown; everything
else passes through untouched. Returns (text, number converted)."""
lines = text.split('\n')
out, i, count = [], 0, 0
while i < len(lines):
if not (_is_box_border(lines[i]) or _is_box_content(lines[i])):
out.append(lines[i])
i += 1
continue
start = i
while i < len(lines) and (_is_box_border(lines[i])
or _is_box_content(lines[i])):
i += 1
md = _box_block_to_md(lines[start:i])
if md is None:
out.extend(lines[start:i])
else:
out.extend(md)
count += 1
return '\n'.join(out), count
# ------------------------------------------------------------------ math
def _braced(s, i):
"""s[i] must be '{'. Return (inner, index_after_closing_brace)."""
depth = 0
j = i
while j < len(s):
c = s[j]
if c == '\\':
j += 2
continue
if c == '{':
depth += 1
elif c == '}':
depth -= 1
if depth == 0:
return s[i + 1:j], j + 1
j += 1
return None, i
def _split_top_pipe(s):
"""Split on a | that is not nested in braces (for \\braket{a|b})."""
depth = 0
i = 0
while i < len(s):
c = s[i]
if c == '\\':
i += 2
continue
if c == '{':
depth += 1
elif c == '}':
depth -= 1
elif c == '|' and depth == 0:
return s[:i], s[i + 1:]
i += 1
return s, None
def _apply_macro(s, name, nargs, fn, kets=False):
"""Replace \\name{..}x nargs by fn(args...), arguments expanded first."""
pat = '\\' + name
out = []
i = 0
while True:
k = s.find(pat, i)
if k < 0:
out.append(s[i:])
break
e = k + len(pat)
if e < len(s) and s[e].isalpha(): # \ket vs \ketbra
out.append(s[i:e])
i = e
continue
args = []
j = e
ok = True
for _ in range(nargs):
while j < len(s) and s[j] == ' ':
j += 1
if j < len(s) and s[j] == '{':
arg, j = _braced(s, j)
if arg is None:
ok = False
break
args.append(arg)
else:
ok = False
break
if not ok: # not a real call, leave it
out.append(s[i:e])
i = e
continue
out.append(s[i:k])
out.append(fn(*[expand_physics(a, kets) for a in args]))
i = j
return ''.join(out)
def _braket(a):
left, right = _split_top_pipe(a)
if right is None:
return r'\langle %s\rangle' % left
return r'\langle %s\vert %s\rangle' % (left, right)
# MathJax autoloads its braket package for \bra \ket \braket \ketbra \Set
# ... so those are left alone by default: they render, and (unlike a bare
# |...>) they carry no pipe to trip the table parser. These are the ones
# it has no idea about - the physics package is NOT in its autoload map.
PHYSICS = [
('dyad', 1, lambda a: r'\vert %s\rangle\!\langle %s\vert' % (a, a)),
('expval', 1, lambda a: r'\langle %s\rangle' % a),
('norm', 1, lambda a: r'\left\Vert %s\right\Vert' % a),
('abs', 1, lambda a: r'\left\vert %s\right\vert' % a),
]
# Only reached with -e, or for \braket/\Braket that contain a literal |.
BRAKET = [
('ketbra', 2, lambda a, b: r'\vert %s\rangle\!\langle %s\vert' % (a, b)),
('Braket', 1, _braket),
('braket', 1, _braket),
('bra', 1, lambda a: r'\langle %s\vert' % a),
('Ket', 1, lambda a: r'\left\vert %s\right\rangle' % a),
('ket', 1, lambda a: r'\vert %s\rangle' % a),
]
def expand_physics(s, kets=False):
"""Expand the macros MathJax does not ship with."""
for name, nargs, fn in PHYSICS + (BRAKET if kets else []):
s = _apply_macro(s, name, nargs, fn, kets)
if not kets: # only those whose | would cut the row
for name in ('Braket', 'braket'):
s = _apply_macro(s, name, 1, lambda a, _n=name: (
_braket(a) if _split_top_pipe(a)[1] is not None
else '\\%s{%s}' % (_n, a)), kets)
return s
def fix_math(s, kets=False):
"""Make a math body safe inside a MediaWiki table cell."""
s = expand_physics(s, kets)
s = re.sub(r'\\\|', r'\\Vert ', s) # \| -> \Vert
s = s.replace('|', r'\vert ') # bare | -> \vert
s = re.sub(r'\\(vert|Vert) +([,.;)\]}])', r'\\\1\2', s)
s = re.sub(r'\\(vert|Vert) +', r'\\\1 ', s)
return re.sub(r' +$', '', s)
# -------------------------------------------------------------- segments
# A row is cut into segments so that pipes inside code spans and math are
# never mistaken for cell separators.
def segment(s):
"""-> list of (kind, raw); kind in text|esc|code|math|dmath."""
segs = []
buf = []
def flush():
if buf:
segs.append(('text', ''.join(buf)))
del buf[:]
i, n = 0, len(s)
while i < n:
c = s[i]
if c == '\\' and s.startswith(('\\(', '\\['), i):
close = '\\)' if s[i + 1] == '(' else '\\]'
j = s.find(close, i + 2)
if j > 0:
flush()
segs.append(('math' if close == '\\)' else 'dmath',
s[i + 2:j]))
i = j + 2
continue
if c == '\\' and i + 1 < n:
flush()
segs.append(('esc', s[i:i + 2]))
i += 2
continue
if c == '`':
m = re.match(r'`+', s[i:])
ticks = m.group(0)
j = s.find(ticks, i + len(ticks))
if j > 0:
flush()
segs.append(('code', s[i + len(ticks):j]))
i = j + len(ticks)
continue
if c == '$':
if s.startswith('$$', i):
j = s.find('$$', i + 2)
if j > 0:
flush()
segs.append(('dmath', s[i + 2:j]))
i = j + 2
continue
else:
j = i + 1
while j < n:
if s[j] == '\\':
j += 2
continue
if s[j] == '$':
break
j += 1
if j < n and s[j] == '$':
flush()
segs.append(('math', s[i + 1:j]))
i = j + 1
continue
buf.append(c)
i += 1
flush()
return segs
def split_cells(line):
"""Cut a Markdown table row into a list of segment-lists."""
st = line.strip()
lead = st.startswith('|')
trail = st.endswith('|') and not st.endswith('\\|')
cells = [[]]
for kind, raw in segment(st):
if kind != 'text':
cells[-1].append((kind, raw))
continue
parts = raw.split('|')
cells[-1].append(('text', parts[0]))
for p in parts[1:]:
cells.append([('text', p)])
if lead and cells:
cells.pop(0)
if trail and cells:
cells.pop()
return cells
# ------------------------------------------------------------ conversion
INLINE = [
(re.compile(r'~~(.+?)~~'), r'<s>\1</s>'),
(re.compile(r'\*\*(.+?)\*\*'), r"'''\1'''"),
(re.compile(r'__(.+?)__'), r"'''\1'''"),
(re.compile(r'(?<!\*)\*(?!\*)(.+?)(?<!\*)\*(?!\*)'), r"''\1''"),
(re.compile(r'(?<![\w\\])_(?!_)([^_]+?)_(?![\w_])'), r"''\1''"),
(re.compile(r'\[([^\]]*)\]\((\S+?)\)'), r'[\2 \1]'),
(re.compile(r'<br\s*/?>', re.I), r'<br/>'),
]
def render_cell(cell, math_mode, header=False, expand_kets=False):
out = []
for kind, raw in cell:
if kind == 'text':
t = raw.replace('|', '|')
for rx, rep in INLINE:
t = rx.sub(rep, t)
out.append(t)
elif kind == 'esc':
ch = raw[1]
out.append('|' if ch == '|' else
ch if ch in '*_`$[]()#+-.!~\\{}' else raw)
elif kind == 'code':
t = raw.replace('&', '&').replace('<', '<')
out.append('<code>%s</code>' % t.replace('|', '|'))
else:
body = fix_math(raw, kets=expand_kets)
if math_mode == 'tag':
out.append('<math%s>%s</math>' %
(' display="block"' if kind == 'dmath' else '',
body))
else:
d = '$$' if kind == 'dmath' else '$'
out.append('%s%s%s' % (d, body, d))
txt = ''.join(out).strip()
if header:
txt = txt.replace('!!', '!!')
return txt
# ------------------------------------------------------------- alignment
NUMERIC = re.compile(r'^[-+−±]?[\d ]*[\d]'
r'[\d .,]*\s*%?$')
def visible_len(cell):
"""Roughly how wide the cell reads once rendered."""
out = []
for kind, raw in cell:
if kind in ('math', 'dmath'):
t = re.sub(r'\\[a-zA-Z]+', 'x', raw)
out.append(re.sub(r'[{}\\]', '', t))
elif kind == 'esc':
out.append(raw[1])
else:
out.append(re.sub(r'[*_`~]', '', raw))
return len(''.join(out).strip())
def auto_align(column, threshold):
"""column: list of rendered/segmented data cells (no header)."""
vals = [c for c in column if visible_len(c) > 0]
if not vals:
return 'left'
flat = [''.join(r for k, r in c).strip() for c in vals]
if all(NUMERIC.match(v) for v in flat):
return 'right'
if max(visible_len(c) for c in vals) <= threshold:
return 'center'
return 'left'
# ------------------------------------------------------------ md parsing
SEP_CELL = re.compile(r'^\s*:?-+:?\s*$')
def is_sep_row(line):
st = line.strip()
if '-' not in st or not st:
return False
core = st[1:] if st.startswith('|') else st
core = core[:-1] if core.endswith('|') else core
parts = core.split('|')
return bool(parts) and all(SEP_CELL.match(p) for p in parts)
def is_row(line):
return '|' in line and line.strip() != ''
def sep_aligns(line):
st = line.strip()
core = st[1:] if st.startswith('|') else st
core = core[:-1] if core.endswith('|') else core
out = []
for p in core.split('|'):
p = p.strip()
if p.startswith(':') and p.endswith(':'):
out.append('center')
elif p.endswith(':'):
out.append('right')
elif p.startswith(':'):
out.append('left')
else:
out.append(None)
return out
# -------------------------------------------------------------- emission
def build_table(header, aligns, rows, opt, warn):
ncol = max([len(r) for r in rows] + [len(header) if header else 0] + [1])
if header and len(header) != ncol:
warn('header has %d cells, widest row has %d' % (len(header), ncol))
if header:
header = header + [[]] * (ncol - len(header))
rows = [r + [[]] * (ncol - len(r)) for r in rows]
aligns = (aligns + [None] * ncol)[:ncol]
cols = [[r[c] for r in rows] for c in range(ncol)]
for c in range(ncol):
if aligns[c] is None:
aligns[c] = (auto_align(cols[c], opt.threshold)
if opt.align == 'auto' else opt.align)
default = max(set(aligns), key=lambda a: (aligns.count(a), a == 'left'))
cls = opt.cls + (' sortable' if opt.sortable else '')
style = ''
if opt.width != 'auto':
style += 'width:%s;' % opt.width
if default != 'left':
style += ' text-align:%s;' % default
if opt.style:
style += ' ' + opt.style.strip().rstrip(';') + ';'
style = style.strip()
L = ['{| class="%s"%s' % (cls, ' style="%s"' % style if style else '')]
if opt.caption:
L.append('|+ ' + opt.caption)
if header:
for c, cell in enumerate(header):
txt = render_cell(cell, opt.math, header=True,
expand_kets=opt.expand_kets)
if opt.header_align in ('column', 'col'):
st = ('' if aligns[c] == 'center'
else ' style="text-align:%s" |' % aligns[c])
else:
st = ''
L.append(('! %s %s' % (st, txt)).replace('! ', '! ').rstrip())
for r in rows:
L.append('|-')
for c, cell in enumerate(r):
txt = render_cell(cell, opt.math, expand_kets=opt.expand_kets)
if c == 0 and opt.row_headers:
L.append(('! scope="row" | ' + txt).rstrip())
continue
st = ('style="text-align:%s" | ' % aligns[c]
if aligns[c] != default else '')
L.append(('| ' + st + txt).rstrip())
L.append('|}')
return L
def convert(text, opt, warn):
lines = text.replace('\r\n', '\n').replace('\r', '\n').lstrip(
'').split('\n')
out = []
n = len(lines)
i = 0
count = 0
while i < n:
head = None
start = i
if is_sep_row(lines[i]): # header-less table
aligns = sep_aligns(lines[i])
i += 1
elif (i + 1 < n and is_row(lines[i]) and is_sep_row(lines[i + 1])
and not is_sep_row(lines[i])):
head = split_cells(lines[i])
aligns = sep_aligns(lines[i + 1])
i += 2
else:
out.append(lines[i])
i += 1
continue
rows = []
while i < n and is_row(lines[i]) and not is_sep_row(lines[i]):
rows.append(split_cells(lines[i]))
i += 1
if not rows and head is None:
out.extend(lines[start:i])
continue
if opt.no_header and head is not None:
rows.insert(0, head)
head = None
count += 1
table = build_table(head, aligns, rows, opt, warn)
if opt.ai:
# every remaining | is structural (cell text ones are already
# | / \vert): {{!}} expands back to | before Module:AI
# reads the text, so the table survives being a template
# parameter. {| becomes {{{!}} and |} becomes {{!}}} for free.
table = [ln.replace('|', '{{!}}') for ln in table]
out.extend(table)
return '\n'.join(out), count
# ---------------------------------------------------------------- clipboard
def _run(cmd):
if not shutil.which(cmd[0]):
return None
try:
return subprocess.run(cmd, capture_output=True, check=True
).stdout.decode('utf-8', 'replace')
except subprocess.CalledProcessError:
return None
def clip_read():
"""Clipboard, preferring text/html: a table selected in a browser is
on the clipboard as real markup, so no Markdown source is needed."""
for cmd in (['wl-paste', '-t', 'text/html'],
['xclip', '-selection', 'clipboard', '-t', 'text/html', '-o']):
got = _run(cmd)
if got and looks_like_html(got):
return got
for cmd in (['wl-paste', '-n'], ['xclip', '-selection', 'clipboard', '-o'],
['xsel', '-ob']):
got = _run(cmd)
if got is not None:
return got
sys.exit('mdtable2wiki: no clipboard tool (wl-clipboard / xclip / xsel)')
def clip_write(text):
for cmd in (['wl-copy'], ['xclip', '-selection', 'clipboard'],
['xsel', '-ib']):
if shutil.which(cmd[0]):
try:
subprocess.run(cmd, input=text.encode(), check=True)
return cmd[0]
except subprocess.CalledProcessError:
pass
return None
# --------------------------------------------------------------------- cli
def notify(body, icon='x-office-spreadsheet'):
"""A desktop toast: the only report a hotkey run can make."""
if not shutil.which('notify-send'):
return
# Popen, not run: notify-send lives as long as the toast is on screen
subprocess.Popen(['notify-send', '-a', 'mdtable2wiki', '-t', '2500',
'-i', icon, 'mdtable2wiki', body],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
def main():
p = argparse.ArgumentParser(
prog='mdtable2wiki',
description='Markdown table -> full-width MediaWiki table.',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog='Text around the tables is passed through unchanged, so you '
'can pipe a whole answer through it.')
p.add_argument('file', nargs='?', help='input file (default: stdin)')
p.add_argument('-x', '--from-clipboard', action='store_true',
help='read the input from the clipboard')
p.add_argument('-c', '--copy', action='store_true',
help='also copy the result to the clipboard')
p.add_argument('-t', '--caption', default=None, help='table caption (|+)')
p.add_argument('-w', '--width', default='100%',
help='table width, or "auto" for none (default: 100%%)')
p.add_argument('-a', '--align',
choices=['auto', 'left', 'center', 'right'], default='auto',
help='alignment for columns the Markdown does not pin '
'down (default: auto)')
p.add_argument('--threshold', type=int, default=10, metavar='N',
help='auto: centre columns whose cells are <= N chars '
'(default: 10)')
p.add_argument('--header-align', choices=['center', 'column'],
default='center',
help='header cells: keep them centred (default) or make '
'them follow their column')
p.add_argument('-r', '--row-headers', action='store_true',
help='render the first column as ! scope="row" headers')
p.add_argument('-s', '--sortable', action='store_true',
help='add the sortable class')
p.add_argument('--class', dest='cls', default='wikitable',
help='table class (default: wikitable)')
p.add_argument('--style', default='', help='extra CSS for the table tag')
p.add_argument('--math', choices=['dollar', 'tag'], default='dollar',
help='keep $...$ (default, SimpleMathJax) or wrap in '
'<math>')
p.add_argument('-e', '--expand-kets', action='store_true',
help='also expand \\ket \\bra \\ketbra \\braket into '
'\\vert/\\langle, for a wiki whose MathJax has no '
'braket package (here it autoloads, so they are '
'left alone)')
p.add_argument('-H', '--no-header', action='store_true',
help='treat the first row as data, not a header')
p.add_argument('--ai', action='store_true',
help='escape every structural pipe as {{!}} so the '
'table can live inside a template parameter, '
'i.e. the a= of {{AI}}')
p.add_argument('-q', '--quiet', action='store_true',
help='no report on stderr')
p.add_argument('-n', '--notify', action='store_true',
help='report with a desktop toast as well (for hotkeys, '
'where there is no terminal to print to)')
p.add_argument('-V', '--version', action='version',
version='mdtable2wiki ' + VERSION)
opt = p.parse_args()
if opt.from_clipboard:
text = clip_read()
elif opt.file:
with open(opt.file, encoding='utf-8') as f:
text = f.read()
else:
if sys.stdin.isatty() and not opt.quiet:
print('mdtable2wiki: paste the Markdown table, then Ctrl-D '
'(Ctrl-C to abort)', file=sys.stderr)
try:
text = sys.stdin.read()
except KeyboardInterrupt:
sys.exit(130)
from_html = looks_like_html(text)
if from_html:
text = html_tables_to_md(text)
text, from_box = box_tables_to_md(text)
warnings = []
result, count = convert(text, opt, warnings.append)
sys.stdout.write(result if result.endswith('\n') else result + '\n')
if not opt.quiet:
for w in warnings:
print('mdtable2wiki: warning: ' + w, file=sys.stderr)
if count == 0:
# nothing converted: leave the clipboard alone rather than write
# the input back over it
if not opt.quiet:
print('mdtable2wiki: no Markdown table found', file=sys.stderr)
if opt.notify:
notify('no table found', 'dialog-warning')
else:
src = (' from HTML' if from_html else
' from box-drawing glyphs' if from_box else '')
msg = '%d table%s converted%s' % (count, 's' * (count > 1), src)
if opt.copy:
who = clip_write(result)
msg += (', copied to the clipboard via ' + who if who
else ' (no clipboard tool found)')
if not opt.quiet:
print('mdtable2wiki: ' + msg, file=sys.stderr)
if opt.notify:
notify(msg + (', as a {{AI|a=}} parameter' if opt.ai else ''))
if __name__ == '__main__':
main()