# subsync Subtitle toolkit with automatic re-timing. Give it a subtitle that is out of sync (constant offset, 23.976 vs 25 fps drift, or a section cut out) and a correctly timed reference, in any language, or just the video's audio, and it works out the scale and offsets by aligning speech activity. It also converts SRT, WebVTT, ASS/SSA, MicroDVD and SBV, runs manual transforms, lints, diffs, and syncs whole season folders. ## Features - **Formats**: read and write SRT, WebVTT (cue ids, settings, NOTE/STYLE blocks kept), ASS/SSA (header and styles kept, dialogue events converted), MicroDVD `.sub` (frame based, `--fps`), SBV. Encoding detection (UTF-8, UTF-16 with or without BOM, cp1252/latin-1 and others via `charset-normalizer`), line-ending normalisation, recovery from malformed files with warnings. - **Manual transforms**: `shift`, `scale` (fps to fps), `two-point`, `split`, `merge`, `cut`, `strip-hi`, `rewrap`, `fix` (minimum duration, minimum gap, overlaps). - **Auto-sync to a reference subtitle**: 10 ms speech signals, FFT cross-correlation, global scale search, RANSAC plus optimal-partitioning DP for splits, confidence score. See [docs/alignment.md](docs/alignment.md). - **Auto-sync to audio**: `ffmpeg` to 16 kHz mono, numpy energy plus spectral-flatness VAD (30 ms frames), same aligner. - **QA**: `lint` (overlaps, reading speed above 21 chars/s, short or long cues, lines over 42 chars, more than 2 lines, empty cues, unbalanced tags, each with a fix suggestion) and `stats`. - **Batch**: match subtitles to videos or reference subtitles by episode (`S01E02`, `s1e2`, `1x02`) and write `name.synced.srt`. - **Diff**: per-cue timing table plus an ASCII offset-over-time plot. ## Install ```bash uv venv .venv && uv pip install -p .venv/bin/python -e '.[dev]' source .venv/bin/activate ``` Python 3.10+. Runtime dependencies: numpy, click, rich, charset-normalizer. `ffmpeg` is only needed for `--audio` and audio batch mode; without it you get a clear message and everything else still works. ## Quickstart ```bash python examples/generate.py # regenerate the example files (also writes examples/*.wav, not committed) cd examples subsync sync broken_combined.srt --ref reference_other_language.srt -o fixed.srt subsync sync broken_cut.srt --audio reference.wav -o fixed_audio.srt subsync batch season --ref-dir season/refs ``` ## Usage Every command that writes a file accepts `-o/--output`, `--format {srt,vtt,ass,sub,sbv}`, `--fps` (MicroDVD), `--encoding` and `--crlf`. Without `-o` the result goes next to the input as `name..`; the output format follows the extension of `-o`. ### Auto-sync `examples/broken_combined.srt` is the reference with a 90 s cut at 20:00, then a 23.976 to 25 fps mismatch, then a shift of -1.7 s. The reference given here is an "other language" version: scrambled text, jittered cue boundaries, 6% of cues missing. ``` $ subsync sync broken_combined.srt --ref reference_other_language.srt -o out/combined.synced.srt scale 1.042709 (25->23.976); segment 1 from 0.0s: offset +1.794s; segment 2 from 1148.5s: offset +91.762s confidence 0.77 (agreement 89.6%, 57 windows, 2 segment(s)) wrote out/combined.synced.srt (286 cues, srt) ``` Against audio (`reference.wav` is a synthetic WAV built from the cue timings, with noise-burst distractors): ``` $ subsync sync broken_cut.srt --audio reference.wav -o out/cut.audio.srt extracting audio and detecting speech... scale 1.000000; segment 1 from 0.0s: offset -0.007s; segment 2 from 1199.4s: offset +89.983s confidence 0.82 (agreement 91.7%, 57 windows, 2 segment(s)) wrote out/cut.audio.srt (286 cues, srt) ``` `--mode offset` finds one constant shift only. `sync` refuses to write when the confidence is below `--min-confidence` (0.3) unless `--force`. Segment start times are in the original subtitle's time, and `scale 1.042709` is the factor applied to times. ### Diff ``` $ subsync diff broken_combined.srt out/combined.synced.srt --rows 5 286 cue pairs (0 only in A, 0 only in B) start offset B-A: min +2.815s median +37.506s max +162.348s ┏━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓ ┃ Cue ┃ A start ┃ B start ┃ Δ start ┃ Δ end ┃ ┡━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩ │ 1 │ 00:00:23.893 │ 00:00:26.708 │ +2.815 │ +2.932 │ │ 58 │ 00:05:24.281 │ 00:05:39.925 │ +15.644 │ +15.713 │ │ 115 │ 00:11:09.691 │ 00:11:40.088 │ +30.397 │ +30.555 │ │ 172 │ 00:16:38.016 │ 00:17:22.435 │ +44.419 │ +44.498 │ │ 229 │ 00:21:48.235 │ 00:24:15.871 │ +147.636 │ +147.781 │ │ 286 │ 00:27:32.705 │ 00:30:15.053 │ +162.348 │ +162.448 │ └─────┴──────────────┴──────────────┴──────────┴──────────┘ +162.35s | * +147.85s | *** *** ******* +133.34s | ******** +118.84s | +104.34s | +89.83s | +75.33s | +60.83s | +46.32s | **** +31.82s | *************** +17.32s | ***** ********* +2.81s |********* ***** +---------------------------------------------------------------------- 24 1653 (cue start in A, seconds) ``` The plot shows what the sync did: a ramp (the 25 to 23.976 correction) plus a 90 s step (the cut). ### Batch Subtitles in the folder are matched to a reference by episode key; `--ref-dir` selects reference subtitles, otherwise the video (or audio) files in the same folder are used. ``` $ subsync batch season --ref-dir season/refs --dry-run ┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ ┃ Episode ┃ Subtitle ┃ Source ┃ Result ┃ Confidence ┃ Timing model ┃ ┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ │ S01E01 │ Lighthouse.S01E01.720p.srt │ Lighthouse.1x01.en.srt │ match │ │ │ │ S01E02 │ Lighthouse.S01E02.720p.srt │ Lighthouse.s01e02.en.srt │ match │ │ │ │ S01E03 │ Lighthouse.S01E03.720p.srt │ Lighthouse.S01E03.en.srt │ match │ │ │ └─────────┴────────────────────────────┴──────────────────────────┴────────┴────────────┴──────────────┘ ``` ``` $ subsync batch season --ref-dir season/refs ┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Episode ┃ Subtitle ┃ Source ┃ Result ┃ Confidence ┃ Timing model ┃ ┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩ │ S01E01 │ Lighthouse.S01E01.720 │ Lighthouse.1x01.en.sr │ Lighthouse.S01E01.720 │ 0.76 │ x1.00000, +4.50s │ │ │ p.srt │ t │ p.synced.srt │ │ │ │ S01E02 │ Lighthouse.S01E02.720 │ Lighthouse.s01e02.en. │ Lighthouse.S01E02.720 │ 0.75 │ x1.04271, +0.03s │ │ │ p.srt │ srt │ p.synced.srt │ │ │ │ S01E03 │ Lighthouse.S01E03.720 │ Lighthouse.S01E03.en. │ Lighthouse.S01E03.720 │ 0.77 │ x1.00000, -1.97s, │ │ │ p.srt │ srt │ p.synced.srt │ │ +58.03s │ └─────────┴───────────────────────┴───────────────────────┴───────────────────────┴────────────┴───────────────────────┘ ``` ``` $ subsync batch season # audio of the Lighthouse.*.wav files ┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Episode ┃ Subtitle ┃ Source ┃ Result ┃ Confidence ┃ Timing model ┃ ┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩ │ S01E01 │ Lighthouse.S01E01.720 │ Lighthouse.S01E01.720 │ Lighthouse.S01E01.720 │ 0.79 │ x1.00000, +4.47s │ │ │ p.srt │ p.wav │ p.synced.srt │ │ │ │ S01E02 │ Lighthouse.S01E02.720 │ Lighthouse.S01E02.720 │ Lighthouse.S01E02.720 │ 0.81 │ x1.04271, -0.02s │ │ │ p.srt │ p.wav │ p.synced.srt │ │ │ │ S01E03 │ Lighthouse.S01E03.720 │ Lighthouse.S01E03.720 │ Lighthouse.S01E03.720 │ 0.82 │ x1.00000, -1.98s, │ │ │ p.srt │ p.wav │ p.synced.srt │ │ +57.98s │ └─────────┴───────────────────────┴───────────────────────┴───────────────────────┴────────────┴───────────────────────┘ ``` ### Lint and stats ``` $ subsync stats reference.srt format srt encoding utf-8 cues 300 first cue 00:00:26.686 last cue 00:33:52.440 span 00:33:25.754 on screen 00:20:34.427 coverage 61.5% words 3398 chars 17849 mean duration 4.11s longest cue 6.50s mean cps 14.6 max cps 17.1 lines per cue 1.90 longest line 42 longest gap 44.6s warnings 0 ``` `lint` exits 1 when it finds issues, so it works in CI: ``` $ subsync lint examples/noisy.srt ┏━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Cue ┃ Time ┃ Rule ┃ Problem ┃ Suggestion ┃ ┡━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ 1 │ 00:00:01 │ short │ duration 0.40s (min 0.7s) │ extend to 0.7s (subsync fix) │ │ 2 │ 00:00:02 │ line-length │ line of 69 chars (max 42) │ rewrap --max-chars 42 │ │ 2 │ 00:00:02 │ overlap │ overlaps cue 3 by 0.500s │ subsync fix trims the earlier cue │ │ 2 │ 00:00:02 │ reading-speed │ 23.0 chars/sec (max 21) │ extend to 3.29s or shorten the text │ │ 3 │ 00:00:04 │ tags │ unbalanced tag(s): i │ close or remove the tag │ │ 4 │ 00:00:07 │ empty │ cue has no text │ delete the cue │ └─────┴──────────┴───────────────┴───────────────────────────┴─────────────────────────────────────┘ 6 issue(s) in 4 cues: empty 1, line-length 1, overlap 1, reading-speed 1, short 1, tags 1 ``` ### Manual transforms ```bash subsync shift movie.srt +2.5s # also -300ms, +1m30s subsync scale movie.srt --from-fps 25 --to-fps 23.976 subsync two-point movie.srt --first 1 0:01:02.5 --second 840 1:55:10 # cue 1 starts 0:01:02.5, cue 840 at 1:55:10 subsync split movie.srt 0:52:00 # movie.part1.srt, movie.part2.srt (second rebased to 0) subsync merge cd1.srt cd2.srt --offset 0:52:03.2 # default: second starts when the first ends subsync cut movie.srt 0:20:00 0:21:30 # drop the range; later cues move back (--no-ripple to keep) subsync strip-hi movie.srt # [MUSIC], (laughs), music notes, SPEAKER: labels subsync rewrap movie.srt --max-chars 38 subsync fix movie.srt --min-duration 0.8 --min-gap 0.08 subsync convert movie.srt --format vtt # or -o movie.ass ``` Times accept `90`, `1:02:03.5`, `02:03,250`, `1m30s`, `300ms`; offsets take a sign. ## Accuracy `examples/evaluate.py` builds five different 300-cue ground-truth subtitles (seeded), breaks each of them four ways, syncs against the exact reference, an other-language reference and (two seeds) a synthetic WAV, and compares every cue's start with the ground truth. Error is the absolute start-time difference per cue; "breakpoint within 1 s" means the detected breakpoint lies between the last cue before the cut and the first cue after it, give or take 1 s. | variant | source | runs | median error | p95 error | cues within 100 ms | segments found | breakpoint within 1 s | mean confidence | |---|---|---|---|---|---|---|---|---| | broken_offset | exact reference | 5 | 0 ms | 0 ms | 100.0% | 5/5 | n/a | 1.00 | | broken_offset | other-language reference | 5 | 9 ms | 9 ms | 100.0% | 5/5 | n/a | 0.78 | | broken_offset | audio (synthetic WAV) | 2 | 10 ms | 10 ms | 100.0% | 2/2 | n/a | 0.82 | | broken_drift | exact reference | 5 | 0 ms | 0 ms | 100.0% | 5/5 | n/a | 1.00 | | broken_drift | other-language reference | 5 | 10 ms | 10 ms | 100.0% | 5/5 | n/a | 0.78 | | broken_drift | audio (synthetic WAV) | 2 | 18 ms | 18 ms | 100.0% | 2/2 | n/a | 0.82 | | broken_cut | exact reference | 5 | 0 ms | 0 ms | 100.0% | 5/5 | 5/5 | 1.00 | | broken_cut | other-language reference | 5 | 12 ms | 22 ms | 99.8% | 5/5 | 4/5 | 0.78 | | broken_cut | audio (synthetic WAV) | 2 | 19 ms | 23 ms | 100.0% | 2/2 | 2/2 | 0.82 | | broken_combined | exact reference | 5 | 1 ms | 1 ms | 100.0% | 5/5 | 5/5 | 1.00 | | broken_combined | other-language reference | 5 | 11 ms | 17 ms | 99.8% | 5/5 | 4/5 | 0.78 | | broken_combined | audio (synthetic WAV) | 2 | 16 ms | 28 ms | 100.0% | 2/2 | 2/2 | 0.82 | Reading it: constant offsets and drift are recovered to within 20 ms against every kind of reference. The number of segments is right in every run. Against the other-language reference the breakpoint lands within 1 s in 4 of 5 runs for both split variants; in the missed run one cue next to the cut is attached to the wrong side (hence 99.8% of cues within 100 ms rather than 100%). The synthetic WAV is easy (clean voiced bursts over a quiet noise floor); real film audio with music and effects will be harder, which is why `confidence` exists. The other-language "translation" is scrambling plus timing jitter, not real translated subtitles, so real pairs of translations will vary more. Nothing here was measured on real films. ## Configuration There is no config file. Defaults you may want to change are command options: `--mode`, `--min-confidence`, `--max-cps`, `--max-line`, `--max-chars`, `--min-duration`, `--min-gap`. Alignment internals (window 60 s, hop 30 s, DP penalty 0.5) are keyword arguments of `subsync.align.align()`. ## Architecture | Module | Role | |---|---| | `subsync/model.py` | `Cue`, `Subtitle`, markup stripping | | `subsync/timefmt.py` | time parsing and per-format formatting | | `subsync/formats.py` | parsers and writers for the five formats, format detection | | `subsync/codec.py` | bytes to text (encoding detection), `load` / `save` | | `subsync/transforms.py` | shift, scale, two-point, split, merge, cut, strip-hi, rewrap, fix | | `subsync/align.py` | signals, normalised FFT correlation, scale scan, RANSAC, DP partitioning, breakpoint search | | `subsync/audio.py` | ffmpeg extraction and the numpy VAD | | `subsync/lint.py` | lint rules and stats | | `subsync/diff.py` | cue pairing and ASCII plot | | `subsync/batch.py` | episode matching and per-episode jobs | | `subsync/cli.py` | click commands | Library use: ```python from subsync import codec, align sub, ref = codec.load("broken.srt"), codec.load("reference.srt") fixed, model = align.sync_to_subtitle(sub, ref) print(model.describe(), model.confidence) codec.save(fixed, "fixed.srt") ``` ## Examples `examples/` holds `reference.srt` (300 invented cues, about 34 minutes), `reference_other_language.srt`, four broken variants (`broken_offset` +3.2 s, `broken_drift` 25 to 23.976, `broken_cut` 90 s removed at 20:00, `broken_combined`), `noisy.srt` for lint, and `season/` with three episodes plus reference subtitles named in different episode styles. `reference.wav` and the season WAVs are synthetic speech-like audio made by `generate.py`; they are large (tens of MB) and git-ignored, so run `python examples/generate.py` to create them (`--no-audio` skips them). `python examples/evaluate.py` reproduces the accuracy table (about 2 minutes). ## Tests ```bash pytest -q # 77 tests, about 20 s ``` They cover round trips for each format, encoding detection, malformed input, every transform, offset recovery to 20 ms, drift recovery, breakpoint detection, VAD on synthetic audio, ffmpeg extraction (skipped without ffmpeg), lint rules, episode matching and CLI end-to-end runs. ## Limitations - ASS: styles and per-event style names are kept, but override tags other than italic/bold/underline are dropped when converting to another format, and `Comment:` events are not carried over. - WebVTT: NOTE/STYLE blocks attach to the following cue, so dropping that cue drops them. - MicroDVD needs a frame rate: the `{1}{1}fps` header is used when present, else `--fps`, else 23.976 with a warning. - Auto-sync assumes the subtitle and the reference describe the same programme. Jumps under 0.5 s are not treated as breakpoints, and drift is modelled as one global scale. Heavy music under dialogue degrades the VAD. - Audio extraction decodes the entire track into memory (about 230 MB per hour as float32).