CLI · commit e7d695d

lingocheck

i18n/l10n auditor: missing keys, placeholder bugs, ICU validation, pseudo-localization

Download lingocheck.zip
Quickstart
unzip lingocheck.zip && cd lingocheck
uv venv .venv && uv pip install -p .venv/bin/python -e '.[dev]'
.venv/bin/python -m pytest -q

lingocheck

lingocheck audits translation catalogs against your source code and against each other, in CI or on your laptop. It reads nine catalog formats, parses ICU MessageFormat with its own parser, knows the CLDR plural rules for about 70 locale codes, and reports missing keys, renamed or retyped placeholders, broken ICU plurals, dropped markup, stale fuzzy entries, unused keys and hard-coded UI strings. It can also sync catalogs, print coverage per locale and generate a pseudo-locale for layout testing.

Features

Install

cd lingocheck
uv venv .venv && uv pip install -p .venv/bin/python -e '.[dev]'
.venv/bin/lingocheck --help

Python 3.12+. Dependencies: click, rich, pyyaml.

Quickstart

.venv/bin/lingocheck check examples/webapp          # audit everything below that directory
.venv/bin/lingocheck stats examples/webapp          # coverage per locale
.venv/bin/lingocheck sync examples/webapp --dry-run # preview fixes

Catalogs are discovered automatically from the path (see Discovery); a lingocheck.toml found in the path or any parent directory is picked up.

Usage

lingocheck check [PATHS]

Real output from lingocheck check examples/webapp (first part; the full run reports 28 errors, 21 warnings and 2 infos):

examples/webapp/android/app/src/main/java/com/shoplite/MainActivity.kt
  error   missing-in-catalog    L15   key 'promo_banner' is used here but missing from the en 
catalog

examples/webapp/android/app/src/main/res/values-de/strings.xml
  error   missing-key                 [de] missing key 'legacy_footer' (present in en)
  error   missing-key                 [de] missing key 'sort_options[2]' (present in en)
  warning untranslated          L12   [de] identical to source text: 'Retry'
  error   markup-mismatch       L13   [de] markup tags differ: </b>, <b>

examples/webapp/android/app/src/main/res/values-fr/strings.xml
  error   missing-key                 [fr] missing key 'legacy_footer' (present in en)
  error   placeholder-mismatch  L3    [fr] placeholder '1' type differs: source string, translation 
int

examples/webapp/android/app/src/main/res/values-ja/strings.xml
  error   missing-key                 [ja] missing key 'legacy_footer' (present in en)
  warning plural-invalid        L4    [ja] plural category 'one' is never selected by CLDR rules for
ja
  error   markup-mismatch       L14   [ja] markup tags differ: </b>, <b>

examples/webapp/android/app/src/main/res/values/strings.xml
  warning unused-key            L17   key 'legacy_footer' is never referenced in source

examples/webapp/django_app/locale/de/LC_MESSAGES/django.po
  error   missing-key                 [de] missing key 'Order history' (present in en)
  error   placeholder-mismatch  L24   [de] missing placeholder 'name'
  error   placeholder-mismatch  L24   [de] unexpected placeholder 'user' (not in source)
  warning stale-fuzzy           L37   [de] fuzzy translation has not been reviewed (source text 
changed since it was written)
  warning unused-key            L40   key 'Retired promotion' is never referenced in source

examples/webapp/django_app/locale/fr/LC_MESSAGES/django.po
  warning empty-translation     L27   [fr] translation is empty

examples/webapp/django_app/locale/pl/LC_MESSAGES/django.po
  error   plural-incomplete     L11   [pl] plural forms for pl need one, few, many; missing many

examples/webapp/django_app/shop/templates/shop/cart.html
  warning hardcoded-string      L3    hard-coded UI text: 'Free shipping on orders over $50'
  warning hardcoded-string      L5    hard-coded UI text: 'alt=Summer promotion'

examples/webapp/django_app/shop/views.py
  error   missing-in-catalog    L14   key 'Contact support' is used here but missing from the en 
...
28 error(s), 21 warning(s), 2 info - 18 catalog file(s), 6 locale(s), 6 source file(s) scanned

Options: -f rich|json|junit|github, -o FILE, --fail-on error|warning|info|never, --select RULE, --ignore RULE, -l LOCALE, --no-source, -b BASE_LOCALE, -c CONFIG.

Exit codes: 0 clean (or below --fail-on), 1 findings at or above --fail-on, 2 usage or config error.

JSON (-f json), trimmed:

{
  "summary": {
    "errors": 3,
    "warnings": 0,
    "infos": 0,
    "total": 3
  },
  "catalogs": 9,
  "locales": 4,
  "source_files": 0,
  "issues": [
    {
      "rule": "plural-incomplete",
      "severity": "error",
      "message": "plural forms for pl need one, few, many; missing many",
      "file": "examples/webapp/django_app/locale/pl/LC_MESSAGES/django.po",
      "line": 11,
      "locale": "pl",
      "key": "%(count)d item"
    },
    {
      "rule": "plural-incomplete",
      "severity": "error",
      "message": "plural 'count' for pl needs one, few, many, other; missing few, many",
      "file": "examples/webapp/locales/pl/checkout.json",
      "line": 2,
      "locale": "pl",
      "key": "summary"
    },
    {
    ...

GitHub annotations (-f github) print ::error file=...,line=...,title=rule::message; JUnit (-f junit) writes one testsuite per file with a failure per error or warning and skipped for info.

CI example:

- run: lingocheck check . --format github --fail-on warning

lingocheck stats [PATHS]

                          Coverage vs en                           
┏━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┓
┃ Locale ┃ Keys ┃ Translated ┃ Fuzzy ┃ Empty ┃ Missing ┃ Coverage ┃
┡━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━┩
│ ar     │   20 │         19 │     0 │     0 │       1 │    95.0% │
│ de     │   41 │         35 │     1 │     0 │       5 │    85.4% │
│ fr     │   41 │         37 │     0 │     2 │       2 │    90.2% │
│ ja     │   29 │         27 │     0 │     0 │       2 │    93.1% │
│ pl     │   32 │         31 │     0 │     0 │       1 │    96.9% │
└────────┴──────┴────────────┴───────┴───────┴─────────┴──────────┘

Only translated, non-fuzzy, non-empty entries count as covered. --fail-under 90 exits 1 when any locale is below 90%; -f json for machines.

lingocheck sync [PATHS]

Adds keys that exist in the base but not in a target (source text by default, --empty for empty strings, flagged fuzzy in PO and XLIFF), completes plural forms to the target locale's CLDR categories, sorts keys naturally and normalizes formatting. --prune removes keys that are not in the base catalog and base keys never referenced in source. --dry-run only reports.

$ lingocheck sync examples/webapp/locales --dry-run
would update locales/en/translation.json: +0 added, -0 removed
would update locales/ar/translation.json: +1 added, -0 removed
would update locales/de/translation.json: +2 added, -0 removed
would update locales/fr/translation.json: +1 added, -0 removed
would update locales/ja/translation.json: +1 added, -0 removed
would update locales/pl/translation.json: +2 added, -0 removed
...

Resulting change to locales/pl/translation.json (excerpt of diff): the missing Polish plural form is added with the English text so it is visible and greppable.

<     "save": "Zapisz",
10c10,11
<     "checkout": "Przejdź do kasy"
---
>     "checkout": "Przejdź do kasy",
>     "save": "Zapisz"
12a14
>     "empty": "Twój koszyk jest pusty",
14a17
>     "items_many": "{{count}} items in your cart",
16d18
<     "empty": "Twój koszyk jest pusty",
18a21,32
>   "errors": {
>     "required": "To pole jest wymagane.",
>     "required_short": "To pole jest wymagane."
>   },
>   "home": {

lingocheck pseudo FILE

$ lingocheck pseudo locales/en/checkout.json
wrote locales/en-XA/checkout.json (4 entries, locale en-XA)
{
  "summary": "[{count, plural, =0 {Ýöûŕ çåŕŧ îš émpŧýļöŕéɱ} one {# îŧém, ŧöŧåļ ļöŕ{total, number, ::currency/USD}} other {# îŧémš, ŧöŧåļ ļöŕé{total, number, ::currency/USD}}}]",
  "shipping": "[Šhîpš öñ ļö{date, date, short}]",
  "gender": "[{gender, select, female {Šhé öŕðéŕéðļöŕé} male {Hé öŕðéŕéðļöŕ} other {Théý öŕðéŕéðļöŕé}}]",
  "rank": "[Ýöû åŕé ļö{n, selectordinal, one {#šŧļ} two {#ñðļ} few {#ŕðļ} other {#ŧhļ}} îñ ļîñéļö]"
}

--rtl wraps each message in U+202E/U+202C and writes the en-XB locale; --expansion 50 changes growth. The output path is derived from the input layout (values-en-rXA, en-XA.lproj, ...). The generated catalog passes check with no placeholder, markup or ICU findings.

lingocheck plurals LOCALE and lingocheck icu MESSAGE

$ lingocheck plurals pl
cardinal:
  one    1
  few    2, 3, 4, 22, 23, 24
  many   0, 5, 6, 7, 8, 9
  other  1.5, 0.5, 2.5, 1.0, 10.5
$ lingocheck icu '{n, plural, one {# file} other {# files}}' -l pl
valid
  argument n: plural
  n: pl needs one, few, many, other; missing few, many

Catalog discovery

A file is a catalog when its path says which locale it holds. Recognised layouts, the locale being replaced by a template marker so that all locales of one catalog are grouped:

Layout Example
directory named by locale locales/de/translation.json, locale/de/LC_MESSAGES/django.po
file named by locale locales/de.json, config/locales/de.yml
locale suffix messages_de.properties, app_de.arb, messages.de.xlf
Android qualifiers res/values/strings.xml, res/values-de/, values-pt-rBR/
iOS lproj en.lproj/Localizable.strings, de.lproj/...

messages.properties is the base of messages_xx.properties. Gettext and XLIFF carry source text in each entry, so the base catalog is rebuilt from msgid / <source>. Directories such as node_modules, .git, .venv, build are skipped.

Configuration

lingocheck.toml (or [tool.lingocheck] in pyproject.toml); paths are relative to the config file. The example project uses:

base_locale = "en"
source = ["src", "django_app", "android/app/src"]
ignore_unused = ["legacy.*", "brand.*"]

[untranslated]
allow = ["^OK$"]
keys = ["brand.*"]

[length]
"button.*" = 180

[scan]
hardcoded_allow = ["^Loading"]

All options:

Key Meaning
base_locale source locale, default en
locales restrict to these target locales
source directories scanned for key usage (default: the paths given on the command line)
exclude glob patterns for files/directories to skip
fail_on error (default), warning, info, never
icu "auto" (default), true or false: whether ICU syntax errors are reported
ignore_unused key globs never reported as unused
[severity] per rule: error, warning, info or off
[untranslated] allow regexes matched against the text, keys globs
[length] key glob to maximum growth in percent
[scan] patterns (extra regexes with a (?P<key>...) group), hardcoded (bool), hardcoded_allow regexes

Architecture

Module Role
catalog.py Entry / Catalog model shared by all formats
formats/ one reader+writer per format (json_fmt, yaml_fmt, po, android, ios, xliff, properties, arb)
icu.py ICU MessageFormat parser producing a small AST
plurals.py CLDR rule strings, rule compiler, operand evaluation
placeholders.py placeholder and tag extraction for all syntaxes at once
project.py locale-from-path detection, grouping, base synthesis
checks.py the rules
scan.py key usage extraction and hard-coded text heuristic
pseudo.py, sync.py, stats.py the other commands
report.py, cli.py, config.py output, command line, configuration

Design notes with worked examples are in docs/: rules.md, formats.md, icu-and-plurals.md.

Tests

.venv/bin/pytest -q

158 tests: format round-trips, a valid/invalid ICU corpus, CLDR plural samples for en/fr/pl/ar/ru/ja and more, every check, key-extraction regexes, pseudo-localization invariants, sync, and the CLI end to end.

Limitations

License

MIT, see LICENSE.