fix: Remediate 1074 glitch tokens in tokenizer.json

#49
by gkielian - opened

Glitch Token Remediation — Tokenizer Patch

Summary

This PR patches tokenizer.json to remediate 1074 glitch tokens identified
by embedding vector analysis. Glitch tokens are vocabulary entries whose embeddings
have collapsed to near-identical representations, causing unpredictable model behavior
when they appear in input.

No model weights are modified — this is a tokenizer-only fix.

Technique: Hybrid (merge pruning + placeholder rename)

The patch uses two complementary strategies:

1. Merge pruning (552 tokens): Removes the BPE merge rules that
produce multi-character glitch tokens. The vocab entry stays (ID preserved) but
becomes unreachable — text that would have matched is now tokenized as trained
constituent subwords instead.

2. Placeholder rename (522 tokens): For single-character glitch tokens
(rare Unicode characters) that have no producing merge rule, the vocab string is renamed to
<glitch_pruned_ID>. The original character now falls through to byte_fallback,
encoding as well-trained UTF-8 byte tokens.

Safety: Placeholder strings cannot be triggered by user input. BPE builds tokens
bottom-up from characters via merge rules only — since no merge rule produces these strings,
they are permanently unreachable. Gemma's split-digits tokenization policy provides an
additional layer of safety.

675 merges were removed for 552 merge-pruned tokens because some glitch tokens have more than one BPE merge path, and every path must be removed.

Before / after examples: google/gemma-3-1b-it

Merge prune example: IMQTTRVM, token ID 68512

Prompt: Please repeat the following string exactly, with nothing else: "IMQTTRVM"

Tokens for the string Model output
Before ['IMQTTRVM'] → [68512] "Please repeat the following string exactly, with nothing else: 'Please repeat the following string … ❌
After ['IM', 'QTTR', 'VM'] → [2783, 67694, 19137] IMQTTRVM ✅

Placeholder rename example: 𡉺 (U+2127A), token ID 249788

Prompt: Please repeat the following string exactly, with nothing else: "𡉺"

Tokens for the string Model output
Before ['𡉺'] → [249788] "Please repeat the following string exactly, with nothing else: " ❌
After ['<0xF0>', '<0xA1>', '<0x89>', '<0xBA>'] → [478, 399, 375, 424] 𡉺 ✅

⚠️ Note for Agentic Systems

Many remediated glitch tokens are shared across Gemma model families (Gemma 1, 2, 3, 4).
In agentic pipelines, care should be taken that these token strings are not inadvertently
injected into prompts for unpatched models. We recommend applying this patch consistently
across all Gemma models used in a pipeline.

What is preserved

  • ✅ len(tokenizer) — unchanged (262,144)
  • ✅ All token IDs — stable, no re-indexing
  • ✅ Chat template — identical to original
  • ✅ tokenizer_config.json — identical to original
  • ✅ All special tokens and added_tokens — unchanged
  • ✅ Normal text tokenization — verified via benchmarks
  • ✅ Zero model weight changes

Diff summary

Original Patched Delta
Vocab size 262,144 262,144 0
Merges 514,906 514,231 −675
Vocab renamed — 522 +522

Vocab renames (522 entries)

Click to expand all 522 renamed vocab entries
ID Original Patched
247832 𒍤 (U+12364) <glitch_pruned_247832>
247835 𒋺 (U+122FA) <glitch_pruned_247835>
248412 𒇧 (U+121E7) <glitch_pruned_248412>
248418 𒊐 (U+12290) <glitch_pruned_248418>
248787 𒐴 (U+12434) <glitch_pruned_248787>
249289 𒆚 (U+1219A) <glitch_pruned_249289>
249290 𒋒 (U+122D2) <glitch_pruned_249290>
249293 𒋜 (U+122DC) <glitch_pruned_249293>
249295 𒋝 (U+122DD) <glitch_pruned_249295>
249303 𒁵 (U+12075) <glitch_pruned_249303>
249305 𒂈 (U+12088) <glitch_pruned_249305>
249313 𒄝 (U+1211D) <glitch_pruned_249313>
249318 𒅊 (U+1214A) <glitch_pruned_249318>
249329 𒈮 (U+1222E) <glitch_pruned_249329>
249331 𒉄 (U+12244) <glitch_pruned_249331>
249334 𒉚 (U+1225A) <glitch_pruned_249334>
249338 𒋁 (U+122C1) <glitch_pruned_249338>
249354 𒈿 (U+1223F) <glitch_pruned_249354>
249357 𒌪 (U+1232A) <glitch_pruned_249357>
249628 (U+F54B) <glitch_pruned_249628>
249779 𒋖 (U+122D6) <glitch_pruned_249779>
249785 𒍞 (U+1235E) <glitch_pruned_249785>
249786 𒐺 (U+1243A) <glitch_pruned_249786>
249788 𡉺 (U+2127A) <glitch_pruned_249788>
250240 ꗑ (U+A5D1) <glitch_pruned_250240>
250241 ꗒ (U+A5D2) <glitch_pruned_250241>
250242 ꗓ (U+A5D3) <glitch_pruned_250242>
250243 ꗔ (U+A5D4) <glitch_pruned_250243>
250244 ꗕ (U+A5D5) <glitch_pruned_250244>
250245 ꗖ (U+A5D6) <glitch_pruned_250245>
250247 ꗘ (U+A5D8) <glitch_pruned_250247>
250248 ꗙ (U+A5D9) <glitch_pruned_250248>
250252 ꗝ (U+A5DD) <glitch_pruned_250252>
250257 ꗢ (U+A5E2) <glitch_pruned_250257>
250260 ꗦ (U+A5E6) <glitch_pruned_250260>
250263 ꗩ (U+A5E9) <glitch_pruned_250263>
250264 ꗪ (U+A5EA) <glitch_pruned_250264>
250265 ꗫ (U+A5EB) <glitch_pruned_250265>
250268 ꗮ (U+A5EE) <glitch_pruned_250268>
250372 𒀁 (U+12001) <glitch_pruned_250372>
250373 𒀂 (U+12002) <glitch_pruned_250373>
250374 𒀅 (U+12005) <glitch_pruned_250374>
250377 𒀐 (U+12010) <glitch_pruned_250377>
250379 𒀙 (U+12019) <glitch_pruned_250379>
250380 𒀚 (U+1201A) <glitch_pruned_250380>
250381 𒀞 (U+1201E) <glitch_pruned_250381>
250383 𒀩 (U+12029) <glitch_pruned_250383>
250386 𒀬 (U+1202C) <glitch_pruned_250386>
250395 𒁆 (U+12046) <glitch_pruned_250395>
250398 𒁎 (U+1204E) <glitch_pruned_250398>
250399 𒁑 (U+12051) <glitch_pruned_250399>
250403 𒁙 (U+12059) <glitch_pruned_250403>
250404 𒁝 (U+1205D) <glitch_pruned_250404>
250405 𒁦 (U+12066) <glitch_pruned_250405>
250406 𒁨 (U+12068) <glitch_pruned_250406>
250407 𒁫 (U+1206B) <glitch_pruned_250407>
250408 𒁬 (U+1206C) <glitch_pruned_250408>
250409 𒁭 (U+1206D) <glitch_pruned_250409>
250412 𒁱 (U+12071) <glitch_pruned_250412>
250414 𒁶 (U+12076) <glitch_pruned_250414>
250417 𒁻 (U+1207B) <glitch_pruned_250417>
250418 𒁼 (U+1207C) <glitch_pruned_250418>
250426 𒂉 (U+12089) <glitch_pruned_250426>
250428 𒂕 (U+12095) <glitch_pruned_250428>
250431 𒂘 (U+12098) <glitch_pruned_250431>
250434 𒂬 (U+120AC) <glitch_pruned_250434>
250436 𒃕 (U+120D5) <glitch_pruned_250436>
250437 𒃞 (U+120DE) <glitch_pruned_250437>
250438 𒃠 (U+120E0) <glitch_pruned_250438>
250439 𒃡 (U+120E1) <glitch_pruned_250439>
250440 𒃢 (U+120E2) <glitch_pruned_250440>
250441 𒃣 (U+120E3) <glitch_pruned_250441>
250442 𒃤 (U+120E4) <glitch_pruned_250442>
250443 𒃥 (U+120E5) <glitch_pruned_250443>
250445 𒃴 (U+120F4) <glitch_pruned_250445>
250447 𒃸 (U+120F8) <glitch_pruned_250447>
250448 𒃽 (U+120FD) <glitch_pruned_250448>
250451 𒄃 (U+12103) <glitch_pruned_250451>
250453 𒄇 (U+12107) <glitch_pruned_250453>
250455 𒄉 (U+12109) <glitch_pruned_250455>
250456 𒄋 (U+1210B) <glitch_pruned_250456>
250459 𒄕 (U+12115) <glitch_pruned_250459>
250461 𒄗 (U+12117) <glitch_pruned_250461>
250463 𒄛 (U+1211B) <glitch_pruned_250463>
250466 𒄡 (U+12121) <glitch_pruned_250466>
250468 𒄤 (U+12124) <glitch_pruned_250468>
250469 𒄦 (U+12126) <glitch_pruned_250469>
250470 𒄧 (U+12127) <glitch_pruned_250470>
250472 𒄫 (U+1212B) <glitch_pruned_250472>
250473 𒄮 (U+1212E) <glitch_pruned_250473>
250475 𒄵 (U+12135) <glitch_pruned_250475>
250477 𒄾 (U+1213E) <glitch_pruned_250477>
250486 𒅘 (U+12158) <glitch_pruned_250486>
250487 𒅚 (U+1215A) <glitch_pruned_250487>
250488 𒅛 (U+1215B) <glitch_pruned_250488>
250490 𒅡 (U+12161) <glitch_pruned_250490>
250491 𒅢 (U+12162) <glitch_pruned_250491>
250493 𒅪 (U+1216A) <glitch_pruned_250493>
250494 𒅫 (U+1216B) <glitch_pruned_250494>
250495 𒅮 (U+1216E) <glitch_pruned_250495>
250496 𒅲 (U+12172) <glitch_pruned_250496>
250498 𒅸 (U+12178) <glitch_pruned_250498>
250499 𒅺 (U+1217A) <glitch_pruned_250499>
250500 𒅻 (U+1217B) <glitch_pruned_250500>
250501 𒅼 (U+1217C) <glitch_pruned_250501>
250502 𒅾 (U+1217E) <glitch_pruned_250502>
250503 𒆁 (U+12181) <glitch_pruned_250503>
250504 𒆂 (U+12182) <glitch_pruned_250504>
250505 𒆃 (U+12183) <glitch_pruned_250505>
250506 𒆉 (U+12189) <glitch_pruned_250506>
250510 𒆑 (U+12191) <glitch_pruned_250510>
250511 𒆒 (U+12192) <glitch_pruned_250511>
250512 𒆓 (U+12193) <glitch_pruned_250512>
250515 𒆛 (U+1219B) <glitch_pruned_250515>
250516 𒆢 (U+121A2) <glitch_pruned_250516>
250517 𒆣 (U+121A3) <glitch_pruned_250517>
250519 𒆦 (U+121A6) <glitch_pruned_250519>
250522 𒆭 (U+121AD) <glitch_pruned_250522>
250526 𒆹 (U+121B9) <glitch_pruned_250526>
250527 𒆼 (U+121BC) <glitch_pruned_250527>
250528 𒇆 (U+121C6) <glitch_pruned_250528>
250529 𒇇 (U+121C7) <glitch_pruned_250529>
250530 𒇋 (U+121CB) <glitch_pruned_250530>
250531 𒇌 (U+121CC) <glitch_pruned_250531>
250532 𒇑 (U+121D1) <glitch_pruned_250532>
250533 𒇒 (U+121D2) <glitch_pruned_250533>
250534 𒇙 (U+121D9) <glitch_pruned_250534>
250536 𒇦 (U+121E6) <glitch_pruned_250536>
250537 𒇭 (U+121ED) <glitch_pruned_250537>
250539 𒇳 (U+121F3) <glitch_pruned_250539>
250541 𒇵 (U+121F5) <glitch_pruned_250541>
250543 𒇸 (U+121F8) <glitch_pruned_250543>
250545 𒈖 (U+12216) <glitch_pruned_250545>
250549 𒈢 (U+12222) <glitch_pruned_250549>
250554 𒈭 (U+1222D) <glitch_pruned_250554>
250556 𒈺 (U+1223A) <glitch_pruned_250556>
250557 𒈻 (U+1223B) <glitch_pruned_250557>
250558 𒈽 (U+1223D) <glitch_pruned_250558>
250560 𒉇 (U+12247) <glitch_pruned_250560>
250562 𒉐 (U+12250) <glitch_pruned_250562>
250563 𒉑 (U+12251) <glitch_pruned_250563>
250564 𒉒 (U+12252) <glitch_pruned_250564>
250565 𒉓 (U+12253) <glitch_pruned_250565>
250566 𒉖 (U+12256) <glitch_pruned_250566>
250568 𒉛 (U+1225B) <glitch_pruned_250568>
250569 𒉞 (U+1225E) <glitch_pruned_250569>
250571 𒉦 (U+12266) <glitch_pruned_250571>
250572 𒉮 (U+1226E) <glitch_pruned_250572>
250573 𒉯 (U+1226F) <glitch_pruned_250573>
250576 𒉾 (U+1227E) <glitch_pruned_250576>
250577 𒊋 (U+1228B) <glitch_pruned_250577>
250582 𒊝 (U+1229D) <glitch_pruned_250582>
250583 𒊠 (U+122A0) <glitch_pruned_250583>
250584 𒊢 (U+122A2) <glitch_pruned_250584>
250590 𒊾 (U+122BE) <glitch_pruned_250590>
250591 𒋂 (U+122C2) <glitch_pruned_250591>
250593 𒋄 (U+122C4) <glitch_pruned_250593>
250595 𒋇 (U+122C7) <glitch_pruned_250595>
250596 𒋋 (U+122CB) <glitch_pruned_250596>
250597 𒋲 (U+122F2) <glitch_pruned_250597>
250599 𒋽 (U+122FD) <glitch_pruned_250599>
250600 𒌀 (U+12300) <glitch_pruned_250600>
250601 𒌁 (U+12301) <glitch_pruned_250601>
250610 𒌘 (U+12318) <glitch_pruned_250610>
250613 𒌢 (U+12322) <glitch_pruned_250613>
250617 𒌲 (U+12332) <glitch_pruned_250617>
250618 𒌴 (U+12334) <glitch_pruned_250618>
250621 𒌺 (U+1233A) <glitch_pruned_250621>
250622 𒍂 (U+12342) <glitch_pruned_250622>
250623 𒍇 (U+12347) <glitch_pruned_250623>
250624 𒍍 (U+1234D) <glitch_pruned_250624>
250625 𒍎 (U+1234E) <glitch_pruned_250625>
250626 𒍐 (U+12350) <glitch_pruned_250626>
250627 𒍒 (U+12352) <glitch_pruned_250627>
250628 𒍗 (U+12357) <glitch_pruned_250628>
250629 𒍚 (U+1235A) <glitch_pruned_250629>
250636 𒍨 (U+12368) <glitch_pruned_250636>
250639 𒐍 (U+1240D) <glitch_pruned_250639>
250640 𒐐 (U+12410) <glitch_pruned_250640>
250641 𒐑 (U+12411) <glitch_pruned_250641>
250642 𒐒 (U+12412) <glitch_pruned_250642>
250643 𒐓 (U+12413) <glitch_pruned_250643>
250644 𒐔 (U+12414) <glitch_pruned_250644>
250645 𒑀 (U+12440) <glitch_pruned_250645>
250646 𒑁 (U+12441) <glitch_pruned_250646>
250647 𒑂 (U+12442) <glitch_pruned_250647>
250648 𒑄 (U+12444) <glitch_pruned_250648>
250649 𒑅 (U+12445) <glitch_pruned_250649>
250650 𒑆 (U+12446) <glitch_pruned_250650>
250651 𒑇 (U+12447) <glitch_pruned_250651>
250653 𒑋 (U+1244B) <glitch_pruned_250653>
250655 𒑑 (U+12451) <glitch_pruned_250655>
250656 𒑒 (U+12452) <glitch_pruned_250656>
250657 𒑚 (U+1245A) <glitch_pruned_250657>
250658 𒑛 (U+1245B) <glitch_pruned_250658>
250659 𒑜 (U+1245C) <glitch_pruned_250659>
250660 𒑱 (U+12471) <glitch_pruned_250660>
250665 𒀿 (U+1203F) <glitch_pruned_250665>
250666 𒁖 (U+12056) <glitch_pruned_250666>
250667 𒁟 (U+1205F) <glitch_pruned_250667>
250668 𒂀 (U+12080) <glitch_pruned_250668>
250669 𒂙 (U+12099) <glitch_pruned_250669>
250673 𒅂 (U+12142) <glitch_pruned_250673>
250674 𒅓 (U+12153) <glitch_pruned_250674>
250675 𒅥 (U+12165) <glitch_pruned_250675>
250676 𒆯 (U+121AF) <glitch_pruned_250676>
250677 𒇀 (U+121C0) <glitch_pruned_250677>
250678 𒇅 (U+121C5) <glitch_pruned_250678>
250679 𒇡 (U+121E1) <glitch_pruned_250679>
250681 𒇹 (U+121F9) <glitch_pruned_250681>
250682 𒈯 (U+1222F) <glitch_pruned_250682>
250683 𒈰 (U+12230) <glitch_pruned_250683>
250685 𒉙 (U+12259) <glitch_pruned_250685>
250686 𒉠 (U+12260) <glitch_pruned_250686>
250690 𒊷 (U+122B7) <glitch_pruned_250690>
250692 𒊿 (U+122BF) <glitch_pruned_250692>
250694 𒌿 (U+1233F) <glitch_pruned_250694>
250695 𒍀 (U+12340) <glitch_pruned_250695>
250696 𒍏 (U+1234F) <glitch_pruned_250696>
250697 𒍩 (U+12369) <glitch_pruned_250697>
250699 𒑈 (U+12448) <glitch_pruned_250699>
250700 𒑏 (U+1244F) <glitch_pruned_250700>
250701 𒑲 (U+12472) <glitch_pruned_250701>
250702 𒑳 (U+12473) <glitch_pruned_250702>
251510 𒀦 (U+12026) <glitch_pruned_251510>
251511 𒁞 (U+1205E) <glitch_pruned_251511>
251513 𒂰 (U+120B0) <glitch_pruned_251513>
251514 𒃋 (U+120CB) <glitch_pruned_251514>
251515 𒃌 (U+120CC) <glitch_pruned_251515>
251516 𒃎 (U+120CE) <glitch_pruned_251516>
251517 𒄁 (U+12101) <glitch_pruned_251517>
251518 𒈶 (U+12236) <glitch_pruned_251518>
251519 𒉔 (U+12254) <glitch_pruned_251519>
251520 𒉕 (U+12255) <glitch_pruned_251520>
251521 𒉜 (U+1225C) <glitch_pruned_251521>
251523 𒋎 (U+122CE) <glitch_pruned_251523>
251524 𒋐 (U+122D0) <glitch_pruned_251524>
251527 𒋟 (U+122DF) <glitch_pruned_251527>
251528 𒋱 (U+122F1) <glitch_pruned_251528>
251530 𒌊 (U+1230A) <glitch_pruned_251530>
251532 𒍫 (U+1236B) <glitch_pruned_251532>
251534 𒐄 (U+12404) <glitch_pruned_251534>
251535 𒐆 (U+12406) <glitch_pruned_251535>
251536 𒐻 (U+1243B) <glitch_pruned_251536>
251537 𒐽 (U+1243D) <glitch_pruned_251537>
251538 𒑔 (U+12454) <glitch_pruned_251538>
251539 𒑙 (U+12459) <glitch_pruned_251539>
251549 𒆟 (U+1219F) <glitch_pruned_251549>
251550 𒈕 (U+12215) <glitch_pruned_251550>
251555 𒑖 (U+12456) <glitch_pruned_251555>
252598 򿪺 (U+BFABA) <glitch_pruned_252598>
252834 𒁜 (U+1205C) <glitch_pruned_252834>
252837 𒐖 (U+12416) <glitch_pruned_252837>
252838 𒐗 (U+12417) <glitch_pruned_252838>
252839 𒐘 (U+12418) <glitch_pruned_252839>
252840 𒐙 (U+12419) <glitch_pruned_252840>
252841 𒐚 (U+1241A) <glitch_pruned_252841>
252842 𒐛 (U+1241B) <glitch_pruned_252842>
252843 𒐜 (U+1241C) <glitch_pruned_252843>
252844 𒐝 (U+1241D) <glitch_pruned_252844>
252846 𒑃 (U+12443) <glitch_pruned_252846>
252848 𒑌 (U+1244C) <glitch_pruned_252848>
252849 𒑍 (U+1244D) <glitch_pruned_252849>
252850 𒑎 (U+1244E) <glitch_pruned_252850>
252851 𒑡 (U+12461) <glitch_pruned_252851>
252852 𒑢 (U+12462) <glitch_pruned_252852>
252869 𒀄 (U+12004) <glitch_pruned_252869>
252870 𒀆 (U+12006) <glitch_pruned_252870>
252871 𒀇 (U+12007) <glitch_pruned_252871>
252875 𒀍 (U+1200D) <glitch_pruned_252875>
252877 𒀑 (U+12011) <glitch_pruned_252877>
252878 𒀒 (U+12012) <glitch_pruned_252878>
252879 𒀓 (U+12013) <glitch_pruned_252879>
252881 𒀛 (U+1201B) <glitch_pruned_252881>
252882 𒀟 (U+1201F) <glitch_pruned_252882>
252883 𒀡 (U+12021) <glitch_pruned_252883>
252884 𒀢 (U+12022) <glitch_pruned_252884>
252885 𒀣 (U+12023) <glitch_pruned_252885>
252886 𒀤 (U+12024) <glitch_pruned_252886>
252887 𒀧 (U+12027) <glitch_pruned_252887>
252888 𒀨 (U+12028) <glitch_pruned_252888>
252890 𒀺 (U+1203A) <glitch_pruned_252890>
252892 𒁋 (U+1204B) <glitch_pruned_252892>
252894 𒁗 (U+12057) <glitch_pruned_252894>
252895 𒁘 (U+12058) <glitch_pruned_252895>
252896 𒁚 (U+1205A) <glitch_pruned_252896>
252897 𒁛 (U+1205B) <glitch_pruned_252897>
252898 𒁠 (U+12060) <glitch_pruned_252898>
252899 𒁡 (U+12061) <glitch_pruned_252899>
252900 𒁢 (U+12062) <glitch_pruned_252900>
252901 𒁣 (U+12063) <glitch_pruned_252901>
252902 𒁤 (U+12064) <glitch_pruned_252902>
252903 𒁥 (U+12065) <glitch_pruned_252903>
252904 𒁧 (U+12067) <glitch_pruned_252904>
252905 𒁩 (U+12069) <glitch_pruned_252905>
252906 𒁪 (U+1206A) <glitch_pruned_252906>
252907 𒂢 (U+120A2) <glitch_pruned_252907>
252908 𒂣 (U+120A3) <glitch_pruned_252908>
252909 𒂤 (U+120A4) <glitch_pruned_252909>
252910 𒂧 (U+120A7) <glitch_pruned_252910>
252911 𒂩 (U+120A9) <glitch_pruned_252911>
252912 𒂫 (U+120AB) <glitch_pruned_252912>
252914 𒂯 (U+120AF) <glitch_pruned_252914>
252915 𒂱 (U+120B1) <glitch_pruned_252915>
252916 𒂲 (U+120B2) <glitch_pruned_252916>
252917 𒂳 (U+120B3) <glitch_pruned_252917>
252918 𒂴 (U+120B4) <glitch_pruned_252918>
252919 𒂶 (U+120B6) <glitch_pruned_252919>
252920 𒂸 (U+120B8) <glitch_pruned_252920>
252921 𒂺 (U+120BA) <glitch_pruned_252921>
252922 𒂽 (U+120BD) <glitch_pruned_252922>
252923 𒂾 (U+120BE) <glitch_pruned_252923>
252924 𒂿 (U+120BF) <glitch_pruned_252924>
252925 𒃂 (U+120C2) <glitch_pruned_252925>
252926 𒃃 (U+120C3) <glitch_pruned_252926>
252927 𒃄 (U+120C4) <glitch_pruned_252927>
252928 𒃅 (U+120C5) <glitch_pruned_252928>
252929 𒃆 (U+120C6) <glitch_pruned_252929>
252930 𒃈 (U+120C8) <glitch_pruned_252930>
252931 𒃉 (U+120C9) <glitch_pruned_252931>
252932 𒃊 (U+120CA) <glitch_pruned_252932>
252933 𒃍 (U+120CD) <glitch_pruned_252933>
252934 𒃐 (U+120D0) <glitch_pruned_252934>
252935 𒃑 (U+120D1) <glitch_pruned_252935>
252936 𒃓 (U+120D3) <glitch_pruned_252936>
252937 𒃔 (U+120D4) <glitch_pruned_252937>
252938 𒃖 (U+120D6) <glitch_pruned_252938>
252939 𒃗 (U+120D7) <glitch_pruned_252939>
252940 𒃘 (U+120D8) <glitch_pruned_252940>
252941 𒃙 (U+120D9) <glitch_pruned_252941>
252942 𒃛 (U+120DB) <glitch_pruned_252942>
252943 𒃜 (U+120DC) <glitch_pruned_252943>
252944 𒃟 (U+120DF) <glitch_pruned_252944>
252945 𒃦 (U+120E6) <glitch_pruned_252945>
252946 𒃧 (U+120E7) <glitch_pruned_252946>
252947 𒃩 (U+120E9) <glitch_pruned_252947>
252948 𒃪 (U+120EA) <glitch_pruned_252948>
252949 𒃫 (U+120EB) <glitch_pruned_252949>
252950 𒃹 (U+120F9) <glitch_pruned_252950>
252951 𒃿 (U+120FF) <glitch_pruned_252951>
252952 𒄅 (U+12105) <glitch_pruned_252952>
252953 𒄌 (U+1210C) <glitch_pruned_252953>
252954 𒄎 (U+1210E) <glitch_pruned_252954>
252955 𒄓 (U+12113) <glitch_pruned_252955>
252956 𒄟 (U+1211F) <glitch_pruned_252956>
252957 𒄨 (U+12128) <glitch_pruned_252957>
252958 𒄪 (U+1212A) <glitch_pruned_252958>
252959 𒄲 (U+12132) <glitch_pruned_252959>
252960 𒄶 (U+12136) <glitch_pruned_252960>
252962 𒅏 (U+1214F) <glitch_pruned_252962>
252963 𒅝 (U+1215D) <glitch_pruned_252963>
252964 𒅞 (U+1215E) <glitch_pruned_252964>
252965 𒅠 (U+12160) <glitch_pruned_252965>
252966 𒅦 (U+12166) <glitch_pruned_252966>
252967 𒅨 (U+12168) <glitch_pruned_252967>
252968 𒅬 (U+1216C) <glitch_pruned_252968>
252970 𒅰 (U+12170) <glitch_pruned_252970>
252971 𒅽 (U+1217D) <glitch_pruned_252971>
252972 𒆀 (U+12180) <glitch_pruned_252972>
252973 𒆄 (U+12184) <glitch_pruned_252973>
252974 𒆇 (U+12187) <glitch_pruned_252974>
252975 𒆈 (U+12188) <glitch_pruned_252975>
252976 𒆋 (U+1218B) <glitch_pruned_252976>
252977 𒆝 (U+1219D) <glitch_pruned_252977>
252979 𒆡 (U+121A1) <glitch_pruned_252979>
252981 𒆩 (U+121A9) <glitch_pruned_252981>
252982 𒆺 (U+121BA) <glitch_pruned_252982>
252983 𒆻 (U+121BB) <glitch_pruned_252983>
252984 𒆽 (U+121BD) <glitch_pruned_252984>
252985 𒆾 (U+121BE) <glitch_pruned_252985>
252986 𒆿 (U+121BF) <glitch_pruned_252986>
252987 𒇁 (U+121C1) <glitch_pruned_252987>
252988 𒇂 (U+121C2) <glitch_pruned_252988>
252989 𒇃 (U+121C3) <glitch_pruned_252989>
252990 𒇄 (U+121C4) <glitch_pruned_252990>
252991 𒇈 (U+121C8) <glitch_pruned_252991>
252992 𒇊 (U+121CA) <glitch_pruned_252992>
252993 𒇏 (U+121CF) <glitch_pruned_252993>
252994 𒇐 (U+121D0) <glitch_pruned_252994>
252995 𒇓 (U+121D3) <glitch_pruned_252995>
252996 𒇕 (U+121D5) <glitch_pruned_252996>
252997 𒇖 (U+121D6) <glitch_pruned_252997>
252998 𒇗 (U+121D7) <glitch_pruned_252998>
252999 𒇘 (U+121D8) <glitch_pruned_252999>
253000 𒇚 (U+121DA) <glitch_pruned_253000>
253001 𒇛 (U+121DB) <glitch_pruned_253001>
253002 𒇜 (U+121DC) <glitch_pruned_253002>
253003 𒇞 (U+121DE) <glitch_pruned_253003>
253004 𒇟 (U+121DF) <glitch_pruned_253004>
253005 𒇠 (U+121E0) <glitch_pruned_253005>
253006 𒇣 (U+121E3) <glitch_pruned_253006>
253007 𒇤 (U+121E4) <glitch_pruned_253007>
253008 𒇨 (U+121E8) <glitch_pruned_253008>
253009 𒇩 (U+121E9) <glitch_pruned_253009>
253010 𒇪 (U+121EA) <glitch_pruned_253010>
253012 𒇮 (U+121EE) <glitch_pruned_253012>
253013 𒇰 (U+121F0) <glitch_pruned_253013>
253014 𒇼 (U+121FC) <glitch_pruned_253014>
253015 𒇾 (U+121FE) <glitch_pruned_253015>
253016 𒇿 (U+121FF) <glitch_pruned_253016>
253017 𒈂 (U+12202) <glitch_pruned_253017>
253018 𒈃 (U+12203) <glitch_pruned_253018>
253019 𒈄 (U+12204) <glitch_pruned_253019>
253020 𒈅 (U+12205) <glitch_pruned_253020>
253021 𒈈 (U+12208) <glitch_pruned_253021>
253022 𒈊 (U+1220A) <glitch_pruned_253022>
253023 𒈌 (U+1220C) <glitch_pruned_253023>
253024 𒈏 (U+1220F) <glitch_pruned_253024>
253025 𒈐 (U+12210) <glitch_pruned_253025>
253026 𒈑 (U+12211) <glitch_pruned_253026>
253027 𒈟 (U+1221F) <glitch_pruned_253027>
253029 𒉁 (U+12241) <glitch_pruned_253029>
253030 𒉂 (U+12242) <glitch_pruned_253030>
253031 𒉃 (U+12243) <glitch_pruned_253031>
253032 𒉉 (U+12249) <glitch_pruned_253032>
253033 𒉊 (U+1224A) <glitch_pruned_253033>
253035 𒉥 (U+12265) <glitch_pruned_253035>
253036 𒉧 (U+12267) <glitch_pruned_253036>
253037 𒉫 (U+1226B) <glitch_pruned_253037>
253039 𒉱 (U+12271) <glitch_pruned_253039>
253040 𒉲 (U+12272) <glitch_pruned_253040>
253041 𒉳 (U+12273) <glitch_pruned_253041>
253042 𒉴 (U+12274) <glitch_pruned_253042>
253043 𒉵 (U+12275) <glitch_pruned_253043>
253044 𒉶 (U+12276) <glitch_pruned_253044>
253045 𒉸 (U+12278) <glitch_pruned_253045>
253046 𒉹 (U+12279) <glitch_pruned_253046>
253047 𒊊 (U+1228A) <glitch_pruned_253047>
253048 𒊔 (U+12294) <glitch_pruned_253048>
253049 𒊖 (U+12296) <glitch_pruned_253049>
253050 𒊗 (U+12297) <glitch_pruned_253050>
253051 𒊙 (U+12299) <glitch_pruned_253051>
253052 𒊚 (U+1229A) <glitch_pruned_253052>
253053 𒊜 (U+1229C) <glitch_pruned_253053>
253054 𒊞 (U+1229E) <glitch_pruned_253054>
253055 𒊟 (U+1229F) <glitch_pruned_253055>
253056 𒊣 (U+122A3) <glitch_pruned_253056>
253057 𒊤 (U+122A4) <glitch_pruned_253057>
253058 𒊥 (U+122A5) <glitch_pruned_253058>
253059 𒊦 (U+122A6) <glitch_pruned_253059>
253060 𒊪 (U+122AA) <glitch_pruned_253060>
253061 𒊰 (U+122B0) <glitch_pruned_253061>
253062 𒊱 (U+122B1) <glitch_pruned_253062>
253063 𒊲 (U+122B2) <glitch_pruned_253063>
253064 𒊳 (U+122B3) <glitch_pruned_253064>
253065 𒊴 (U+122B4) <glitch_pruned_253065>
253066 𒊵 (U+122B5) <glitch_pruned_253066>
253068 𒋅 (U+122C5) <glitch_pruned_253068>
253069 𒋈 (U+122C8) <glitch_pruned_253069>
253070 𒋉 (U+122C9) <glitch_pruned_253070>
253071 𒋊 (U+122CA) <glitch_pruned_253071>
253072 𒋌 (U+122CC) <glitch_pruned_253072>
253073 𒋍 (U+122CD) <glitch_pruned_253073>
253074 𒋏 (U+122CF) <glitch_pruned_253074>
253082 𒋬 (U+122EC) <glitch_pruned_253082>
253083 𒋭 (U+122ED) <glitch_pruned_253083>
253084 𒋯 (U+122EF) <glitch_pruned_253084>
253085 𒌖 (U+12316) <glitch_pruned_253085>
253086 𒌙 (U+12319) <glitch_pruned_253086>
253087 𒌛 (U+1231B) <glitch_pruned_253087>
253088 𒌞 (U+1231E) <glitch_pruned_253088>
253089 𒌟 (U+1231F) <glitch_pruned_253089>
253090 𒌠 (U+12320) <glitch_pruned_253090>
253091 𒌡 (U+12321) <glitch_pruned_253091>
253094 𒌮 (U+1232E) <glitch_pruned_253094>
253095 𒌯 (U+1232F) <glitch_pruned_253095>
253096 𒌰 (U+12330) <glitch_pruned_253096>
253097 𒌱 (U+12331) <glitch_pruned_253097>
253098 𒌶 (U+12336) <glitch_pruned_253098>
253099 𒌸 (U+12338) <glitch_pruned_253099>
253100 𒌼 (U+1233C) <glitch_pruned_253100>
253101 𒌾 (U+1233E) <glitch_pruned_253101>
253102 𒍁 (U+12341) <glitch_pruned_253102>
253103 𒍉 (U+12349) <glitch_pruned_253103>
253104 𒍋 (U+1234B) <glitch_pruned_253104>
253106 𒍕 (U+12355) <glitch_pruned_253106>
253107 𒍛 (U+1235B) <glitch_pruned_253107>
253110 𒐂 (U+12402) <glitch_pruned_253110>
253111 𒐃 (U+12403) <glitch_pruned_253111>
253112 𒐅 (U+12405) <glitch_pruned_253112>
253113 𒐇 (U+12407) <glitch_pruned_253113>
253116 𒑗 (U+12457) <glitch_pruned_253116>
253117 𒑘 (U+12458) <glitch_pruned_253117>
253118 𒑠 (U+12460) <glitch_pruned_253118>
254551 􀍷 (U+100377) <glitch_pruned_254551>
255586 𒀻 (U+1203B) <glitch_pruned_255586>
255587 𒃝 (U+120DD) <glitch_pruned_255587>
255588 𒃨 (U+120E8) <glitch_pruned_255588>
255589 𒅿 (U+1217F) <glitch_pruned_255589>
255591 𒈚 (U+1221A) <glitch_pruned_255591>
255592 𒋔 (U+122D4) <glitch_pruned_255592>
255593 𒋕 (U+122D5) <glitch_pruned_255593>
255594 𒋘 (U+122D8) <glitch_pruned_255594>
255596 𒋪 (U+122EA) <glitch_pruned_255596>
255597 𒋮 (U+122EE) <glitch_pruned_255597>
255598 𒌃 (U+12303) <glitch_pruned_255598>
255599 𒌄 (U+12304) <glitch_pruned_255599>
255600 𒍓 (U+12353) <glitch_pruned_255600>
255601 𒍟 (U+1235F) <glitch_pruned_255601>
255602 𒐎 (U+1240E) <glitch_pruned_255602>
255603 𒐟 (U+1241F) <glitch_pruned_255603>
255604 𒐠 (U+12420) <glitch_pruned_255604>
255605 𒐡 (U+12421) <glitch_pruned_255605>
255606 𒐢 (U+12422) <glitch_pruned_255606>
255609 𒐥 (U+12425) <glitch_pruned_255609>
255610 𒐦 (U+12426) <glitch_pruned_255610>
255611 𒐧 (U+12427) <glitch_pruned_255611>
255612 𒐨 (U+12428) <glitch_pruned_255612>
255613 𒐩 (U+12429) <glitch_pruned_255613>
255616 𒐬 (U+1242C) <glitch_pruned_255616>
255617 𒐵 (U+12435) <glitch_pruned_255617>
255618 𒐶 (U+12436) <glitch_pruned_255618>
255619 𒐷 (U+12437) <glitch_pruned_255619>
255621 𒐹 (U+12439) <glitch_pruned_255621>
255622 𒐾 (U+1243E) <glitch_pruned_255622>
255623 𒐿 (U+1243F) <glitch_pruned_255623>
255624 𒑓 (U+12453) <glitch_pruned_255624>
255625 𒑕 (U+12455) <glitch_pruned_255625>
255626 𒑝 (U+1245D) <glitch_pruned_255626>
255627 𒑞 (U+1245E) <glitch_pruned_255627>
255628 𒑟 (U+1245F) <glitch_pruned_255628>
255632 𩆩 (U+291A9) <glitch_pruned_255632>

Removed merges (675)

Click to expand the full list of 675 removed merges
["", ""]  →  
[")", "$\--"]  →  )$\--
[")$\", "--"]  →  )$\--
["."", "+"|"]  →  ."+"|
["."+"|", ""+""]  →  ."+"|"+"
[".",", "[("]  →  .",[(
[".", "",[("]  →  .",[(
[".",[", "("]  →  .",[(
["AB", "Tracker"]  →  ABTracker
["AGE", "moji"]  →  AGEmoji
["AG", "Emoji"]  →  AGEmoji
["AWS", "JavaScript"]  →  AWSJavaScript
["Ad", "xRtList"]  →  AdxRtList
["Aile", "Droite"]  →  AileDroite
["Aile", "Gauche"]  →  AileGauche
["AllFilesIn", "MkvDir"]  →  AllFilesInMkvDir
["Ani", "Paf"]  →  AniPaf
["AreaData", "Index"]  →  AreaDataIndex
["Autor", "TextBox"]  →  AutorTextBox
["Avg", "Summer"]  →  AvgSummer
["Backup", "Arch"]  →  BackupArch
["Base", "codes"]  →  Basecodes
["Black", "Sholes"]  →  BlackSholes
["Blitz", "ID"]  →  BlitzID
["CBC", "Pksc"]  →  CBCPksc
["C", "ImgBoard"]  →  CImgBoard
["CYCL", "ONEDB"]  →  CYCLONEDB
["Case", "Missense"]  →  CaseMissense
["Case", "PTV"]  →  CasePTV
["ChooseServer", "Grid"]  →  ChooseServerGrid
["Collin", "ToDO"]  →  CollinToDO
["Consec", "HtIdx"]  →  ConsecHtIdx
["Control", "PTV"]  →  ControlPTV
["Custom", "Glare"]  →  CustomGlare
["CustomGlare", "Def"]  →  CustomGlareDef
["Cxx", "Interop"]  →  CxxInterop
["DT", "MakeRectInCM"]  →  DTMakeRectInCM
["Damage", "Took"]  →  DamageTook
["Days", "GE"]  →  DaysGE
["Denovo", "Mis"]  →  DenovoMis
["DetailUI", "Job"]  →  DetailUIJob
["DetailUIJob", "Icon"]  →  DetailUIJobIcon
["EC", "aut"]  →  ECaut
["E", "Caut"]  →  ECaut
["ELEASE", "STR"]  →  ELEASESTR
["ENEMY", "PLACE"]  →  ENEMYPLACE
["En", "CPP"]  →  EnCPP
["Ephemeral", "Shroud"]  →  EphemeralShroud
["FBSDK", "Bridge"]  →  FBSDKBridge
["FBSDKBridge", "API"]  →  FBSDKBridgeAPI
["FBSDK", "Macros"]  →  FBSDKMacros
["Fcm", "Php"]  →  FcmPhp
["Flex", "str"]  →  Flexstr
["Focus", "Bracketing"]  →  FocusBracketing
["Focus", "Moving"]  →  FocusMoving
["Foldout", "GC"]  →  FoldoutGC
["Func", "ParamNum"]  →  FuncParamNum
["GTBase", "Alert"]  →  GTBaseAlert
["GetChar", "ByIndex"]  →  GetCharByIndex
["Get", "ParaPkg"]  →  GetParaPkg
["GoObject", "AllRef"]  →  GoObjectAllRef
["Go", "PrintError"]  →  GoPrintError
["Go", "RawContext"]  →  GoRawContext
["Go", "SrvGroupIndex"]  →  GoSrvGroupIndex
["HAO", "AVOA"]  →  HAOAVOA
["HDR", "Off"]  →  HDROff
["HD", "ROff"]  →  HDROff
["HDR", "Offsets"]  →  HDROffsets
["HDROff", "sets"]  →  HDROffsets
["HDR", "Processor"]  →  HDRProcessor
["H", "IMQTTRVM"]  →  HIMQTTRVM
["HR", "MGMT"]  →  HRMGMT
["Histogram", "Details"]  →  HistogramDetails
["Hist", "ogramDetails"]  →  HistogramDetails
["Ht", "Idx"]  →  HtIdx
["IM", "QTTR"]  →  IMQTTR
["IMQTTR", "VM"]  →  IMQTTRVM
["Iam", "AuthConfig"]  →  IamAuthConfig
["ImageQueue", "Empty"]  →  ImageQueueEmpty
["Image", "TargetPath"]  →  ImageTargetPath
["Im", "putted"]  →  Imputted
["Is", "ClearedBy"]  →  IsClearedBy
["Jpeg", "Len"]  →  JpegLen
["KAK", "KIA"]  →  KAKKIA
["KAKKIA", "INEN"]  →  KAKKIAINEN
["Lens", "Glare"]  →  LensGlare
["LuaPush", "Int"]  →  LuaPushInt
["Lua", "ToJavaResult"]  →  LuaToJavaResult
["MAPb", "BrI"]  →  MAPbBrI
["MDEw", "Ol"]  →  MDEwOl
["MI", "Tank"]  →  MITank
["MIT", "ank"]  →  MITank
["MainNav", "Show"]  →  MainNavShow
["MakeRect", "InCM"]  →  MakeRectInCM
["Malay", "Marks"]  →  MalayMarks
["Mesh", "VD"]  →  MeshVD
["Metaxy", "onycha"]  →  Metaxyonycha
["Mkv", "Dir"]  →  MkvDir
["Module", "InitFlag"]  →  ModuleInitFlag
["Mogo", "Button"]  →  MogoButton
["MyShop", "name"]  →  MyShopname
["NavBar", "PanelView"]  →  NavBarPanelView
["New", "ParaPkg"]  →  NewParaPkg
["OfDeath", "Condition"]  →  OfDeathCondition
["OrderFlight", "Dto"]  →  OrderFlightDto
["Ordinate", "Tuple"]  →  OrdinateTuple
["PAF", "Struct"]  →  PAFStruct
["PE", "GBuf"]  →  PEGBuf
["PEG", "Buf"]  →  PEGBuf
["P", "QAry"]  →  PQAry
["PQ", "Ary"]  →  PQAry
["PRIM", "ATEC"]  →  PRIMATEC
["P", "Seqlist"]  →  PSeqlist
["Paf", "Handle"]  →  PafHandle
["Para", "Package"]  →  ParaPackage
["Para", "Pkg"]  →  ParaPkg
["Phase", "Hound"]  →  PhaseHound
["Photo", "VideoButton"]  →  PhotoVideoButton
["Pol", "ynucleaires"]  →  Polynucleaires
["Pos", "Delimiter"]  →  PosDelimiter
["Powers", "Controls"]  →  PowersControls
["Price", "Dtls"]  →  PriceDtls
["Print", "NumConst"]  →  PrintNumConst
["Process", "Avg"]  →  ProcessAvg
["RefIn", "SrvGroup"]  →  RefInSrvGroup
["Resources", "Errors"]  →  ResourcesErrors
["Ret", "JpegLen"]  →  RetJpegLen
["Rite", "OfThe"]  →  RiteOfThe
["RxDocument", "Base"]  →  RxDocumentBase
["SAFER", "ELEASESTR"]  →  SAFERELEASESTR
["SAF", "PreferenceKey"]  →  SAFPreferenceKey
["SCE", "corr"]  →  SCEcorr
["SHUNT", "LIST"]  →  SHUNTLIST
["SQL", "form"]  →  SQLform
["SRP", "Basic"]  →  SRPBasic
["SRP", "BinBuf"]  →  SRPBinBuf
["SRP", "Control"]  →  SRPControl
["SRPC", "ontrol"]  →  SRPControl
["SRPC", "ore"]  →  SRPCore
["SRP", "Core"]  →  SRPCore
["SRPCore", "Shell"]  →  SRPCoreShell
["SRP", "Interface"]  →  SRPInterface
["SRPInterface", "Item"]  →  SRPInterfaceItem
["SRP", "Object"]  →  SRPObject
["SRP", "ParaPkg"]  →  SRPParaPkg
["SRPS", "XML"]  →  SRPSXML
["SRP", "SrvGroup"]  →  SRPSrvGroup
["SRPS", "rvGroup"]  →  SRPSrvGroup
["Same", "WordInterval"]  →  SameWordInterval
["Send", "AllResult"]  →  SendAllResult
["Simulator", "Flood"]  →  SimulatorFlood
["Slice", "Temp"]  →  SliceTemp
["SrvGroup", "Index"]  →  SrvGroupIndex
["Star", "BinBuf"]  →  StarBinBuf
["StarBinBuf", "Body"]  →  StarBinBufBody
["StarObject", "Body"]  →  StarObjectBody
["Star", "ParaPkg"]  →  StarParaPkg
["StarParaPkg", "Body"]  →  StarParaPkgBody
["Star", "SXml"]  →  StarSXml
["StarSXml", "Body"]  →  StarSXmlBody
["StarService", "Body"]  →  StarServiceBody
["Star", "SrvGroup"]  →  StarSrvGroup
["StarSrvGroup", "Body"]  →  StarSrvGroupBody
["StructOf", "Object"]  →  StructOfObject
["Sub", "lcd"]  →  Sublcd
["Swift", "lyPlugin"]  →  SwiftlyPlugin
["Syn", "Socket"]  →  SynSocket
["T", "AniPaf"]  →  TAniPaf
["TP", "ASDW"]  →  TPASDW
["Term", "ObjectDefer"]  →  TermObjectDefer
["ToGo", "Result"]  →  ToGoResult
["ToJava", "Result"]  →  ToJavaResult
["TrackAngle", "Cut"]  →  TrackAngleCut
["Track", "TypeValue"]  →  TrackTypeValue
["Transc", "Hist"]  →  TranscHist
["U", "imcoords"]  →  Uimcoords
["Underv", "ault"]  →  Undervault
["Under", "vault"]  →  Undervault
["VS", "FAULT"]  →  VSFAULT
["VSLU", "ATYPE"]  →  VSLUATYPE
["V", "SScript"]  →  VSScript
["VS", "Script"]  →  VSScript
["VST", "YPE"]  →  VSTYPE
["VS", "TYPE"]  →  VSTYPE
["Void", "Hunter"]  →  VoidHunter
["WMN", "CA"]  →  WMNCA
["WM", "NDA"]  →  WMNDA
["WMN", "DA"]  →  WMNDA
["WMN", "MDA"]  →  WMNMDA
["WeakTable", "Mutex"]  →  WeakTableMutex
["WithSize", "InCM"]  →  WithSizeInCM
["Ziel", "Panel"]  →  ZielPanel
["abca", "ti"]  →  abcati
["abc", "ati"]  →  abcati
["ab", "cati"]  →  abcati
["addKill", "Bonus"]  →  addKillBonus
["addKill", "Penalty"]  →  addKillPenalty
["adig", "anam"]  →  adiganam
["adigan", "am"]  →  adiganam
["adig", "anik"]  →  adiganik
["adigan", "ik"]  →  adiganik
["aes", "keygen"]  →  aeskeygen
["ak", "arantadhatu"]  →  akarantadhatu
["alignment", "Good"]  →  alignmentGood
["angolo", "Rad"]  →  angoloRad
["angolo", "Tocco"]  →  angoloTocco
["arantad", "hatu"]  →  arantadhatu
["arant", "adhatu"]  →  arantadhatu
["atthakath", "ayam"]  →  atthakathayam
["attha", "vid"]  →  atthavid
["atth", "avid"]  →  atthavid
["atthavid", "u"]  →  atthavidu
["attup", "adani"]  →  attupadani
["attu", "vasena"]  →  attuvasena
["autor", "Binding"]  →  autorBinding
["b", "NeedConnect"]  →  bNeedConnect
["bac", "anam"]  →  bacanam
["bani", "pi"]  →  banipi
["ban", "ipi"]  →  banipi
["bek", "oru"]  →  bekoru
["block", "idcoin"]  →  blockidcoin
["blusas", "Fem"]  →  blusasFem
["btn", "browser"]  →  btnbrowser
["c", "TempVal"]  →  cTempVal
["cal", "counterx"]  →  calcounterx
["cal", "countery"]  →  calcountery
["calleft", "counter"]  →  calleftcounter
["callow", "counter"]  →  callowcounter
["cal", "radian"]  →  calradian
["calright", "counter"]  →  calrightcounter
["calup", "counter"]  →  calupcounter
["camera", "Continuous"]  →  cameraContinuous
["capture", "cpu"]  →  capturecpu
["cart", "Imp"]  →  cartImp
["cells", "Box"]  →  cellsBox
["chat", "iti"]  →  chatiti
["c", "hatiti"]  →  chatiti
["ch", "atiti"]  →  chatiti
["ch", "ccgi"]  →  chccgi
["check", "katore"]  →  checkkatore
["co", "OrdinateTuple"]  →  coOrdinateTuple
["coef", "double"]  →  coefdouble
["colour", "CodeDict"]  →  colourCodeDict
["comstock", "music"]  →  comstockmusic
["convolve", "Material"]  →  convolveMaterial
["country", "geocode"]  →  countrygeocode
["course", "Survey"]  →  courseSurvey
["custom", "Glare"]  →  customGlare
["cyt", "yle"]  →  cytyle
["cy", "tyle"]  →  cytyle
["datat", "imetable"]  →  datatimetable
["data", "timetable"]  →  datatimetable
["df", "sonic"]  →  dfsonic
["dfs", "onic"]  →  dfsonic
["done", "ProcessAvg"]  →  doneProcessAvg
["draw", "counterx"]  →  drawcounterx
["draw", "countery"]  →  drawcountery
["drawingCode", "hint"]  →  drawingCodehint
["dtd", "software"]  →  dtdsoftware
["dual", "pr"]  →  dualpr
["dw", "RetJpegLen"]  →  dwRetJpegLen
["eco", "expr"]  →  ecoexpr
["eltat", "double"]  →  eltatdouble
["exposure", "Button"]  →  exposureButton
["faulse", "Ans"]  →  faulseAns
["featur", "de"]  →  featurde
["feat", "urde"]  →  featurde
["feder", "icoterzi"]  →  federicoterzi
["fg", "ScrollView"]  →  fgScrollView
["first", "Sending"]  →  firstSending
["foe", "Place"]  →  foePlace
["gdock", "dict"]  →  gdockdict
["getAvg", "SampleSize"]  →  getAvgSampleSize
["getCql", "Left"]  →  getCqlLeft
["getFocus", "Value"]  →  getFocusValue
["get", "FocusValue"]  →  getFocusValue
["get", "HDRProcessor"]  →  getHDRProcessor
["get", "starcore"]  →  getstarcore
["getstarcore", "data"]  →  getstarcoredata
["gmzy", "h"]  →  gmzyh
["grafo", "Existe"]  →  grafoExiste
["hasFocus", "Area"]  →  hasFocusArea
["he", "iboard"]  →  heiboard
["hei", "board"]  →  heiboard
["him", "qttrvm"]  →  himqttrvm
["hiq", "sol"]  →  hiqsol
["hr", "Percent"]  →  hrPercent
["hr", "Volume"]  →  hrVolume
["hr", "assets"]  →  hrassets
["icoter", "zi"]  →  icoterzi
["ineed", "follower"]  →  ineedfollower
["inertia", "Seq"]  →  inertiaSeq
["int", "Fragmentation"]  →  intFragmentation
["invasion", "Mobs"]  →  invasionMobs
["isInt", "fNil"]  →  isIntfNil
["isTrack", "Core"]  →  isTrackCore
["jols", "endev"]  →  jolsendev
["js", "bpmOb"]  →  jsbpmOb
["kit", "opssynth"]  →  kitopssynth
["last", "DamageTook"]  →  lastDamageTook
["lbl", "Integrante"]  →  lblIntegrante
["lens", "Glare"]  →  lensGlare
["local", "instrList"]  →  localinstrList
["login", "MessageNow"]  →  loginMessageNow
["m", "BlitzID"]  →  mBlitzID
["mark", "UpdateChoice"]  →  markUpdateChoice
["match", "StudentNo"]  →  matchStudentNo
["method", "ict"]  →  methodict
["my", "MDP"]  →  myMDP
["next", "LinkedList"]  →  nextLinkedList
["noh", "VP"]  →  nohVP
["og", "Choice"]  →  ogChoice
["opencamer", "astudio"]  →  opencamerastudio
["opencamera", "studio"]  →  opencamerastudio
["opss", "ynth"]  →  opssynth
["ops", "synth"]  →  opssynth
["pJ", "PEGBuf"]  →  pJPEGBuf
["p", "NOMBRE"]  →  pNOMBRE
["pN", "OMBRE"]  →  pNOMBRE
["p", "ResBuffer"]  →  pResBuffer
["paren", "macro"]  →  parenmacro
["partial", "owner"]  →  partialowner
["per", "Dif"]  →  perDif
["player", "WinCounter"]  →  playerWinCounter
["pmm", "Imp"]  →  pmmImp
["pos", "Tocco"]  →  posTocco
["qttr", "vm"]  →  qttrvm
["que", "Cluster"]  →  queCluster
["respArray", "All"]  →  respArrayAll
["ri", "Tracking"]  →  riTracking
["right", "squig"]  →  rightsquig
["sad", "urdu"]  →  sadurdu
["sadurdu", "poetry"]  →  sadurdupoetry
["sal", "cond"]  →  salcond
["sal", "expr"]  →  salexpr
["sbom", "Json"]  →  sbomJson
["segments", "Bar"]  →  segmentsBar
["selectTable", "X"]  →  selectTableX
["selectTable", "Y"]  →  selectTableY
["sendLog", "Payload"]  →  sendLogPayload
["simple", "Triplet"]  →  simpleTriplet
["smo", "io"]  →  smoio
["smoothed", "df"]  →  smootheddf
["smoothed", "series"]  →  smoothedseries
["snake", "CubeSize"]  →  snakeCubeSize
["soc", "Index"]  →  socIndex
["sor", "finaly"]  →  sorfinaly
["squarePos", "Vecchio"]  →  squarePosVecchio
["ssc", "Type"]  →  sscType
["start", "ZielPanel"]  →  startZielPanel
["stato", "Gioco"]  →  statoGioco
["step", "Character"]  →  stepCharacter
["step", "Deploy"]  →  stepDeploy
["sync", "Each"]  →  syncEach
["syscall", "UUID"]  →  syscallUUID
["tcp", "UniqueID"]  →  tcpUniqueID
["testGet", "Popup"]  →  testGetPopup
["test", "HDR"]  →  testHDR
["test", "Panorama"]  →  testPanorama
["test", "TakePhoto"]  →  testTakePhoto
["test", "TakeVideo"]  →  testTakeVideo
["time", "PlusEvents"]  →  timePlusEvents
["tochy", "odikwa"]  →  tochyodikwa
["tool", "tabname"]  →  tooltabname
["total", "BlockFit"]  →  totalBlockFit
["trad", "uitEnCPP"]  →  traduitEnCPP
["tr", "recl"]  →  trrecl
["ts", "recl"]  →  tsrecl
["uit", "EnCPP"]  →  uitEnCPP
["verifica", "Altura"]  →  verificaAltura
["waved", "eltatdouble"]  →  wavedeltatdouble
["widgetTo", "FullName"]  →  widgetToFullName
["wired", "Elems"]  →  wiredElems
["xRt", "List"]  →  xRtList
["ysh", "Exception"]  →  yshException
["जीर", "एक्स"]  →  जीरएक्स
["চ", "ন্্র"]  →  চন্্র
["চন", "্্র"]  →  চন্্র
["ন্", "্র"]  →  ন্্র
["ন", "্্র"]  →  ন্্র
["িয়েত", "নাম"]  →  িয়েতনাম
["িয়ে", "তনাম"]  →  িয়েতনাম
["ৃ", "তরাং"]  →  ৃতরাং
["ৃত", "রাং"]  →  ৃতরাং
["தல", "மோங்கு"]  →  தலமோங்கு
["ய்ய", "மணி"]  →  ய்யமணி
["ఋతం", "భర"]  →  ఋతంభర
["వెట్‌", "స్కీ"]  →  వెట్‌స్కీ
["▁+#", "+#+#"]  →  ▁+#+#+#
["▁.+", ""]),"]  →  ▁.+"]),
["▁", "::::::::"]  →  ▁::::::::
["▁::", "::::::"]  →  ▁::::::::
["▁A", "fdPar"]  →  ▁AfdPar
["▁Afd", "Par"]  →  ▁AfdPar
["▁Archers", "Unit"]  →  ▁ArchersUnit
["▁Atr", "ayam"]  →  ▁Atrayam
["▁Atra", "yam"]  →  ▁Atrayam
["▁BVHTree", "Node"]  →  ▁BVHTreeNode
["▁Beer", "Obj"]  →  ▁BeerObj
["▁C", "AdxRtList"]  →  ▁CAdxRtList
["▁CC", "BUNDLE"]  →  ▁CCBUNDLE
["▁CC", "Bundle"]  →  ▁CCBundle
["▁Control", "Missense"]  →  ▁ControlMissense
["▁Control", "PTV"]  →  ▁ControlPTV
["▁", "ControlPTV"]  →  ▁ControlPTV
["▁DT", "MakeRect"]  →  ▁DTMakeRect
["▁Dh", "wast"]  →  ▁Dhwast
["▁Dummy", "Computed"]  →  ▁DummyComputed
["▁FROM", "VS"]  →  ▁FROMVS
["▁Func", "ParamNum"]  →  ▁FuncParamNum
["▁", "FuncParamNum"]  →  ▁FuncParamNum
["▁GoObject", "To"]  →  ▁GoObjectTo
["▁", "GoRawContext"]  →  ▁GoRawContext
["▁Go", "RawContext"]  →  ▁GoRawContext
["▁Go", "SRP"]  →  ▁GoSRP
["▁Go", "SrvGroupIndex"]  →  ▁GoSrvGroupIndex
["▁", "GoSrvGroupIndex"]  →  ▁GoSrvGroupIndex
["▁H", "IMQTTRVM"]  →  ▁HIMQTTRVM
["▁", "HIMQTTRVM"]  →  ▁HIMQTTRVM
["▁HRM", "Client"]  →  ▁HRMClient
["▁Hack", "Search"]  →  ▁HackSearch
["▁Hb", "dm"]  →  ▁Hbdm
["▁H", "bdm"]  →  ▁Hbdm
["▁I", "HAOAVOA"]  →  ▁IHAOAVOA
["▁", "Imputted"]  →  ▁Imputted
["▁Im", "putted"]  →  ▁Imputted
["▁I", "octrl"]  →  ▁Ioctrl
["▁Io", "ctrl"]  →  ▁Ioctrl
["▁LG", "AGEmoji"]  →  ▁LGAGEmoji
["▁Lua", "ToGoObject"]  →  ▁LuaToGoObject
["▁", "LuaToJavaResult"]  →  ▁LuaToJavaResult
["▁Lua", "ToJavaResult"]  →  ▁LuaToJavaResult
["▁M", "Bine"]  →  ▁MBine
["▁MB", "ine"]  →  ▁MBine
["▁ND", "IndexArray"]  →  ▁NDIndexArray
["▁NS", "amps"]  →  ▁NSamps
["▁", "NewParaPkg"]  →  ▁NewParaPkg
["▁New", "ParaPkg"]  →  ▁NewParaPkg
["▁ODDS", "AR"]  →  ▁ODDSAR
["▁PL", "DNN"]  →  ▁PLDNN
["▁P", "QAry"]  →  ▁PQAry
["▁", "PQAry"]  →  ▁PQAry
["▁PQ", "Ary"]  →  ▁PQAry
["▁Paf", "Handle"]  →  ▁PafHandle
["▁", "PafHandle"]  →  ▁PafHandle
["▁Pays", "yses"]  →  ▁Paysyses
["▁", "Polynucleaires"]  →  ▁Polynucleaires
["▁Pol", "ynucleaires"]  →  ▁Polynucleaires
["▁RUTARE", "AL"]  →  ▁RUTAREAL
["▁RUTARE", "L"]  →  ▁RUTAREL
["▁RUTAREL", "ATIV"]  →  ▁RUTARELATIV
["▁Ref", "ToGoObject"]  →  ▁RefToGoObject
["▁SIINFEKL", "C"]  →  ▁SIINFEKLC
["▁SIINFE", "KLC"]  →  ▁SIINFEKLC
["▁SRP", "Go"]  →  ▁SRPGo
["▁SRPGo", "Get"]  →  ▁SRPGoGet
["▁SRPGoGet", "Str"]  →  ▁SRPGoGetStr
["▁SRPGo", "SetStr"]  →  ▁SRPGoSetStr
["▁SRP", "Interface"]  →  ▁SRPInterface
["▁", "SRPInterface"]  →  ▁SRPInterface
["▁", "SRPObject"]  →  ▁SRPObject
["▁SRP", "Object"]  →  ▁SRPObject
["▁Smart", "yLint"]  →  ▁SmartyLint
["▁Spring", "ObjectID"]  →  ▁SpringObjectID
["▁SrvGroup", "Class"]  →  ▁SrvGroupClass
["▁Star", "ParaPkg"]  →  ▁StarParaPkg
["▁", "StarParaPkg"]  →  ▁StarParaPkg
["▁", "StarSXml"]  →  ▁StarSXml
["▁Star", "SXml"]  →  ▁StarSXml
["▁StarSXml", "Class"]  →  ▁StarSXmlClass
["▁TCP", "Assignment"]  →  ▁TCPAssignment
["▁TC", "PAssignment"]  →  ▁TCPAssignment
["▁TP", "ASDW"]  →  ▁TPASDW
["▁", "TPASDW"]  →  ▁TPASDW
["▁Term", "ObjectDefer"]  →  ▁TermObjectDefer
["▁", "TermObjectDefer"]  →  ▁TermObjectDefer
["▁TestAvg", "Callback"]  →  ▁TestAvgCallback
["▁Tiki", "Man"]  →  ▁TikiMan
["▁V", "uttab"]  →  ▁Vuttab
["▁YY", "YY"]  →  ▁YYYY
["▁", "YYYY"]  →  ▁YYYY
["▁Y", "fine"]  →  ▁Yfine
["▁aby", "att"]  →  ▁abyatt
["▁add", "ConfigureArg"]  →  ▁addConfigureArg
["▁add", "SBOM"]  →  ▁addSBOM
["▁ak", "ammak"]  →  ▁akammak
["▁allClasses", "Link"]  →  ▁allClassesLink
["▁", "angoloRad"]  →  ▁angoloRad
["▁angolo", "Rad"]  →  ▁angoloRad
["▁app", "asidd"]  →  ▁appasidd
["▁ativar", "botao"]  →  ▁ativarbotao
["▁atthakath", "ac"]  →  ▁atthakathac
["▁atth", "udd"]  →  ▁atthudd
["▁b", "NeedConnect"]  →  ▁bNeedConnect
["▁", "bNeedConnect"]  →  ▁bNeedConnect
["▁bhuv", "adigane"]  →  ▁bhuvadigane
["▁bingo", "CardArray"]  →  ▁bingoCardArray
["▁browsing", "Stamp"]  →  ▁browsingStamp
["▁burn", "Area"]  →  ▁burnArea
["▁check", "HDROffsets"]  →  ▁checkHDROffsets
["▁coi", "Alarm"]  →  ▁coiAlarm
["▁cre", "aterData"]  →  ▁createrData
["▁", "cytyle"]  →  ▁cytyle
["▁cy", "tyle"]  →  ▁cytyle
["▁cyt", "yle"]  →  ▁cytyle
["▁dSample", "Height"]  →  ▁dSampleHeight
["▁dSample", "Width"]  →  ▁dSampleWidth
["▁d", "assanato"]  →  ▁dassanato
["▁dass", "anato"]  →  ▁dassanato
["▁deleted", "Drinks"]  →  ▁deletedDrinks
["▁dhatu", "nam"]  →  ▁dhatunam
["▁dhat", "unam"]  →  ▁dhatunam
["▁dhatu", "ssa"]  →  ▁dhatussa
["▁dhat", "ussa"]  →  ▁dhatussa
["▁diff", "formul"]  →  ▁diffformul
["▁diff", "play"]  →  ▁diffplay
["▁dir", "BL"]  →  ▁dirBL
["▁dir", "BR"]  →  ▁dirBR
["▁dir", "FL"]  →  ▁dirFL
["▁dir", "FR"]  →  ▁dirFR
["▁ditt", "iyam"]  →  ▁dittiyam
["▁", "doneProcessAvg"]  →  ▁doneProcessAvg
["▁done", "ProcessAvg"]  →  ▁doneProcessAvg
["▁eco", "expr"]  →  ▁ecoexpr
["▁", "ecoexpr"]  →  ▁ecoexpr
["▁ete", "nati"]  →  ▁etenati
["▁eten", "ati"]  →  ▁etenati
["▁ettha", "ti"]  →  ▁etthati
["▁ett", "hati"]  →  ▁etthati
["▁et", "thati"]  →  ▁etthati
["▁evam", "adisu"]  →  ▁evamadisu
["▁", "exposureButton"]  →  ▁exposureButton
["▁exposure", "Button"]  →  ▁exposureButton
["▁flower", "Total"]  →  ▁flowerTotal
["▁gacchati", "ti"]  →  ▁gacchatiti
["▁g", "acchatiti"]  →  ▁gacchatiti
["▁gacch", "atiti"]  →  ▁gacchatiti
["▁gac", "chatiti"]  →  ▁gacchatiti
["▁gat", "iyam"]  →  ▁gatiyam
["▁gati", "yam"]  →  ▁gatiyam
["▁generic", "Overview"]  →  ▁genericOverview
["▁hetuk", "att"]  →  ▁hetukatt
["▁hims", "ayam"]  →  ▁himsayam
["▁icc", "adini"]  →  ▁iccadini
["▁iccad", "ini"]  →  ▁iccadini
["▁icc", "api"]  →  ▁iccapi
["▁ic", "capi"]  →  ▁iccapi
["▁inner", "WallArray"]  →  ▁innerWallArray
["▁jaû", "nes"]  →  ▁jaûnes
["▁jinfo", "Home"]  →  ▁jinfoHome
["▁jinfo", "Next"]  →  ▁jinfoNext
["▁jobSearch", "Repo"]  →  ▁jobSearchRepo
["▁kapp", "agen"]  →  ▁kappagen
["▁kap", "pagen"]  →  ▁kappagen
["▁kappa", "gen"]  →  ▁kappagen
["▁kib", "ci"]  →  ▁kibci
["▁lg", "PlatformImage"]  →  ▁lgPlatformImage
["▁life", "Guy"]  →  ▁lifeGuy
["▁m", "SwisTrackCore"]  →  ▁mSwisTrackCore
["▁magick", "woods"]  →  ▁magickwoods
["▁", "matchStudentNo"]  →  ▁matchStudentNo
["▁match", "StudentNo"]  →  ▁matchStudentNo
["▁max", "Goto"]  →  ▁maxGoto
["▁mdl", "MeshVD"]  →  ▁mdlMeshVD
["▁min", "Goto"]  →  ▁minGoto
["▁morphOutput", "Comp"]  →  ▁morphOutputComp
["▁neighbor", "Indexs"]  →  ▁neighborIndexs
["▁newInput", "Btn"]  →  ▁newInputBtn
["▁nibb", "acan"]  →  ▁nibbacan
["▁nibbacan", "am"]  →  ▁nibbacanam
["▁nib", "bacanam"]  →  ▁nibbacanam
["▁nid", "assanam"]  →  ▁nidassanam
["▁nigg", "ahit"]  →  ▁niggahit
["▁nigga", "hit"]  →  ▁niggahit
["▁", "nohVP"]  →  ▁nohVP
["▁noh", "VP"]  →  ▁nohVP
["▁o", "LetterLocation"]  →  ▁oLetterLocation
["▁o", "Locations"]  →  ▁oLocations
["▁", "oLocations"]  →  ▁oLocations
["▁padam", "ala"]  →  ▁padamala
["▁pada", "mala"]  →  ▁padamala
["▁pali", "yam"]  →  ▁paliyam
["▁pal", "iyam"]  →  ▁paliyam
["▁pay", "ogo"]  →  ▁payogo
["▁payog", "o"]  →  ▁payogo
["▁pid", "Motor"]  →  ▁pidMotor
["▁plotting", "Area"]  →  ▁plottingArea
["▁process", "PerRow"]  →  ▁processPerRow
["▁product", "Moles"]  →  ▁productMoles
["▁rud", "had"]  →  ▁rudhad
["▁saka", "mm"]  →  ▁sakamm
["▁sak", "amm"]  →  ▁sakamm
["▁sak", "ammak"]  →  ▁sakammak
["▁sakamm", "ak"]  →  ▁sakammak
["▁sayam", "eva"]  →  ▁sayameva
["▁say", "ameva"]  →  ▁sayameva
["▁sc", "StudentVector"]  →  ▁scStudentVector
["▁sdx", "Concept"]  →  ▁sdxConcept
["▁set", "Penis"]  →  ▁setPenis
["▁smoothed", "series"]  →  ▁smoothedseries
["▁", "smoothedseries"]  →  ▁smoothedseries
["▁", "snakeCubeSize"]  →  ▁snakeCubeSize
["▁snake", "CubeSize"]  →  ▁snakeCubeSize
["▁", "socIndex"]  →  ▁socIndex
["▁soc", "Index"]  →  ▁socIndex
["▁student", "LVector"]  →  ▁studentLVector
["▁subTest", "Avg"]  →  ▁subTestAvg
["▁subTest", "HDR"]  →  ▁subTestHDR
["▁subTest", "Panorama"]  →  ▁subTestPanorama
["▁suddhak", "att"]  →  ▁suddhakatt
["▁test", "HDR"]  →  ▁testHDR
["▁", "testHDR"]  →  ▁testHDR
["▁", "testPanorama"]  →  ▁testPanorama
["▁test", "Panorama"]  →  ▁testPanorama
["▁", "testTakeVideo"]  →  ▁testTakeVideo
["▁test", "TakeVideo"]  →  ▁testTakeVideo
["▁than", "esu"]  →  ▁thanesu
["▁th", "anesu"]  →  ▁thanesu
["▁total", "BlockUsed"]  →  ▁totalBlockUsed
["▁totalBlockUsed", "A"]  →  ▁totalBlockUsedA
["▁tp", "Const"]  →  ▁tpConst
["▁", "trrecl"]  →  ▁trrecl
["▁tr", "recl"]  →  ▁trrecl
["▁ts", "recl"]  →  ▁tsrecl
["▁", "tsrecl"]  →  ▁tsrecl
["▁ven", "uedata"]  →  ▁venuedata
["▁venue", "data"]  →  ▁venuedata
["▁vero", "nya"]  →  ▁veronya
["▁ver", "onya"]  →  ▁veronya
["▁vuccati", "ti"]  →  ▁vuccatiti
["▁vucc", "atiti"]  →  ▁vuccatiti
["▁wave", "formul"]  →  ▁waveformul
["▁waveform", "ul"]  →  ▁waveformul
["▁yath", "araham"]  →  ▁yatharaham
["▁yatha", "raham"]  →  ▁yatharaham
["▁yath", "asambhavam"]  →  ▁yathasambhavam
["▁yojet", "abbam"]  →  ▁yojetabbam
["▁yojet", "abbani"]  →  ▁yojetabbani
["▁yy", "yy"]  →  ▁yyyy
["▁", "yyyy"]  →  ▁yyyy
["▁Гинд", "зё"]  →  ▁Гиндзё
["▁शब्द", "लय"]  →  ▁शब्दलय
["▁অচ", "িতের"]  →  ▁অচিতের
["▁অ", "চিতের"]  →  ▁অচিতের
["▁অন্", "ু"]  →  ▁অন্ু
["▁অন", "্ু"]  →  ▁অন্ু
["▁অব", "শ্ত"]  →  ▁অবশ্ত
["▁আর", "ম্ত"]  →  ▁আরম্ত
["▁খু", "ষ্ট"]  →  ▁খুষ্ট
["▁খ", "ুষ্ট"]  →  ▁খুষ্ট
["▁চিদা", "ভ"]  →  ▁চিদাভ
["▁চিদা", "ভাসের"]  →  ▁চিদাভাসের
["▁চিদাভাস", "ের"]  →  ▁চিদাভাসের
["▁চিদাভ", "াসের"]  →  ▁চিদাভাসের
["▁ছু", "ঃ"]  →  ▁ছুঃ
["▁জ্ঞাত", "ত্ব"]  →  ▁জ্ঞাতত্ব
["▁ন্যায়", "রত্ব"]  →  ▁ন্যায়রত্ব
["▁বোপ", "দেবের"]  →  ▁বোপদেবের
["▁বোপদেব", "ের"]  →  ▁বোপদেবের
["▁ভোগ", "্যের"]  →  ▁ভোগ্যের
["▁ভোগ্য", "ের"]  →  ▁ভোগ্যের
["▁শরনার্থ", "ি"]  →  ▁শরনার্থি
["▁শরনার্থি", "দের"]  →  ▁শরনার্থিদের
["▁শরনার্থ", "িদের"]  →  ▁শরনার্থিদের
["▁স্থ", "তরাং"]  →  ▁স্থতরাং
["▁வால்", "பே"]  →  ▁வால்பே
["▁బ్లా", "వెట్‌స్కీ"]  →  ▁బ్లావెట్‌స్కీ
["▁ഹ", "ോളിക"]  →  ▁ഹോളിക
["▁ഹോളി", "ക"]  →  ▁ഹോളിക
["▁⏮", "","]  →  ▁⏮",
["▁加入", "层"]  →  ▁加入层
["加入", "参数向量中"]  →  加入参数向量中
["感謝", "石頭"]  →  感謝石頭
["渦", "柱"]  →  渦柱
["변경된", "파일명"]  →  변경된파일명

Files changed

  • tokenizer.json — BPE merge rules pruned, 522 vocab entries renamed

Algorithm Details

For full details on the glitch token collection algorithm and remediation techniques,
reach out to gkielian.

Related

This is part of a series of tokenizer fixes across all Gemma model repositories
(Gemma 1, 2, 3, 4, and MedGemma — 18 models total).

cc @dougreid @ssmoot

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment