ModernBERT pre-trained from scratch on various obfuscated representations of the same 10B tokens.
🔄 In a Training Loop
Daniel Gallagher
DanielGallagherIRE
AI & ML interests
I like encoders because they don't talk back.
Recent Activity
updated a model about 16 hours ago
crawka/Crawka-Mixed published a model about 16 hours ago
crawka/Crawka-Mixed liked a model 3 days ago
zai-org/GLM-5.3-FlashOrganizations
A collection of obfuscated version of a 20B-token sample of the FineWeb Edu dataset.
-
DanielGallagherIRE/FineWeb-Edu-10B-Obfuscation
Viewer • Updated • 19.4M • 761 -
DanielGallagherIRE/FineWeb-Edu-10B-Nouns-Only
Viewer • Updated • 9.71M • 2.94k -
DanielGallagherIRE/FineWeb-Edu-10B-Shuffled
Viewer • Updated • 9.68M • 814 -
DanielGallagherIRE/FineWeb-Edu-10B-PMI-Filtered
Viewer • Updated • 9.71M • 364
ModernBERT pre-trained from scratch on various obfuscated representations of the same 10B tokens.
A collection of obfuscated version of a 20B-token sample of the FineWeb Edu dataset.
-
DanielGallagherIRE/FineWeb-Edu-10B-Obfuscation
Viewer • Updated • 19.4M • 761 -
DanielGallagherIRE/FineWeb-Edu-10B-Nouns-Only
Viewer • Updated • 9.71M • 2.94k -
DanielGallagherIRE/FineWeb-Edu-10B-Shuffled
Viewer • Updated • 9.68M • 814 -
DanielGallagherIRE/FineWeb-Edu-10B-PMI-Filtered
Viewer • Updated • 9.71M • 364
models 42
DanielGallagherIRE/ModernBERT-O3-Shuffling-SENT
0.2B • Updated • 10
DanielGallagherIRE/ModernBERT-O3-Shuffling-DOC
0.2B • Updated • 12
DanielGallagherIRE/ModernBERT-O3-Baseline
0.2B • Updated • 62
DanielGallagherIRE/ModernBERT-O3-PMI-5
0.2B • Updated • 4
DanielGallagherIRE/ModernBERT-O3-PMI-2.5
0.2B • Updated • 5
DanielGallagherIRE/ModernBERT-O3-PMI-10
0.2B • Updated • 10
DanielGallagherIRE/ModernBERT-O3-Nouns-POS
0.2B • Updated • 15
DanielGallagherIRE/ModernBERT-O3-PMI-7.5
0.2B • Updated • 11
DanielGallagherIRE/ModernBERT-O3-PMI-12.5
0.2B • Updated • 11
DanielGallagherIRE/ModernBERT-O3-Nouns-DEL
0.2B • Updated • 9
datasets 23
DanielGallagherIRE/FineWeb-Edu-Irish
Viewer • Updated • 700k • 52
DanielGallagherIRE/FineWeb-Edu-10B-Shuffled-DOC-Truncated
Viewer • Updated • 11.3M • 218
DanielGallagherIRE/FineWeb-Edu-10B-Shuffled-SENT-Truncated
Viewer • Updated • 11.3M • 532
DanielGallagherIRE/FineWeb-Edu-10B-Obfuscation-Truncated
Viewer • Updated • 11.3M • 404
DanielGallagherIRE/FineWeb-Edu-10B-PMI-Filtered-Delete
Viewer • Updated • 9.71M • 318
DanielGallagherIRE/FineWeb-Edu-20B-E2-OBF-FULL
Viewer • Updated • 9.69M • 169
DanielGallagherIRE/FineWeb-Edu-20B-E2-v3-Obfuscated
Viewer • Updated • 9.69M • 168
DanielGallagherIRE/FineWeb-Edu-20B-E2-v3
Viewer • Updated • 9.69M • 401
DanielGallagherIRE/FineWeb-Edu-20B-E2-v2
Updated • 5
DanielGallagherIRE/FineWeb-Edu-20B-E2
Updated • 5