sudani_lm in progress
A language model for Sudanese Arabic, trained from scratch. Trains its own tokenizer, pretrains a decoder-only transformer (4 layers, 512-dim, 8 heads, 1k context) on Arabic text, then finetunes on Sudanese dialect chat. Config-driven runs with pluggable model and dataset classes, MMLU checks and temperature-swept generation samples during training.