Raw outputs from evaluating recent LLMs on li-lab/MMLU-ProX-Lite (revision e82aafb) with lm-evaluation-harness 0.4.13, in 10 languages (en, zh, ja, ko, pt, de, id, ru, ar, sw). The purpose is within-model diagnosis: for each model, which languages (relative to its own English accuracy on the same parallel questions) and which subjects it is weak in. It is not a leaderboard, and models are not meant to be compared with each other: reasoning effort differs by model (the developer's default for most of them; see the GitHub README). Code, protocol, model settings and summary tables: One folder per model, in the layout lm-evaluation-harness writes: - results.json: accuracy per language and…
Independent publisher
3ae3ae
3ae3ae3ae
Models in Library0
Datasets in Library1
Models on Hugging Face—
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Question answering.
Reported licenses: mit. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.