Rows—
Configurations—
Size100.3 MB
License—
AccessPublicly accessible
Monthly Downloads1.6k
Dataset Card
The publisher has not written a card for this dataset.
Details
- Repository
- hr1018/tokencal-results
- Publisher
- HABIB
- Task category
- Not stated by the source
- Tags
- Not stated by the source
- Size category
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 284a67b5f5520694a67f09c3ea423226fc0386f7
- Last updated
- 2026-10-04
Files
419 files, 100.3 MB in total.
Data387 files · 89.7 MB
Tokenizer5 files · 5.0 KB
Other26 files · 10.6 MB
Repository1 file · 2.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| ablations/base_m0_m1_best_test_metrics.csv | Data | 1.7 KB | — |
| ablations/base_m0_m1_m2_best_test_metrics.csv | Data | 2.5 KB | — |
| ablations/base_m0_m1_m2_m3_best_test_metrics.csv | Data | 3.3 KB | — |
| ablations/m0_best_vs_base_test_instability.csv | Data | 327 B | — |
| ablations/m0_best_vs_base_test_metrics.csv | Data | 978 B | — |
| ablations/m0_m1_m2_best_test_instability.csv | Data | 331 B | — |
| ablations/m0_m1_m2_m3_best_test_instability.csv | Data | 405 B | — |
| ablations/m0_standard_sft_training_summary.csv | Data | 795 B | — |
| ablations/m0_vs_m1_best_test_instability.csv | Data | 386 B | — |
| ablations/m0_vs_m1_best_test_metrics.csv | Data | 996 B | — |
| ablations/m0_vs_m2_best_test_metrics.csv | Data | 991 B | — |
| ablations/m0_vs_m3_best_test_metrics.csv | Data | 989 B | — |
| ablations/m1_calibration_training_summary.csv | Data | 837 B | — |
| ablations/m1_vs_m2_best_test_metrics.csv | Data | 977 B | — |
| ablations/m1_vs_m3_best_test_metrics.csv | Data | 988 B | — |
| ablations/m2_pair_calibration_training_summary.csv | Data | 846 B | — |
| ablations/m2_vs_m3_best_test_metrics.csv | Data | 984 B | — |
| ablations/m3_tokencal_training_summary.csv | Data | 848 B | — |
| baseline/base_accuracy_pattern_summary.csv | Data | 192 B | — |
| baseline/base_full_inference_gpu.csv | Data | 235 B | — |
| baseline/base_full_inference_runtime.csv | Data | 320 B | — |
| baseline/base_full_predictions_long.csv | Data | 3.5 MB | — |
| baseline/base_model_peak_vram.csv | Data | 112 B | — |
| baseline/base_model_runtime.csv | Data | 579 B | — |
| baseline/base_model_vram_audit.csv | Data | 680 B | — |
| baseline/base_pairwise_predictions.csv | Data | 4.4 MB | — |
| baseline/base_quick_summary.csv | Data | 364 B | — |
| baseline/base_split_language_summary.csv | Data | 767 B | — |
| baseline/choice_logit_sanity.csv | Data | 549 B | — |
| baseline/dual_gpu_inference_runtime.csv | Data | 311 B | — |
| baseline/inference_batch_benchmark.csv | Data | 3.2 KB | — |
| baseline/inference_batch_policy.csv | Data | 259 B | — |
| baseline/optimized_choice_scoring_equivalence.csv | Data | 343 B | — |
| config/ablation_loss_coefficients.csv | Data | 578 B | — |
| config/ablation_objective_tensor_contract.csv | Data | 567 B | — |
| config/chat_template_count_correction.csv | Data | 314 B | — |
| config/checkpoint_resume_policy.csv | Data | 554 B | — |
| config/choice_token_mapping.csv | Data | 699 B | — |
| config/context_length_policy.csv | Data | 432 B | — |
| config/dependency_lock.csv | Data | 540 B | — |
| config/full_training_schedule.csv | Data | 404 B | — |
| config/learning_rate_schedule_audit.csv | Data | 357 B | — |
| config/length_aware_pair_sampler_policy.csv | Data | 393 B | — |
| config/lora_config.json | Data | 513 B | — |
| config/master_config.json | Data | 9.2 KB | — |
| config/optimizer_scheduler_policy.csv | Data | 369 B | — |
| config/pair_aware_collator_policy.csv | Data | 364 B | — |
| config/precision_policy.csv | Data | 366 B | — |
| config/qlora_training_policy.csv | Data | 640 B | — |
| config/runtime_hardware.csv | Data | 772 B | — |
| config/runtime_software.csv | Data | 394 B | — |
| config/training_batch_tensor_contract.csv | Data | 1.4 KB | — |
| config/training_environment.json | Data | 595 B | — |
| config/training_fragmentation_normalization.csv | Data | 216 B | — |
| config/training_loss_definitions.csv | Data | 507 B | — |
| config/training_supervision_policy.csv | Data | 484 B | — |
| datasets/b_reaso_benqa_exact_overlap_audit.csv | Data | 1.3 MB | — |
| datasets/b_reaso_benqa_exact_overlap_excluded_rows.csv | Data | 292 B | — |
| datasets/b_reaso_benqa_question_only_overlap_audit.csv | Data | 292 B | — |
| datasets/b_reaso_val_benqa_overlap_audit.csv | Data | 150.4 KB | — |
| datasets/b_reaso_val_conflicting_duplicate_rows.csv | Data | 2.9 KB | — |
| datasets/b_reaso_val_ood_audit_summary.json | Data | 1.1 KB | — |
| datasets/b_reaso_val_ood_canonical.csv | Data | 1.3 MB | — |
| datasets/b_reaso_val_ood_label_counts.csv | Data | 43 B | — |
| datasets/b_reaso_val_ood_subject_counts.csv | Data | 1.1 KB | — |
| datasets/b_reaso_val_question_conflict_audit.csv | Data | 676 B | — |
| datasets/b_reaso_val_source_audit.csv | Data | 2.6 KB | — |
| datasets/b_reaso_val_source_invalid_rows.csv | Data | 16.2 KB | — |
| datasets/benqa_answer_distribution.csv | Data | 159 B | — |
| datasets/benqa_clean_pair_metadata.csv | Data | 719.5 KB | — |
| datasets/benqa_cleaning_flow.csv | Data | 150 B | — |
| datasets/benqa_csv_inventory.csv | Data | 11.7 KB | — |
| datasets/benqa_duplicate_audit_summary.csv | Data | 205 B | — |
| datasets/benqa_file_inventory.csv | Data | 11.8 KB | — |
| datasets/benqa_final_answer_distribution.csv | Data | 134 B | — |
| datasets/benqa_final_population_summary.csv | Data | 546 B | — |
| datasets/benqa_final_subject_distribution.csv | Data | 648 B | — |
| datasets/benqa_group_split_audit.csv | Data | 225.1 KB | — |
| datasets/benqa_main_schema_validation.csv | Data | 1.8 KB | — |
| datasets/benqa_pair_integrity_summary.csv | Data | 384 B | — |
| datasets/benqa_pair_metadata.csv | Data | 480.8 KB | — |
| datasets/benqa_question_group_audit.csv | Data | 313.3 KB | — |
| datasets/benqa_recovered_bengali_content_overlap_audit.csv | Data | 2.5 MB | — |
| datasets/benqa_source_provenance.csv | Data | 784 B | — |
| datasets/benqa_split_answer_distribution.csv | Data | 481 B | — |
| datasets/benqa_split_assignments.csv | Data | 626.1 KB | — |
| datasets/benqa_split_candidate_search.csv | Data | 472.9 KB | — |
| datasets/benqa_split_distribution.csv | Data | 4.0 KB | — |
| datasets/benqa_split_distribution_tvd.csv | Data | 501 B | — |
| datasets/benqa_split_leakage_audit.csv | Data | 255 B | — |
| datasets/benqa_split_sizes.csv | Data | 344 B | — |
| datasets/benqa_subject_statistics.csv | Data | 801 B | — |
| datasets/milu_duplicate_audit.csv | Data | 202 B | — |
| datasets/milu_duplicate_resolution_policy.csv | Data | 181 B | — |
| datasets/milu_exact_duplicate_excluded_rows.csv | Data | 317 B | — |
| datasets/milu_final_test_language_counts.csv | Data | 99 B | — |
| datasets/milu_final_test_lock_summary.json | Data | 797 B | — |
| datasets/milu_load_audit.csv | Data | 1.0 KB | — |
| datasets/milu_multilingual_audit_summary.json | Data | 1.7 KB | — |
| datasets/milu_schema_audit.csv | Data | 3.2 KB | — |
| datasets/milu_source_invalid_rows_audit.csv | Data | 89 B | — |
| datasets/milu_structural_validity_audit.csv | Data | 309 B | — |
| datasets/milu_test_language_counts.csv | Data | 172 B | — |
| datasets/pair_aware_dataset_summary.csv | Data | 140 B | — |
| datasets/shroom_cap_label_availability_audit.csv | Data | 639 B | — |
| datasets/shroom_cap_language_counts.csv | Data | 293 B | — |
| datasets/shroom_cap_low_resource_suitability_audit.csv | Data | 236 B | — |
| datasets/shroom_cap_multilingual_audit_summary.json | Data | 1.4 KB | — |
| datasets/shroom_cap_schema_audit.csv | Data | 1.8 KB | — |
| datasets/shroom_cap_source_inventory.csv | Data | 318 B | — |
| datasets/training_pair_length_summary.csv | Data | 233 B | — |
| datasets/training_pair_metadata.csv | Data | 479.3 KB | — |
| datasets/training_supervision_summary.csv | Data | 394 B | — |
| datasets/training_target_distribution.csv | Data | 344 B | — |
| evaluations/m0_best_test_language_gap.csv | Data | 921 B | — |
| evaluations/m0_best_test_language_metrics.csv | Data | 826 B | — |
| evaluations/m0_best_test_loss_summary.csv | Data | 362 B | — |
| evaluations/m0_best_test_predictions.csv | Data | 308.7 KB | — |
| evaluations/m0_standard_sft_validation_epochs.csv | Data | 1.6 KB | — |
| evaluations/m1_best_test_language_gap.csv | Data | 992 B | — |
| evaluations/m1_best_test_language_metrics.csv | Data | 822 B | — |
| evaluations/m1_best_test_loss_summary.csv | Data | 355 B | — |
| evaluations/m1_best_test_predictions.csv | Data | 245.2 KB | — |
| evaluations/m1_calibration_validation_epochs.csv | Data | 1.6 KB | — |
| evaluations/m2_best_test_language_gap.csv | Data | 995 B | — |
| evaluations/m2_best_test_language_metrics.csv | Data | 833 B | — |
| evaluations/m2_best_test_loss_summary.csv | Data | 352 B | — |
| evaluations/m2_best_test_predictions.csv | Data | 244.9 KB | — |
| evaluations/m2_pair_calibration_validation_epochs.csv | Data | 1.6 KB | — |
| evaluations/m3_best_test_language_gap.csv | Data | 996 B | — |
| evaluations/m3_best_test_language_metrics.csv | Data | 816 B | — |
| evaluations/m3_best_test_loss_summary.csv | Data | 380 B | — |
| evaluations/m3_best_test_predictions.csv | Data | 258.3 KB | — |
| evaluations/m3_tokencal_validation_epochs.csv | Data | 1.6 KB | — |
| evaluations/strict_m0_standard_sft_torch_adamw_v2_validation_epochs.csv | Data | 771 B | — |
| evaluations/strict_m0_standard_sft_validation_epochs.csv | Data | 1.2 KB | — |
| figure_data/cell43_figure_index.csv | Data | 638 B | — |
| figure_data/fig01_metadata.csv | Data | 670 B | — |
| figure_data/fig01_tokenization_fragmentation_source.csv | Data | 555.6 KB | — |
| figure_data/fig02_metadata.csv | Data | 529 B | — |
| figure_data/fig02_reliability_source.csv | Data | 10.8 KB | — |
| figure_data/fig02_risk_coverage_source.csv | Data | 201.3 KB | — |
| figure_data/fig03_fragmentation_calibration_binned.csv | Data | 2.2 KB | — |
| figure_data/fig03_fragmentation_calibration_source.csv | Data | 100.5 KB | — |
| figure_data/fig03_metadata.csv | Data | 835 B | — |
| figure_data/fig04_accuracy_pattern_composition.csv | Data | 854 B | — |
| figure_data/fig04_composition_standardized_delta_ce.csv | Data | 855 B | — |
| figure_data/fig04_metadata.csv | Data | 695 B | — |
| figure_data/fig05_accuracy_transition_odds_ratios.csv | Data | 1.1 KB | — |
| figure_data/fig05_accuracy_transition_rates.csv | Data | 700 B | — |
| figure_data/fig05_metadata.csv | Data | 747 B | — |
| figure_data/fig06_fragmentation_distribution_instability_binned.csv | Data | 2.2 KB | — |
| figure_data/fig06_fragmentation_distribution_instability_pairs.csv | Data | 112.6 KB | — |
| figure_data/fig06_fragmentation_distribution_instability_quartiles.csv | Data | 804 B | — |
| figure_data/fig06_metadata.csv | Data | 722 B | — |
| figure_data/figure1_fragmentation_association_source.csv | Data | 262 B | — |
| figure_data/figure2_accuracy_brier_tradeoff_source.csv | Data | 937 B | — |
| figure_data/figure3_milu_multilingual_accuracy_source.csv | Data | 2.2 KB | — |
| figure_data/figure4_native_translated_accuracy_ci_source.csv | Data | 633 B | — |
| manifests/benqa_source_manifest.json | Data | 865 B | — |
| manifests/cell02_hf_setup_verification.json | Data | 654 B | — |
| manifests/cell03_runtime_audit_verification.json | Data | 1.2 KB | — |
| manifests/cell04_dependency_precision_verification.json | Data | 1.7 KB | — |
| manifests/cell05_benqa_download_verification.json | Data | 1.9 KB | — |
| manifests/cell06_benqa_pair_construction.json | Data | 978 B | — |
| manifests/cell07_cleaning_policy.json | Data | 1.2 KB | — |
| manifests/cell08_group_aware_split.json | Data | 1.0 KB | — |
| manifests/cell10_full_tokenization.json | Data | 1.3 KB | — |
| manifests/cell11_tokenization_statistics_figure1.json | Data | 1.6 KB | — |
| manifests/cell12_context_length_policy.json | Data | 975 B | — |
| manifests/cell13_base_model_load_sanity.json | Data | 1.4 KB | — |
| manifests/cell14_inference_batch_benchmark.json | Data | 1.0 KB | — |
| manifests/cell15_full_baseline_inference.json | Data | 1.3 KB | — |
| manifests/cell16_baseline_calibration.json | Data | 1.1 KB | — |
| manifests/cell17_fragmentation_calibration_hypothesis.json | Data | 1.8 KB | — |
| manifests/cell18_accuracy_pattern_decomposition.json | Data | 1.2 KB | — |
| manifests/cell19_fragmentation_accuracy_transitions.json | Data | 885 B | — |
| manifests/cell20_fragmentation_distribution_instability.json | Data | 1.3 KB | — |
| manifests/cell21_training_supervision_construction.json | Data | 1.5 KB | — |
| manifests/cell22_pair_aware_dataset_collator.json | Data | 822 B | — |
| manifests/cell23_length_aware_batching.json | Data | 1.2 KB | — |
| manifests/cell24_qlora_model_preparation.json | Data | 1.7 KB | — |
| manifests/cell25_training_objectives.json | Data | 1.9 KB | — |
| manifests/cell26_training_infrastructure.json | Data | 1.3 KB | — |
| manifests/cell27_m0_standard_sft_full_training.json | Data | 1.2 KB | — |
| manifests/cell28_m0_best_test_evaluation.json | Data | 1.0 KB | — |
| manifests/cell29_m1_calibration_full_training.json | Data | 1.3 KB | — |
| manifests/cell30_m1_best_test_evaluation.json | Data | 955 B | — |
| manifests/cell31_m2_pair_calibration_full_training.json | Data | 1.4 KB | — |
| manifests/cell32_m2_best_test_evaluation.json | Data | 941 B | — |
| manifests/cell33_m3_tokencal_full_training.json | Data | 1.4 KB | — |
| manifests/cell34_m3_best_test_evaluation.json | Data | 958 B | — |
| manifests/cell35b_b_reaso_val_ood_final.json | Data | 613 B | — |
| manifests/cell35e_b_reaso_benqa_overlap_final.json | Data | 628 B | — |
| manifests/cell36_b_reaso_ood_evaluation.json | Data | 414 B | — |
| manifests/cell37_shroom_cap_multilingual_audit.json | Data | 470 B | — |
| manifests/cell38_milu_multilingual_audit.json | Data | 582 B | — |
| manifests/cell38c_milu_final_test_lock.json | Data | 434 B | — |
| manifests/cell39_milu_multilingual_evaluation.json | Data | 591 B | — |
| manifests/cell40_milu_multilingual_bootstrap.json | Data | 343 B | — |
| manifests/cell41_milu_native_translated_robustness.json | Data | 434 B | — |
| manifests/cell42_final_consolidated_tables.json | Data | 278 B | — |
| manifests/cell43_publication_figures.json | Data | 1.7 KB | — |
| manifests/cell44_ablation_component_interpretation.json | Data | 516 B | — |
| manifests/cell45_dataset_characteristics.json | Data | 604 B | — |
| manifests/cell45b_milu_sample_recovery.json | Data | 671 B | — |
| manifests/cell46_qualitative_case_studies.json | Data | 601 B | — |
| manifests/cell46b_qualitative_patch.json | Data | 594 B | — |
| manifests/cell46c_final_qualitative_panel.json | Data | 489 B | — |
| manifests/training_private_local_files.csv | Data | 338 B | — |
| multilingual/cell39_chunks/BASE/part_0_manifest.json | Data | 685 B | — |
| multilingual/cell39_chunks/BASE/part_0_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/BASE/part_1_manifest.json | Data | 685 B | — |
| multilingual/cell39_chunks/BASE/part_1_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M0/part_0_manifest.json | Data | 710 B | — |
| multilingual/cell39_chunks/M0/part_0_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M0/part_1_manifest.json | Data | 711 B | — |
| multilingual/cell39_chunks/M0/part_1_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M1/part_0_manifest.json | Data | 710 B | — |
| multilingual/cell39_chunks/M1/part_0_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M1/part_1_manifest.json | Data | 709 B | — |
| multilingual/cell39_chunks/M1/part_1_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M2/part_0_manifest.json | Data | 714 B | — |
| multilingual/cell39_chunks/M2/part_0_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M2/part_1_manifest.json | Data | 715 B | — |
| multilingual/cell39_chunks/M2/part_1_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M3/part_0_manifest.json | Data | 707 B | — |
| multilingual/cell39_chunks/M3/part_0_predictions.csv | Data | 3.1 MB | — |
| multilingual/cell39_chunks/M3/part_1_manifest.json | Data | 707 B | — |
| multilingual/cell39_chunks/M3/part_1_predictions.csv | Data | 3.1 MB | — |
| multilingual/milu_base_m0_m1_m2_m3_metrics.csv | Data | 9.2 KB | — |
| multilingual/milu_base_m0_m1_m2_m3_predictions.csv | Data | 31.0 MB | b38612cedb9d |
| multilingual/milu_bootstrap_summary.json | Data | 1.2 KB | — |
| multilingual/milu_multilingual_evaluation_summary.json | Data | 28.2 KB | — |
| multilingual/milu_native_translated_bootstrap_10k.csv | Data | 37.5 KB | — |
| multilingual/milu_native_translated_counts.csv | Data | 201 B | — |
| multilingual/milu_native_translated_metrics.csv | Data | 12.6 KB | — |
| multilingual/milu_native_translated_summary.json | Data | 1.7 KB | — |
| multilingual/milu_unseen_paired_bootstrap_10k.csv | Data | 25.5 KB | — |
| ood/b_reaso_base_m0_m1_m2_m3_metrics.csv | Data | 1.4 KB | — |
| ood/b_reaso_base_m0_m1_m2_m3_predictions.csv | Data | 1.2 MB | — |
| ood/b_reaso_model_comparison_bootstrap_10k.csv | Data | 4.0 KB | — |
| ood/b_reaso_ood_evaluation_summary.json | Data | 4.2 KB | — |
| ood/b_reaso_subject_metrics.csv | Data | 23.7 KB | — |
| paper_tables/ablation_interpretation_narrative.json | Data | 1.4 KB | — |
| paper_tables/cell42_consolidated_results_summary.json | Data | 1.7 KB | — |
| paper_tables/dataset_representative_samples.json | Data | 17.9 KB | — |
| paper_tables/milu_native_translated_accuracy_robustness.csv | Data | 1.2 KB | — |
| paper_tables/milu_native_translated_primary_robustness.csv | Data | 6.0 KB | — |
| paper_tables/milu_rq3_primary_bootstrap.csv | Data | 3.6 KB | — |
| paper_tables/paper_claim_evidence_map.csv | Data | 2.4 KB | — |
| paper_tables/paper_safe_wording.csv | Data | 1.0 KB | — |
| paper_tables/qualitative_analysis_summary.json | Data | 583 B | — |
| paper_tables/qualitative_case_studies.json | Data | 6.9 KB | — |
| paper_tables/qualitative_final_panel.json | Data | 10.7 KB | — |
| paper_tables/table01_tokenization_fragmentation.csv | Data | 1.7 KB | — |
| paper_tables/table02_baseline_calibration.csv | Data | 614 B | — |
| paper_tables/table03_fragmentation_calibration_link.csv | Data | 576 B | — |
| paper_tables/table04_accuracy_pattern_heterogeneity.csv | Data | 1.0 KB | — |
| paper_tables/table05_fragmentation_accuracy_transitions.csv | Data | 624 B | — |
| paper_tables/table06_fragmentation_distribution_instability.csv | Data | 610 B | — |
| paper_tables/table_ablation_component_deltas.csv | Data | 10.2 KB | — |
| paper_tables/table_ablation_component_interpretation.csv | Data | 934 B | — |
| paper_tables/table_ablation_component_summary.csv | Data | 232 B | — |
| paper_tables/table_ablation_key_transitions.csv | Data | 557 B | — |
| paper_tables/table_benqa_heldout_ablation.csv | Data | 458 B | — |
| paper_tables/table_breaso_bootstrap.csv | Data | 606 B | — |
| paper_tables/table_breaso_ood.csv | Data | 691 B | — |
| paper_tables/table_dataset_characteristics.csv | Data | 1.4 KB | — |
| paper_tables/table_dataset_cleaning_audit.csv | Data | 672 B | — |
| paper_tables/table_dataset_context_lengths.csv | Data | 212 B | — |
| paper_tables/table_dataset_language_counts.csv | Data | 378 B | — |
| paper_tables/table_dataset_representative_samples.csv | Data | 5.1 KB | — |
| paper_tables/table_main_cross_benchmark_overview.csv | Data | 1.4 KB | — |
| paper_tables/table_milu_by_language.csv | Data | 5.2 KB | — |
| paper_tables/table_milu_native_translated_bootstrap.csv | Data | 6.0 KB | — |
| paper_tables/table_milu_native_translated_counts.csv | Data | 96 B | — |
| paper_tables/table_milu_native_translated_metrics.csv | Data | 2.8 KB | — |
| paper_tables/table_milu_reviewer_samples.csv | Data | 2.0 KB | — |
| paper_tables/table_milu_rq3_bootstrap.csv | Data | 3.6 KB | — |
| paper_tables/table_milu_unseen_pooled.csv | Data | 1.6 KB | — |
| paper_tables/table_qualitative_case_studies.csv | Data | 3.0 KB | — |
| paper_tables/table_qualitative_final_panel.csv | Data | 4.1 KB | — |
| statistics/baseline_calibration_language_gaps.csv | Data | 3.6 KB | — |
| statistics/baseline_calibration_metrics.csv | Data | 2.3 KB | — |
| statistics/baseline_test_bootstrap_calibration_gaps.csv | Data | 636 B | — |
| statistics/baseline_test_paired_accuracy_test.csv | Data | 269 B | — |
| statistics/baseline_test_paired_tests.csv | Data | 701 B | — |
| statistics/fragmentation_calibration_correlations.csv | Data | 2.7 KB | — |
| statistics/m0_best_test_language_gap_bootstrap.csv | Data | 604 B | — |
| statistics/m0_best_vs_base_test_bootstrap.csv | Data | 1.2 KB | — |
| statistics/m0_vs_m1_best_test_bootstrap.csv | Data | 1.3 KB | — |
| statistics/m0_vs_m2_best_test_bootstrap.csv | Data | 1.3 KB | — |
| statistics/m0_vs_m3_best_test_bootstrap.csv | Data | 1.3 KB | — |
| statistics/m1_best_test_language_gap_bootstrap.csv | Data | 604 B | — |
| statistics/m1_vs_m2_best_test_bootstrap.csv | Data | 1.2 KB | — |
| statistics/m1_vs_m3_best_test_bootstrap.csv | Data | 1.2 KB | — |
| statistics/m2_best_test_language_gap_bootstrap.csv | Data | 607 B | — |
| statistics/m2_vs_m3_best_test_bootstrap.csv | Data | 1.2 KB | — |
| statistics/m2_vs_prior_best_test_instability_bootstrap.csv | Data | 702 B | — |
| statistics/m3_best_test_language_gap_bootstrap.csv | Data | 603 B | — |
| statistics/m3_vs_prior_best_test_instability_bootstrap.csv | Data | 974 B | — |
| statistics/test_accuracy_pattern_conditional_slopes.csv | Data | 960 B | — |
| statistics/test_accuracy_pattern_correlations.csv | Data | 487 B | — |
| statistics/test_accuracy_pattern_descriptives.csv | Data | 477 B | — |
| statistics/test_accuracy_pattern_fragmentation_summary.csv | Data | 659 B | — |
| statistics/test_fragmentation_accuracy_pattern_interaction_wald.csv | Data | 155 B | — |
| statistics/test_fragmentation_accuracy_transition_bootstrap.csv | Data | 543 B | — |
| statistics/test_fragmentation_accuracy_transition_logistic.csv | Data | 2.8 KB | — |
| statistics/test_fragmentation_accuracy_transition_quartiles.csv | Data | 700 B | — |
| statistics/test_fragmentation_accuracy_transition_trends.csv | Data | 228 B | — |
| statistics/test_fragmentation_calibration_bootstrap_draws.csv | Data | 448.4 KB | — |
| statistics/test_fragmentation_calibration_bootstrap_summary.csv | Data | 275 B | — |
| statistics/test_fragmentation_calibration_regressions.csv | Data | 1.1 KB | — |
| statistics/test_fragmentation_calibration_residualized_association.csv | Data | 308 B | — |
| statistics/test_fragmentation_calibration_standardized_regression.csv | Data | 275 B | — |
| statistics/test_fragmentation_distribution_instability_correlations.csv | Data | 620 B | — |
| statistics/test_fragmentation_distribution_instability_q4_q1_bootstrap.csv | Data | 503 B | — |
| statistics/test_fragmentation_distribution_instability_quartiles.csv | Data | 804 B | — |
| statistics/test_fragmentation_distribution_instability_trends.csv | Data | 221 B | — |
| statistics/test_fragmentation_js_bootstrap.csv | Data | 263 B | — |
| statistics/test_fragmentation_js_regression.csv | Data | 524 B | — |
| statistics/test_fragmentation_prediction_disagreement_logistic.csv | Data | 329 B | — |
| statistics/test_fragmentation_quartile_accuracy_pattern_association.csv | Data | 260 B | — |
| statistics/test_fragmentation_quartile_accuracy_pattern_composition.csv | Data | 854 B | — |
| statistics/test_fragmentation_quartile_composition_standardized.csv | Data | 855 B | — |
| statistics/test_fragmentation_quartile_pattern_means.csv | Data | 1.4 KB | — |
| statistics/test_fragmentation_quartile_standardized_trend.csv | Data | 150 B | — |
| statistics/test_fragmentation_quartile_trend.csv | Data | 95 B | — |
| statistics/test_fragmentation_quartiles.csv | Data | 989 B | — |
| statistics/tokenization_bootstrap_ci.csv | Data | 787 B | — |
| statistics/tokenization_paired_statistical_tests.csv | Data | 3.5 KB | — |
| tokenization/b_reaso_val_ood_tokenization_audit.csv | Data | 68.3 KB | — |
| tokenization/benqa_context_length_audit.csv | Data | 919.8 KB | — |
| tokenization/benqa_context_length_split_summary.csv | Data | 237 B | — |
| tokenization/benqa_fragmentation_direction.csv | Data | 134 B | — |
| tokenization/benqa_overlength_prompt_audit.csv | Data | 2.9 KB | — |
| tokenization/benqa_overlength_resolution.csv | Data | 4.2 KB | — |
| tokenization/benqa_prompt_length_audit.csv | Data | 274 B | — |
| tokenization/benqa_tokenization_pairs.csv | Data | 2.2 MB | — |
| tokenization/benqa_tokenization_split_summary.csv | Data | 2.6 KB | — |
| tokenization/benqa_tokenization_summary.csv | Data | 2.2 KB | — |
| tokenization/milu_final_language_tokenization_summary.csv | Data | 292 B | — |
| tokenization/milu_final_test_tokenization_audit.csv | Data | 1.0 MB | — |
| tokenization/milu_language_tokenization_summary.csv | Data | 294 B | — |
| tokenization/milu_test_tokenization_audit.csv | Data | 1.8 MB | — |
| training_logs/cell22_full_collation_audit.csv | Data | 381 B | — |
| training_logs/cell23_padding_efficiency_comparison.csv | Data | 447 B | — |
| training_logs/cell23_train_sampler_epoch_audit.csv | Data | 1.0 KB | — |
| training_logs/cell24_gpu_memory.csv | Data | 476 B | — |
| training_logs/cell24_lora_target_module_audit.csv | Data | 10.9 KB | — |
| training_logs/cell24_parameter_device_audit.csv | Data | 36 B | — |
| training_logs/cell24_trainable_parameter_audit.csv | Data | 45.2 KB | — |
| training_logs/cell24_trainable_parameter_summary.csv | Data | 344 B | — |
| training_logs/cell25_forward_equivalence.csv | Data | 318 B | — |
| training_logs/cell25_gradient_audit.csv | Data | 35.8 KB | — |
| training_logs/cell25_loss_sanity.csv | Data | 1.0 KB | — |
| training_logs/cell26_initial_adapter_hf_verification.csv | Data | 653 B | — |
| training_logs/cell26_reversible_optimizer_dry_run.csv | Data | 645 B | — |
| training_logs/cell26_validation_smoke.csv | Data | 668 B | — |
| training_logs/cell28_test_construction_audit.csv | Data | 63.7 KB | — |
| training_logs/m0_standard_sft_checkpoint_verification.csv | Data | 291 B | — |
| training_logs/m0_standard_sft_epoch_summary.csv | Data | 1.3 KB | — |
| training_logs/m0_standard_sft_resume_progress.json | Data | 393 B | — |
| training_logs/m0_standard_sft_step_log.csv | Data | 171.6 KB | — |
| training_logs/m1_calibration_checkpoint_verification.csv | Data | 133 B | — |
| training_logs/m1_calibration_epoch_summary.csv | Data | 1.5 KB | — |
| training_logs/m1_calibration_fp16_recovery_log.csv | Data | 142 B | — |
| training_logs/m1_calibration_resume_progress.json | Data | 521 B | — |
| training_logs/m1_calibration_step_log.csv | Data | 153.8 KB | — |
| training_logs/m2_pair_calibration_checkpoint_verification.csv | Data | 291 B | — |
| training_logs/m2_pair_calibration_epoch_summary.csv | Data | 1.6 KB | — |
| training_logs/m2_pair_calibration_fp16_recovery_log.csv | Data | 233 B | — |
| training_logs/m2_pair_calibration_resume_progress.json | Data | 553 B | — |
| training_logs/m2_pair_calibration_step_log.csv | Data | 160.1 KB | — |
| training_logs/m3_tokencal_checkpoint_verification.csv | Data | 307 B | — |
| training_logs/m3_tokencal_epoch_summary.csv | Data | 1.5 KB | — |
| training_logs/m3_tokencal_fp16_recovery_log.csv | Data | 142 B | — |
| training_logs/m3_tokencal_resume_progress.json | Data | 545 B | — |
| training_logs/m3_tokencal_step_log.csv | Data | 155.0 KB | — |
| training_logs/strict_m0_standard_sft_epoch_summary.csv | Data | 1.0 KB | — |
| training_logs/strict_m0_standard_sft_resume_progress.json | Data | 446 B | — |
| training_logs/strict_m0_standard_sft_step_log.csv | Data | 117.2 KB | — |
| training_logs/strict_m0_standard_sft_torch_adamw_v2_epoch_summary.csv | Data | 730 B | — |
| training_logs/strict_m0_standard_sft_torch_adamw_v2_optimizer_policy.json | Data | 468 B | — |
| training_logs/strict_m0_standard_sft_torch_adamw_v2_resume_progress.json | Data | 500 B | — |
| training_logs/strict_m0_standard_sft_torch_adamw_v2_step_log.csv | Data | 66.1 KB | — |
| figures/fig01_tokenization_fragmentation.pdf | Other | 31.5 KB | — |
| figures/fig01_tokenization_fragmentation.png | Other | 468.4 KB | b19b182c9b6c |
| figures/fig02_baseline_calibration.pdf | Other | 26.7 KB | — |
| figures/fig02_baseline_calibration.png | Other | 682.1 KB | cdf2d3b7895c |
| figures/fig03_fragmentation_calibration_link.pdf | Other | 34.8 KB | — |
| figures/fig03_fragmentation_calibration_link.png | Other | 1.3 MB | 1284f0222096 |
| figures/fig04_accuracy_pattern_decomposition.pdf | Other | 20.2 KB | — |
| figures/fig04_accuracy_pattern_decomposition.png | Other | 607.0 KB | b8f54bca3009 |
| figures/fig05_fragmentation_accuracy_transitions.pdf | Other | 21.4 KB | — |
| figures/fig05_fragmentation_accuracy_transitions.png | Other | 562.7 KB | 1e7dd0e2c04e |
| figures/fig06_fragmentation_distribution_instability.pdf | Other | 35.6 KB | — |
| figures/fig06_fragmentation_distribution_instability.png | Other | 1.4 MB | b1a9d73a64ef |
| figures/figure1_fragmentation_instability_association.pdf | Other | 14.2 KB | — |
| figures/figure1_fragmentation_instability_association.png | Other | 393.7 KB | fc076cb8ec9e |
| figures/figure2_cross_benchmark_accuracy_brier_tradeoff.pdf | Other | 15.9 KB | — |
| figures/figure2_cross_benchmark_accuracy_brier_tradeoff.png | Other | 322.8 KB | b66c32b8bcdc |
| figures/figure3_milu_accuracy_by_language.pdf | Other | 13.5 KB | — |
| figures/figure3_milu_accuracy_by_language.png | Other | 238.7 KB | 09d2332fc19f |
| figures/figure4_native_translated_accuracy_robustness.pdf | Other | 13.9 KB | — |
| figures/figure4_native_translated_accuracy_robustness.png | Other | 351.6 KB | 00b196cfb330 |
| figures/figure5_qualitative_case_studies.pdf | Other | 33.2 KB | — |
| figures/figure5_qualitative_case_studies.png | Other | 1.7 MB | 459e9b7a16de |
| figures/figure6_final_bengali_multilingual_qualitative_panel.pdf | Other | 38.0 KB | — |
| figures/figure6_final_bengali_multilingual_qualitative_panel.png | Other | 2.2 MB | 36271bbba96d |
| paper_tables/qualitative_case_studies.html | Other | 14.5 KB | — |
| paper_tables/qualitative_final_panel.html | Other | 21.1 KB | — |
| .gitattributes | Repository | 2.6 KB | — |
| config/tokenizer_language_sanity.csv | Tokenizer | 187 B | — |
| config/tokenizer_provenance.csv | Tokenizer | 708 B | — |
| config/tokenizer_snapshot_inventory.csv | Tokenizer | 3.0 KB | — |
| config/tokenizer_special_tokens.csv | Tokenizer | 121 B | — |
| manifests/cell09_tokenizer_audit.json | Tokenizer | 965 B | — |
License and Download
- License
- Not stated by the source
- Access
- No access gate
Download from HABIB
Released by HABIB through its official repository on Hugging Face.