Skip to page content
SAVRN
Search Plan a project

Open-weight model

code-HFD

by Tinghong Ye Tinghong-Ye22/code-HFD

code-HFD is an open-weight model from Tinghong Ye. Its published files total 149.3 GB.

HFD研究持续交互中的人形机器人全双工行为:接收语音与可选的图像/视频,连续生成动作,并在视觉问答时生成语音回答。 任务包括 turn-taking、interruption(含stop)、backchannel、hesitation、pause。视觉问答叠加到这些交互任务中,不另设独立的SWA任务。代码中保留的swa名称用于模块与历史配置兼容。 直接执行bash inferrenderauto.sh会选择兼容的最新run并持续监听。…

Parameters—
Context—
Weights149.3 GB
License—
AccessOpen weights
Monthly Downloads—

Model Card

HFD研究持续交互中的人形机器人全双工行为:接收语音与可选的图像/视频,连续生成动作,并在视觉问答时生成语音回答。 任务包括 turn-taking、interruption(含stop)、backchannel、hesitation、pause。视觉问答叠加到这些交互任务中,不另设独立的SWA任务。代码中保留的swa名称用于模块与历史配置兼容。 直接执行bash inferrenderauto.sh会选择兼容的最新run并持续监听。 该入口接替原inferrenderautomultiturn.sh;任务集合、train/val样例、多卡队列、 checkpoint间隔和异步渲染逻辑不变。无需另起内部worker,也不要同时启动两份自动推理。 训练环境复用.venv-psi。依赖、模型和数据路径必须可用,启动器会检查;不要把配置打印成功当成完整训练验证。关键参数在训练脚本顶部,固定配置与环境变量映射在当前配方。完整续训与WandB续接参数见训练脚本末尾说明。 backup/preparefullduplexcontinuousaudio.sh用于连续背景音数据重建(旧流程还会导出hidden), backup/prepareomnitrainingdata.sh及其调用的backup/cachevideoomniembeds.py 不要为清理目录而重新执行造数据。旧canonical-motion消融、旧OSS索引预构建、 早期hidden包装入口已退役,恢复方式见清理记录。…

Excerpt from the card by Tinghong Ye.

Identity and Version

Repository
Tinghong-Ye22/code-HFD
Publisher
Tinghong Ye
Task
Not stated by the source
Modality
Other
Library
Not stated by the source
Parameters
Not stated by the source
Languages
Not stated by the source
Revision
e543bf080ad10310773513113d207cacd89d5443
First published
2026-10-14
Last updated
2026-10-16

Files and Weights

558 files, 149.3 GB in total.

Configuration396 files · 5.6 MB
Tokenizer2 files · 5.7 KB
Documentation80 files · 1.9 MB
Other75 files · 149.3 GB
Repository5 files · 2.8 KB
Every file
FileTypeSizeSHA-256
FULL_SNAPSHOT.manifest.jsonConfiguration1.5 KB —
assets/stats/egodex_stat_all.jsonConfiguration9.3 KB —
assets/stats/he_raw_rel_stats_combined.jsonConfiguration9.6 KB —
assets/stats/he_raw_rel_stats_combined_no_static.jsonConfiguration9.6 KB —
assets/stats/psi0_dance_gvhmr_motion70_textop_stats.jsonConfiguration18.0 KB —
assets/stats/seed_all_g1_motion36_abs_joint_delta_root_stats.jsonConfiguration11.8 KB —
assets/stats/seed_all_g1_motion36_delta_stats.jsonConfiguration11.9 KB —
assets/stats/seed_all_g1_motion39_textop_stats.jsonConfiguration12.9 KB —
assets/stats/seed_all_g1_motion70_textop_stats.jsonConfiguration17.7 KB —
assets/stats/seed_all_g1_root40_bounds.jsonConfiguration4.1 KB —
assets/stats/seed_g1_motion36_abs_joint_delta_root_stats.jsonConfiguration11.8 KB —
assets/stats/seed_g1_motion36_delta_stats.jsonConfiguration10.3 KB —
assets/stats/seed_g1_motion36_stats.jsonConfiguration11.8 KB —
assets/stats/seed_g1_motion36_stats_new.jsonConfiguration11.8 KB —
assets/stats/seed_g1_motion70_textop_stats.jsonConfiguration17.6 KB —
assets/stats/task_description_dict.jsonConfiguration46.0 KB —
backup/cache_video_omni_embeds.pyConfiguration37.6 KB —
benchmark/audit.pyConfiguration6.8 KB —
benchmark/bones_seed_soma_to_smplx/convert.pyConfiguration18.8 KB —
benchmark/bones_seed_soma_to_smplx/dashboard.pyConfiguration6.4 KB —
benchmark/bones_seed_soma_to_smplx/finalize.pyConfiguration8.5 KB —
benchmark/bones_seed_soma_to_smplx/fit_identity.pyConfiguration2.9 KB —
benchmark/bones_seed_soma_to_smplx/inspect_sample.pyConfiguration3.7 KB —
benchmark/bones_seed_soma_to_smplx/pipeline.pyConfiguration11.1 KB —
benchmark/bones_seed_soma_to_smplx/prepare.pyConfiguration10.8 KB —
benchmark/bones_seed_soma_to_smplx/select_pilot.pyConfiguration2.2 KB —
benchmark/bones_seed_soma_to_smplx/tests/test_preparation.pyConfiguration2.8 KB —
benchmark/bones_seed_soma_to_smplx/tests/test_recovery.pyConfiguration2.7 KB —
benchmark/bones_seed_soma_to_smplx/tests/test_schema.pyConfiguration2.2 KB —
benchmark/bones_seed_soma_to_smplx/track.pyConfiguration3.1 KB —
benchmark/bones_seed_soma_to_smplx/validate.pyConfiguration6.6 KB —
benchmark/bones_seed_soma_to_smplx/verify_archive.pyConfiguration1.2 KB —
benchmark/common.pyConfiguration2.1 KB —
benchmark/evaluate.pyConfiguration14.1 KB —
benchmark/external_features.pyConfiguration7.3 KB —
benchmark/external_speech.pyConfiguration4.0 KB —
benchmark/external_split.pyConfiguration4.6 KB —
benchmark/external_visual.pyConfiguration8.0 KB —
benchmark/generate.pyConfiguration5.0 KB —
benchmark/native_eval.pyConfiguration3.4 KB —
benchmark/prepare.pyConfiguration12.7 KB —
benchmark/preview.pyConfiguration5.7 KB —
benchmark/review.pyConfiguration3.1 KB —
benchmark/review_external.pyConfiguration9.1 KB —
benchmark/revision.pyConfiguration6.0 KB —
benchmark/select_cohort.pyConfiguration5.3 KB —
benchmark/session_data.pyConfiguration6.1 KB —
benchmark/session_overlay.pyConfiguration6.5 KB —
benchmark/test_adapters.pyConfiguration6.6 KB —
benchmark/test_external.pyConfiguration6.5 KB —
benchmark/test_native_eval.pyConfiguration10.9 KB —
benchmark/vendor.pyConfiguration2.3 KB —
doc/hf_audio_motion/PATH_MAP.jsonConfiguration987 B —
paper_context_v4/architecture_spec.jsonConfiguration3.7 KB —
paper_context_v4/data_release_snapshot.jsonConfiguration3.7 KB —
paper_context_v4/evidence_manifest.jsonConfiguration7.8 KB —
scripts/convert_video_records_to_lazy.pyConfiguration3.6 KB —
scripts/data/audit_hfd_live_video_windows.pyConfiguration17.8 KB —
scripts/data/audit_source_split_artifacts.pyConfiguration16.5 KB —
scripts/data/audit_third_party.pyConfiguration5.3 KB —
scripts/data/build_canonical_motion_manifest.pyConfiguration17.5 KB —
scripts/data/build_continuous_audio_command_sources.pyConfiguration27.2 KB —
scripts/data/build_hfd_gqa_scale.pyConfiguration9.0 KB —
scripts/data/build_hfd_multimodal_overlay_catalog.pyConfiguration10.3 KB —
scripts/data/build_hfd_vqa_candidate_pilot.pyConfiguration14.4 KB —
scripts/data/build_swa_charades_ego.pyConfiguration18.8 KB —
scripts/data/build_video_clip_dataset.pyConfiguration43.0 KB —
scripts/data/cache_omni_hidden_states.pyConfiguration136.2 KB —
scripts/data/cache_omni_hidden_states_test.pyConfiguration10.9 KB —
scripts/data/calc_dance_audio_textop_stats.pyConfiguration5.1 KB —
scripts/data/calc_modality_stats.pyConfiguration2.9 KB —
scripts/data/calc_seed_textop_stats.pyConfiguration16.1 KB —
scripts/data/check_hfd_overlay_manifest.pyConfiguration5.7 KB —
scripts/data/consolidate_visual_dataset.pyConfiguration24.6 KB —
scripts/data/download.pyConfiguration3.0 KB —
scripts/data/encode_native_visual.pyConfiguration4.3 KB —
scripts/data/encode_swa_vqav2.pyConfiguration23.3 KB —
scripts/data/expand_hfd_video_voices.pyConfiguration32.6 KB —
scripts/data/export_hfd_overlay_assets.pyConfiguration3.9 KB —
scripts/data/extract_audio_npy_cache.pyConfiguration16.6 KB —
scripts/data/filter_2264_video_clip_safe_rows.pyConfiguration6.2 KB —
scripts/data/filter_swa_charades_reviews.pyConfiguration4.6 KB —
scripts/data/fix_lerobot_parquet.pyConfiguration381 B —
scripts/data/generate_hfd_backchannels.pyConfiguration14.0 KB —
scripts/data/generate_hfd_control_commands.pyConfiguration12.8 KB —
scripts/data/generate_hfd_hesitation_pause.pyConfiguration51.2 KB —
scripts/data/generate_office_background_candidates.pyConfiguration20.9 KB —
scripts/data/hf_export_dashboard.pyConfiguration14.1 KB —
scripts/data/manage_source_split_artifacts.pyConfiguration15.4 KB —
scripts/data/manage_source_split_artifacts_test.pyConfiguration2.6 KB —
scripts/data/materialize_audio_wav.pyConfiguration14.9 KB —
scripts/data/merge_omni_hidden_for_oss.pyConfiguration53.5 KB —
scripts/data/merge_swa_source_plans.pyConfiguration2.1 KB —
scripts/data/package_hf_audio_motion.pyConfiguration20.0 KB —
scripts/data/package_hf_smplx.pyConfiguration11.6 KB —
scripts/data/patch_2264_source_motion_dir.pyConfiguration13.9 KB —
scripts/data/patch_lerobot_meta.pyConfiguration404 B —
scripts/data/prebuild_multimodal_records_cache.pyConfiguration21.3 KB —
scripts/data/prepare_full_duplex_indexes.pyConfiguration4.9 KB —
scripts/data/prepare_hfd_gqa_assets.pyConfiguration5.8 KB —
scripts/data/prepare_hfd_video_assets.pyConfiguration28.7 KB —
scripts/data/prepare_omni_oss_training.pyConfiguration7.6 KB —
scripts/data/prepare_swa_vqav2.pyConfiguration9.4 KB —
scripts/data/raw_dexmate_psi0.pyConfiguration9.4 KB —
scripts/data/raw_he_to_psi0.pyConfiguration43.7 KB —
scripts/data/raw_to_lerobot.pyConfiguration26.3 KB —
scripts/data/raw_to_lerobot_v2.pyConfiguration30.6 KB —
scripts/data/read_hf_audio_motion.pyConfiguration4.2 KB —
scripts/data/recover_2264_video_clip_metadata.pyConfiguration19.5 KB —
scripts/data/render_hfd_hesitation_pause_gt.pyConfiguration20.5 KB —
scripts/data/render_hfd_unified_multimodal_gt.pyConfiguration23.5 KB —
scripts/data/render_live_overlay_gt.pyConfiguration3.4 KB —
scripts/data/repack_omni_hidden_shard1_bucket.pyConfiguration17.2 KB —
scripts/data/repartition_motion_record_cache.pyConfiguration4.0 KB —
scripts/data/repartition_video_lazy_record_cache.pyConfiguration9.4 KB —
scripts/data/review_dashboard/server.pyConfiguration21.0 KB —
scripts/data/run_causal_video_audit.pyConfiguration11.0 KB —
scripts/data/run_swa_vqav2_pipeline.pyConfiguration9.7 KB —
scripts/data/swa_pipeline_queue.pyConfiguration3.2 KB —
scripts/data/task_description_dict.jsonConfiguration59 B —
scripts/data/tts_codec_graph.pyConfiguration5.3 KB —
scripts/data/validate_hfd_video_assets.pyConfiguration17.0 KB —
scripts/data/validate_swa_manifest.pyConfiguration8.5 KB —
scripts/data/validate_visual_dataset.pyConfiguration3.2 KB —
scripts/data/watch_hfd_video_assets.pyConfiguration2.2 KB —
scripts/deepspeed/zero2.jsonConfiguration556 B —
scripts/deepspeed/zero3.jsonConfiguration831 B —
scripts/deepspeed/zero3_offload.jsonConfiguration1.3 KB —
scripts/deploy/offline_psi0_sonic_motion_tracking.pyConfiguration5.3 KB —
scripts/deploy/online_psi0_sonic_bridge.pyConfiguration46.1 KB —
scripts/deploy/run_psi0_sonic_online_jog.pyConfiguration29.4 KB —
scripts/deploy/run_sonic_sim_record_online.pyConfiguration13.1 KB —
scripts/deploy_real/g1_real_state_writer.pyConfiguration7.3 KB —
scripts/deploy_real/online_psi0_sonic_bridge_real.pyConfiguration29.5 KB —
scripts/deploy_real/run_psi0_sonic_online_jog.pyConfiguration26.9 KB —
scripts/diagnostics/probe_hfd_control_semantics.pyConfiguration20.8 KB —
scripts/infer/check_run_visual_data.pyConfiguration841 B —
scripts/infer/full_duplex_audio_to_motion.pyConfiguration107.4 KB —
scripts/infer/multiturn_auto_scenario_backchannel_train_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_backchannel_val_v1.jsonConfiguration2.4 KB —
scripts/infer/multiturn_auto_scenario_emergency_stop_train_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_emergency_stop_val_v1.jsonConfiguration1.6 KB —
scripts/infer/multiturn_auto_scenario_hesitation_filled_train_v1.jsonConfiguration2.1 KB —
scripts/infer/multiturn_auto_scenario_hesitation_filled_train_v2.jsonConfiguration2.2 KB —
scripts/infer/multiturn_auto_scenario_hesitation_filled_val_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_hesitation_filled_val_v2.jsonConfiguration2.4 KB —
scripts/infer/multiturn_auto_scenario_hesitation_repair_train_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_hesitation_repair_train_v2.jsonConfiguration2.5 KB —
scripts/infer/multiturn_auto_scenario_hesitation_repair_val_v1.jsonConfiguration2.5 KB —
scripts/infer/multiturn_auto_scenario_hesitation_repair_val_v2.jsonConfiguration2.7 KB —
scripts/infer/multiturn_auto_scenario_interruption_train_v1.jsonConfiguration2.8 KB —
scripts/infer/multiturn_auto_scenario_interruption_val_v1.jsonConfiguration3.3 KB —
scripts/infer/multiturn_auto_scenario_pause_train_v1.jsonConfiguration2.2 KB —
scripts/infer/multiturn_auto_scenario_pause_train_v2.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_pause_val_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_pause_val_v2.jsonConfiguration2.4 KB —
scripts/infer/multiturn_auto_scenario_safe_hold_train_v1.jsonConfiguration2.3 KB —
scripts/infer/multiturn_auto_scenario_safe_hold_val_v1.jsonConfiguration1.7 KB —
scripts/infer/multiturn_auto_scenario_v1.jsonConfiguration2.9 KB —
scripts/infer/multiturn_auto_scenario_val_v1.jsonConfiguration3.9 KB —
scripts/infer/multiturn_auto_suite_v2.jsonConfiguration3.0 KB —
scripts/infer/multiturn_auto_suite_v3.jsonConfiguration3.0 KB —
scripts/infer/mux_full_duplex_audio.pyConfiguration6.0 KB —
scripts/infer/render_swa_result.pyConfiguration17.3 KB —
scripts/infer/run_egl_renderer.pyConfiguration1.4 KB —
scripts/infer/run_multiturn_infer_suite.pyConfiguration47.6 KB —
scripts/infer/seed_audio_to_motion.pyConfiguration22.3 KB —
scripts/infer/seed_motion_teacher_reconstruct.pyConfiguration20.2 KB —
scripts/infer/seed_text_to_motion.pyConfiguration37.8 KB —
scripts/infer/seed_video_to_motion.pyConfiguration23.8 KB —
scripts/infer/swa_visual_qa.pyConfiguration4.1 KB —
scripts/infer/unified_overlay.pyConfiguration10.1 KB —
scripts/infer/validate_speech_qa_runtime.pyConfiguration2.9 KB —
scripts/launch_hfd_dlc.pyConfiguration7.4 KB —
scripts/launch_hfd_training.pyConfiguration5.0 KB —
scripts/predownload_qwen3vl.pyConfiguration742 B —
scripts/prepare_hfd_continuation.pyConfiguration8.8 KB —
scripts/save_motion_teacher_hidden_cache.pyConfiguration5.0 KB —
scripts/save_posttrain_action_expert.pyConfiguration1.6 KB —
scripts/save_pretrain_qwen3vl_backbone.pyConfiguration2.6 KB —
scripts/train.pyConfiguration33.6 KB —
scripts/train/ddp.launch.yamlConfiguration367 B —
scripts/train/deepspeed.launch.yamlConfiguration428 B —
scripts/training_dashboard.pyConfiguration44.4 KB —
scripts/viz/fk.pyConfiguration6.2 KB —
scripts/viz/g1.pyConfiguration2.8 KB —
scripts/viz/viz_episode_real.pyConfiguration11.4 KB —
scripts/viz/viz_episode_simple.pyConfiguration12.5 KB —
src/lerobot_patch/common/datasets/lerobot_dataset.pyConfiguration51.8 KB —
src/psi/__init__.pyConfiguration366 B —
src/psi/async_render_queue.pyConfiguration8.0 KB —
src/psi/checkpoint_lease.pyConfiguration3.2 KB —
src/psi/config/__init__.pyConfiguration —
src/psi/config/augmentation.pyConfiguration4.0 KB —
src/psi/config/config.pyConfiguration9.1 KB —
src/psi/config/data_egodex.pyConfiguration1.5 KB —
src/psi/config/data_full_duplex_audio_command.pyConfiguration24.3 KB —
src/psi/config/data_he.pyConfiguration1.4 KB —
src/psi/config/data_lerobot.pyConfiguration2.0 KB —
src/psi/config/data_mix.pyConfiguration3.7 KB —
src/psi/config/data_seed.pyConfiguration1.9 KB —
src/psi/config/data_seed_audio.pyConfiguration2.5 KB —
src/psi/config/data_seed_text_audio_mix.pyConfiguration5.2 KB —
src/psi/config/data_seed_text_audio_video_mix.pyConfiguration30.8 KB —
src/psi/config/data_seed_video.pyConfiguration2.0 KB —
src/psi/config/hfd_current_recipe.pyConfiguration17.6 KB —
src/psi/config/model_act.pyConfiguration1.0 KB —
src/psi/config/model_dp.pyConfiguration464 B —
src/psi/config/model_full_duplex_psi0.pyConfiguration8.9 KB —
src/psi/config/model_motion_distill.pyConfiguration1.1 KB —
src/psi/config/model_psi0.pyConfiguration4.2 KB —
src/psi/config/model_qwen3vl.pyConfiguration2.0 KB —
src/psi/config/train/finetune_real_psi0_config.pyConfiguration649 B —
src/psi/config/train/finetune_simple_psi0_config.pyConfiguration705 B —
src/psi/config/train/posttrain_full_duplex_audio_command_psi0_config.pyConfiguration18.7 KB —
src/psi/config/train/posttrain_he_psi0_config.pyConfiguration689 B —
src/psi/config/train/posttrain_mix_psi0_config.pyConfiguration732 B —
src/psi/config/train/posttrain_v1_audio_psi0_config.pyConfiguration577 B —
src/psi/config/train/posttrain_v1_psi0_config.pyConfiguration752 B —
src/psi/config/train/posttrain_v1_seed_text_audio_mix_psi0_config.pyConfiguration600 B —
src/psi/config/train/posttrain_v1_seed_text_audio_video_mix_psi0_config.pyConfiguration616 B —
src/psi/config/train/posttrain_v1_seed_text_audio_video_mix_psi0_motion_distill_config.pyConfiguration742 B —
src/psi/config/train/pretrain_egodex_qwen3vl_config.pyConfiguration743 B —
src/psi/config/train/pretrain_he_qwen3vl_config.pyConfiguration741 B —
src/psi/config/train/pretrain_mix_qwen3vl_config.pyConfiguration695 B —
src/psi/config/train/real_act_config.pyConfiguration645 B —
src/psi/config/train/real_dp_config.pyConfiguration684 B —
src/psi/config/train/simple_act_config.pyConfiguration647 B —
src/psi/config/train/simple_dp_config.pyConfiguration686 B —
src/psi/config/transform.pyConfiguration67.3 KB —
src/psi/config/transform_motion_distill.pyConfiguration2.1 KB —
src/psi/config/transform_seed_test.pyConfiguration2.7 KB —
src/psi/data/__init__.pyConfiguration105 B —
src/psi/data/dataset.pyConfiguration8.7 KB —
src/psi/data/egodex/egodex_dataset.pyConfiguration24.5 KB —
src/psi/data/egodex/utils/data_utils.pyConfiguration1.5 KB —
src/psi/data/egodex/utils/draw_utils.pyConfiguration2.3 KB —
src/psi/data/egodex/utils/skeleton_tfs.pyConfiguration2.8 KB —
src/psi/data/humanoid/he_raw_dataset.pyConfiguration9.8 KB —
src/psi/data/lerobot/__init__.pyConfiguration46 B —
src/psi/data/lerobot/compat.pyConfiguration572 B —
src/psi/data/lerobot/lerobot_ext.pyConfiguration2.4 KB —
src/psi/data/sampler.pyConfiguration26.2 KB —
src/psi/data/sampler_test.pyConfiguration7.8 KB —
src/psi/data/seed/causal_video_qa.pyConfiguration4.2 KB —
src/psi/data/seed/full_duplex_audio_command_dataset.pyConfiguration366.5 KB —
src/psi/data/seed/full_duplex_audio_contract.pyConfiguration5.9 KB —
src/psi/data/seed/full_duplex_timing.pyConfiguration1.3 KB —
src/psi/data/seed/hfd_multimodal_overlay.pyConfiguration12.4 KB —
src/psi/data/seed/live_multimodal_overlay.pyConfiguration20.8 KB —
src/psi/data/seed/motion_frame_index.pyConfiguration4.5 KB —
src/psi/data/seed/motion_repr.pyConfiguration12.7 KB —
src/psi/data/seed/motion_repr40.pyConfiguration12.1 KB —
src/psi/data/seed/native_visual.pyConfiguration5.3 KB —
src/psi/data/seed/omni_hidden_cache.pyConfiguration25.9 KB —
src/psi/data/seed/omni_hidden_cache_test.pyConfiguration7.8 KB —
src/psi/data/seed/omni_oss_sampler.pyConfiguration34.5 KB —
src/psi/data/seed/omni_oss_sampler_test.pyConfiguration3.4 KB —
src/psi/data/seed/omni_sqlite_cache.pyConfiguration5.7 KB —
src/psi/data/seed/progress.pyConfiguration2.0 KB —
src/psi/data/seed/qwen_streaming_audio.pyConfiguration58.9 KB —
src/psi/data/seed/seed_audio_motion_dataset.pyConfiguration55.7 KB —
src/psi/data/seed/seed_audio_motion_dataset_test.pyConfiguration568 B —
src/psi/data/seed/seed_motion_dataset.pyConfiguration28.0 KB —
src/psi/data/seed/seed_video_motion_dataset.pyConfiguration87.2 KB —
src/psi/data/seed/seed_video_motion_dataset_test.pyConfiguration1.0 KB —
src/psi/data/seed/source_split.pyConfiguration3.7 KB —
src/psi/data/seed/source_split_test.pyConfiguration3.0 KB —
src/psi/data/seed/startup_index_cache.pyConfiguration4.0 KB —
src/psi/data/seed/storage_paths.pyConfiguration1.8 KB —
src/psi/data/seed/swa_frontend.pyConfiguration1.7 KB —
src/psi/data/seed/swa_visual_qa_dataset.pyConfiguration9.5 KB —
src/psi/data/seed/text_prompt.pyConfiguration5.3 KB —
src/psi/data/seed/third_party_dataset.pyConfiguration13.3 KB —
src/psi/data/seed/video_motion_local_cache.pyConfiguration10.5 KB —
src/psi/data/seed/video_overlay.pyConfiguration4.1 KB —
src/psi/data/seed/visual_dataset.pyConfiguration8.4 KB —
src/psi/deploy/helpers.pyConfiguration4.3 KB —
src/psi/deploy/psi0_serve_simple.pyConfiguration9.6 KB —
src/psi/deploy/psi_serve_rtc-dexmate.pyConfiguration22.0 KB —
src/psi/deploy/psi_serve_rtc-trainingtimertc-sonic.pyConfiguration58.0 KB —
src/psi/deploy/psi_serve_rtc-trainingtimertc.pyConfiguration25.8 KB —
src/psi/dlc_storage.pyConfiguration8.1 KB —
src/psi/hfd_training_launcher.pyConfiguration19.0 KB —
src/psi/hfd_training_preflight.pyConfiguration8.2 KB —
src/psi/models/causal_av.pyConfiguration16.4 KB —
src/psi/models/full_duplex_psi0.pyConfiguration146.0 KB —
src/psi/models/full_duplex_swa.pyConfiguration14.9 KB —
src/psi/models/interaction_arch/__init__.pyConfiguration77 B —
src/psi/models/interaction_arch/control_metrics.pyConfiguration5.2 KB —
src/psi/models/interaction_arch/control_tokens.pyConfiguration13.5 KB —
src/psi/models/interaction_arch/coupled_joint.pyConfiguration23.4 KB —
src/psi/models/interaction_arch/coupled_tail_cross.pyConfiguration15.7 KB —
src/psi/models/interaction_arch/loading.pyConfiguration1.0 KB —
src/psi/models/interaction_arch/runtime.pyConfiguration22.1 KB —
src/psi/models/interaction_arch/shared_av.pyConfiguration12.7 KB —
src/psi/models/interaction_arch/shared_prefix.pyConfiguration24.0 KB —
src/psi/models/motion_distill.pyConfiguration13.8 KB —
src/psi/models/psi0.pyConfiguration161.9 KB —
src/psi/models/psi0_rtc_test.pyConfiguration5.5 KB —
src/psi/models/relative_instruction_memory.pyConfiguration3.0 KB —
src/psi/models/swa_qa_branch.pyConfiguration3.4 KB —
src/psi/runtime/__init__.pyConfiguration1.1 KB —
src/psi/runtime/local_metrics.pyConfiguration1.2 KB —
src/psi/runtime/motion_rollout.pyConfiguration9.1 KB —
src/psi/runtime/native_streaming_speech.pyConfiguration50.2 KB —
src/psi/runtime/overlay_cases.pyConfiguration343 B —
src/psi/runtime/qa_private_kv.pyConfiguration3.7 KB —
src/psi/runtime/speech.pyConfiguration12.3 KB —
src/psi/runtime/swa_rollout.pyConfiguration7.2 KB —
src/psi/runtime/swa_speech.pyConfiguration8.3 KB —
src/psi/tokenizer/__init__.pyConfiguration54 B —
src/psi/trainers/__init__.pyConfiguration43 B —
src/psi/trainers/act_g1.pyConfiguration12.4 KB —
src/psi/trainers/diffusion_policy_g1.pyConfiguration13.4 KB —
src/psi/trainers/distributed_flow_sampling.pyConfiguration15.1 KB —
src/psi/trainers/distributed_flow_sampling_test.pyConfiguration5.0 KB —
src/psi/trainers/finetune.pyConfiguration25.5 KB —
src/psi/trainers/posttrain.pyConfiguration25.0 KB —
src/psi/trainers/posttrain_full_duplex.pyConfiguration312.3 KB —
src/psi/trainers/posttrain_v1.pyConfiguration88.9 KB —
src/psi/trainers/posttrain_v1_audio_dance.pyConfiguration367 B —
src/psi/trainers/posttrain_v1_motion_distill.pyConfiguration22.2 KB —
src/psi/trainers/posttrain_v1_rtc_test.pyConfiguration5.9 KB —
src/psi/trainers/posttrain_v1_trimodal.pyConfiguration110 B —
src/psi/trainers/pretrain.pyConfiguration15.6 KB —
src/psi/trainers/qwen2_5omni_mixin.pyConfiguration30.4 KB —
src/psi/trainers/qwen3vl_mixin.pyConfiguration22.9 KB —
src/psi/trainers/stage2_replay.pyConfiguration7.5 KB —
src/psi/trainers/staged_full_duplex.pyConfiguration40.0 KB —
src/psi/trainers/swa_optimizer.pyConfiguration1.2 KB —
src/psi/trainers/swa_validation.pyConfiguration4.4 KB —
src/psi/trainers/trainer.pyConfiguration30.1 KB —
src/psi/trainers/trainer_performance_test.pyConfiguration1.1 KB —
src/psi/utils/__init__.pyConfiguration65 B —
src/psi/utils/overwatch.pyConfiguration5.8 KB —
src/psi/utils/utils.pyConfiguration16.7 KB —
tests/test_action_input_precision.pyConfiguration6.9 KB —
tests/test_action_observation_positions.pyConfiguration6.8 KB —
tests/test_build_hfd_gqa_scale.pyConfiguration590 B —
tests/test_causal_audit_shards.pyConfiguration3.0 KB —
tests/test_causal_audit_supervisor.pyConfiguration2.7 KB —
tests/test_causal_av_packing.pyConfiguration8.8 KB —
tests/test_causal_video_qa.pyConfiguration3.1 KB —
tests/test_checkpoint_marker_atomicity.pyConfiguration1.5 KB —
tests/test_continuous_audio_command_sources.pyConfiguration5.1 KB —
tests/test_control_metrics_distributed.pyConfiguration1.6 KB —
tests/test_dlc_storage.pyConfiguration3.6 KB —
tests/test_export_hfd_overlay_assets.pyConfiguration1.8 KB —
tests/test_full_duplex_audio_command.pyConfiguration144.4 KB —
tests/test_full_duplex_audio_contract.pyConfiguration1.8 KB —
tests/test_full_duplex_continuous_cache.pyConfiguration5.0 KB —
tests/test_full_duplex_ddp_sync.pyConfiguration4.7 KB —
tests/test_full_duplex_streaming_infer.pyConfiguration31.3 KB —
tests/test_hf_audio_motion_handoff.pyConfiguration9.3 KB —
tests/test_hf_export_dashboard.pyConfiguration2.9 KB —
tests/test_hf_smplx_handoff.pyConfiguration2.9 KB —
tests/test_hfd_backchannel_inventory.pyConfiguration1.2 KB —
tests/test_hfd_continuation.pyConfiguration2.0 KB —
tests/test_hfd_launchers.pyConfiguration31.3 KB —
tests/test_hfd_multimodal_overlay.pyConfiguration4.4 KB —
tests/test_hfd_staged_training.pyConfiguration18.1 KB —
tests/test_hfd_storage_paths.pyConfiguration1.8 KB —
tests/test_hfd_training_recipe.pyConfiguration17.3 KB —
tests/test_hfd_vqa_candidate_pilot.pyConfiguration3.0 KB —
tests/test_hfdv2_qwen_streaming.pyConfiguration179.5 KB —
tests/test_listen_hold_targets.pyConfiguration3.0 KB —
tests/test_live_multimodal_overlay.pyConfiguration7.9 KB —
tests/test_motion_frame_index.pyConfiguration7.8 KB —
tests/test_native_visual.pyConfiguration5.0 KB —
tests/test_office_background_candidates.pyConfiguration3.2 KB —
tests/test_production_a1_control.pyConfiguration13.8 KB —
tests/test_root40_production.pyConfiguration8.4 KB —
tests/test_speech_runtime_contract.pyConfiguration2.1 KB —
tests/test_stage2_single_pass.pyConfiguration9.8 KB —
tests/test_startup_index_cache.pyConfiguration5.1 KB —
tests/test_streaming_inference_freeze.pyConfiguration990 B —
tests/test_swa_alignment.pyConfiguration7.8 KB —
tests/test_swa_formal_integration.pyConfiguration14.6 KB —
tests/test_swa_manifest_contract.pyConfiguration4.3 KB —
tests/test_swa_native_speech.pyConfiguration3.3 KB —
tests/test_swa_pipeline_queue.pyConfiguration5.5 KB —
tests/test_swa_regressions.pyConfiguration3.9 KB —
tests/test_swa_render.pyConfiguration10.0 KB —
tests/test_swa_scaleup.pyConfiguration3.7 KB —
tests/test_swa_speech_failures.pyConfiguration1.0 KB —
tests/test_third_party_dataset.pyConfiguration4.5 KB —
tests/test_training_dashboard.pyConfiguration16.4 KB —
tests/test_training_storage_overlay.pyConfiguration3.0 KB —
tests/test_tts_codec_graph.pyConfiguration2.2 KB —
tests/test_unified_overlay_review.pyConfiguration4.9 KB —
tests/test_video_asset_admission.pyConfiguration5.8 KB —
tests/test_video_eight_voices.pyConfiguration5.4 KB —
tests/test_video_overlay.pyConfiguration4.1 KB —
tests/test_visual_dataset_bundle.pyConfiguration12.5 KB —
tests/test_vqa_review_dashboard.pyConfiguration8.4 KB —
AGENTS.mdDocumentation2.6 KB —
CHANGELOG.mdDocumentation439.8 KB —
DIALOG.mdDocumentation427.2 KB —
HF_MIGRATION_HANDOFF.mdDocumentation21.1 KB —
README.mdDocumentation4.5 KB —
backup/README.mdDocumentation1.0 KB —
benchmark/README.mdDocumentation13.0 KB —
benchmark/bones_seed_soma_to_smplx/HANDOFF.mdDocumentation10.1 KB —
benchmark/bones_seed_soma_to_smplx/README.mdDocumentation9.9 KB —
doc/2607182225-loss-bug.mdDocumentation19.7 KB —
doc/26072112-reuse6-task3131.mdDocumentation4.6 KB —
doc/2608121521-conclusion.mdDocumentation26.3 KB —
doc/260904-kv_memory.mdDocumentation14.9 KB —
doc/260921-任务划分与bench.mdDocumentation9.4 KB —
doc/260922-tmp/README.mdDocumentation2.6 KB —
doc/260922-tmp/future_experiments.mdDocumentation14.7 KB —
doc/260922-tmp/project_reproduction.mdDocumentation21.7 KB —
doc/260922-tmp/数据扩容进度.mdDocumentation6.7 KB —
doc/260928-data.mdDocumentation8.5 KB —
doc/261001-data.mdDocumentation6.3 KB —
doc/261001-原生视觉输入与特征缓存.mdDocumentation11.8 KB —
doc/261001-视觉数据重构.mdDocumentation103.8 KB —
doc/261001-视频来源与类型盘点.mdDocumentation8.1 KB —
doc/261005-架构实验阶段结论.mdDocumentation3.2 KB —
doc/261011-VQA数据需求与交付规范.mdDocumentation19.6 KB —
doc/EXPERIMENT.mdDocumentation75.6 KB —
doc/FULL_DUPLEX_TASK_RESEARCH.mdDocumentation8.2 KB —
doc/OMNI_DATA_PREPROCESS.mdDocumentation7.1 KB —
doc/Psi0-训练阶段-state-action总览.mdDocumentation6.6 KB —
doc/README.mdDocumentation1.1 KB —
doc/V26_ROOT_CAUSE_FIXES.mdDocumentation4.7 KB —
doc/experiment_setup.mdDocumentation6.4 KB —
doc/hf_audio_motion/CODE_README.mdDocumentation3.3 KB —
doc/hf_audio_motion/DATA_README.mdDocumentation9.9 KB —
doc/hf_audio_motion/PROGRESS.mdDocumentation4.8 KB —
doc/hf_audio_motion/SMPLX_HANDOFF.mdDocumentation7.9 KB —
doc/hf_audio_motion/UPLOAD.mdDocumentation4.8 KB —
doc/legacy/260716-omni-sqlite-recovery-manifest.mdDocumentation1.4 KB —
doc/legacy/psi0-notes/video_clip训练分析.mdDocumentation10.5 KB —
doc/legacy/psi0-preprocess-README.mdDocumentation594 B —
doc/plan_0.mdDocumentation12.9 KB —
doc/工作留痕.mdDocumentation24.1 KB —
doc/视频QA历史证据与长度匹配.mdDocumentation2.2 KB —
doc/视频VQA八音色扩充.mdDocumentation9.1 KB —
doc/视频VQA接入管线.mdDocumentation10.2 KB —
doc/视频数据扩充看板.mdDocumentation3.2 KB —
experiments/260908-三组实验结论.mdDocumentation16.4 KB —
experiments/260909-Stage1批量与步数.mdDocumentation9.6 KB —
experiments/260910-Stage2单遍与预取.mdDocumentation7.0 KB —
experiments/260910-speaking-while-acting.mdDocumentation198.5 KB —
experiments/260916-formal-runtime-contract.mdDocumentation2.7 KB —
experiments/260916-speech-latency-answer.mdDocumentation5.3 KB —
experiments/260918-swa-supervision-scaleup.mdDocumentation9.6 KB —
experiments/260919-swa-motion-audit.mdDocumentation8.0 KB —
experiments/260920-swa-body-alignment.mdDocumentation14.5 KB —
experiments/260922-visual-assets-audit.mdDocumentation5.8 KB —
experiments/260922-vqa-candidate-pilot.mdDocumentation3.8 KB —
experiments/260923-unified-multimodal-overlay.mdDocumentation24.7 KB —
experiments/260924-video-vqa-scale.mdDocumentation43.0 KB —
experiments/260927-unified-overlay-three-gpu.mdDocumentation7.3 KB —
experiments/260928-causal-audit-throughput.mdDocumentation4.9 KB —
experiments/HFDV2_QWEN_KV_ABLATION.mdDocumentation10.0 KB —
experiments/HFD_FUSED_QWEN_SERIAL.mdDocumentation3.9 KB —
paper_context_v4/00_project_overview.mdDocumentation2.0 KB —
paper_context_v4/01_code_structure.mdDocumentation2.0 KB —
paper_context_v4/02_method_architecture.mdDocumentation15.7 KB —
paper_context_v4/03_data_pipeline.mdDocumentation13.6 KB —
paper_context_v4/04_training_objective.mdDocumentation11.6 KB —
paper_context_v4/05_streaming_inference.mdDocumentation9.3 KB —
paper_context_v4/06_robot_deployment.mdDocumentation3.8 KB —
paper_context_v4/07_experiments_and_metrics.mdDocumentation7.9 KB —
paper_context_v4/08_code_to_claim_mapping.mdDocumentation2.6 KB —
paper_context_v4/09_architecture_figure_notes.mdDocumentation4.9 KB —
paper_context_v4/10_potential_contributions.mdDocumentation1.4 KB —
paper_context_v4/11_open_questions.mdDocumentation1.9 KB —
paper_context_v4/12_reproduction_and_verification.mdDocumentation4.8 KB —
paper_context_v4/13_root40_and_fp32.mdDocumentation6.2 KB —
paper_context_v4/README.mdDocumentation2.8 KB —
paper_context_v4/model_architecture_deep_dive.mdDocumentation46.0 KB —
scripts/data/review_dashboard/README.mdDocumentation6.5 KB —
FULL_SNAPSHOT.tar.zst.part-0000Other21.5 GB 94f4ffb7a741
FULL_SNAPSHOT.tar.zst.part-0001Other21.5 GB 2fb6e94fc1cf
FULL_SNAPSHOT.tar.zst.part-0002Other21.5 GB 945f4bb162cf
FULL_SNAPSHOT.tar.zst.part-0003Other21.5 GB bf36edb7dde4
FULL_SNAPSHOT.tar.zst.part-0004Other21.5 GB 5a9d576cf33a
FULL_SNAPSHOT.tar.zst.part-0005Other21.5 GB 5c415a9560e5
FULL_SNAPSHOT.tar.zst.part-0006Other20.4 GB 38f274b742e1
backup/prepare_full_duplex_continuous_audio.shOther11.5 KB —
backup/prepare_omni_training_data.shOther50.8 KB —
benchmark/bones_seed_soma_to_smplx/run_convert.shOther844 B —
benchmark/requirements-eval.txtOther288 B —
benchmark/run.shOther3.0 KB —
doc/260922-tmp/hfd_architecture_fallback.svgOther4.6 KB —
doc/260922-tmp/hfd_architecture_gpt-image-2.prompt.txtOther1.3 KB —
doc/260922-tmp/hfd_architecture_previous_swa.pngOther1.3 MB 604ccba02b35
doc/260922-tmp/hfd_architecture_previous_swa.prompt.txtOther5.0 KB —
doc/260922-tmp/hfd_architecture_previous_swa_kv.pngOther1.4 MB 6b8a1cd63b87
doc/260922-tmp/hfd_architecture_previous_swa_kv.prompt.txtOther3.7 KB —
doc/OmniHM_Zero__arxiv_1012.pdfOther56.7 MB a483f433ffbe
doc/gpt_doc/paper_context_261003.tar.gzOther47.2 KB cd558d359d27
doc/gpt_doc/paper_context_v2_261004.tar.gzOther83.5 KB ec18d8b20e8c
doc/gpt_doc/paper_context_v3_261007.tar.gzOther64.8 KB 86bdc670441b
doc/gpt_doc/paper_context_v4_261008.tar.gzOther73.1 KB 6323762a6748
doc/legacy/psi0-notes/codex.txtOther11.1 KB —
doc/legacy/psi0-notes/多数据集接入说明.txtOther5.5 KB —
doc/legacy/psi0-notes/换vlm和t2m.txtOther4.3 KB —
doc/legacy/psi0-notes/接入seed数据.txtOther9.0 KB —
doc/legacy/psi0-notes/推理修改方案.txtOther7.4 KB —
doc/legacy/psi0-notes/数据层面补充文件.txtOther4.7 KB —
doc/legacy/psi0-notes/是否dit.txtOther1.9 KB —
doc/legacy/psi0-notes/纯text输入.txtOther5.1 KB —
figures/260911-hfd-swa-architecture-kv.pngOther1.4 MB 6b8a1cd63b87
figures/260911-hfd-swa-architecture-kv.prompt.txtOther3.7 KB —
figures/260911-hfd-swa-architecture.pngOther1.3 MB 604ccba02b35
figures/260911-hfd-swa-architecture.prompt.txtOther5.0 KB —
infer_render_auto.shOther5.2 KB —
paper_context_v4/architecture.mmdOther1.5 KB —
posttrain-full-duplex-audio-command.shOther4.9 KB —
pyproject.tomlOther1.8 KB —
scripts/activate_psi_env.shOther507 B —
scripts/data/calc_dance_audio_textop_stats.shOther959 B —
scripts/data/calc_seed_all_textop_stats.shOther646 B —
scripts/data/calc_seed_textop_stats.shOther574 B —
scripts/data/complete_video_eight_voices.shOther2.7 KB —
scripts/data/complete_video_vqa.shOther1.2 KB —
scripts/data/complete_video_vqa_with_keeper.shOther1.6 KB —
scripts/data/prepare_full_duplex_white_noise.shOther6.0 KB —
scripts/data/review_dashboard/app.jsOther10.0 KB —
scripts/data/review_dashboard/index.htmlOther3.5 KB —
scripts/data/review_dashboard/progress.htmlOther1.6 KB —
scripts/data/review_dashboard/progress.jsOther2.1 KB —
scripts/data/review_dashboard/style.cssOther4.9 KB —
scripts/deploy/offline_psi0_sonic_motion_tracking.shOther1.8 KB —
scripts/deploy/online_psi0_sonic_bridge.shOther1.7 KB —
scripts/deploy/run_psi0_sonic_online_jog.shOther3.4 KB —
scripts/deploy/run_psi0_sonic_online_jog_planner.shOther2.5 KB —
scripts/deploy/run_sonic_sim_record_online.shOther1.1 KB —
scripts/deploy/serve_psi0-rtc-dexmate.shOther364 B —
scripts/deploy/serve_psi0-rtc-sonic.shOther2.0 KB —
scripts/deploy/serve_psi0-rtc.shOther372 B —
scripts/deploy/serve_psi0_simple.shOther522 B —
scripts/deploy/start_sonic_wbc_zmq_manager_no_planner.shOther1.0 KB —
scripts/deploy/start_sonic_wbc_zmq_manager_planner.shOther712 B —
scripts/deploy_real.tar.gzOther69.0 KB 04d6dbd8f660
scripts/deploy_real/g1_real_state_writer.shOther885 B —
scripts/deploy_real/online_psi0_sonic_bridge_real.shOther2.6 KB —
scripts/deploy_real/run_psi0_sonic_online_jog.shOther2.2 KB —
scripts/deploy_real/serve_psi0-rtc-sonic.shOther1.6 KB —
scripts/deploy_real/start_sonic_wbc_zmq_manager_no_planner_real.shOther1.4 KB —
scripts/infer/run_audio_case.shOther18.4 KB —
scripts/infer/watch_swa_visual_qa.shOther1.3 KB —
scripts/setup_autodl_audio_motion.shOther1.7 KB —
scripts/train_with_gpu_monitor.shOther1.3 KB —
train_dlc.shOther568 B —
uv.lockOther316.3 KB —
.gitattributesRepository1.9 KB —
.gitignoreRepository688 B —
.gitmodulesRepository101 B —
assets/.gitignoreRepository6 B —
benchmark/.gitignoreRepository64 B —
src/psi/config/tokenizer.pyTokenizer2.5 KB —
src/psi/tokenizer/fast_action_tokenizer.pyTokenizer3.2 KB —

License and Download

License
Not stated by the source
Access
Open weights, no gate
Download from Tinghong Ye

Released by Tinghong Ye through its official repository on Hugging Face.