embeddinggemma2-global-birds-2000-litert

EmbeddingGemma 2 — Global 2,000 Bird Species On-Device (LiteRT .tflite + WebGPU .onnx) & Binary Index

English | 한국어

English Overview

This repository provides the knowledge-distilled, ultra-compact on-device encoders (LiteRT .tflite and ONNX Runtime WebGPU .onnx) and precomputed Float32 binary vector databases (.f32) distilled from the full 740M teacher model (jaichang/embeddinggemma2-global-birds-2000).

Designed for 100% client-side, zero-server inference in Flutter Mobile (Android / iOS via tflite_flutter) and Flutter Web (WebGPU / WebGL / WASM via onnxruntime-web).

Knowledge Distillation Architecture & Accuracy

Instead of downloading the 1.2 GB model.safetensors onto a phone or browser, we trained lightweight student networks to directly regress the exact 768-dimensional L2-normalized embedding coordinates of EmbeddingGemma 2 (740M) using Cosine + InfoNCE Distillation Loss:

File Inventory

Quick Start (ONNX Runtime in Python / Web)

import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download

repo_id = "jaichang/embeddinggemma2-global-birds-2000-litert"
onnx_path = hf_hub_download(repo_id=repo_id, filename="embeddinggemma2_vision_768d.onnx")
f32_path = hf_hub_download(repo_id=repo_id, filename="global_2000_birds_photo_index.f32")
cat_path = hf_hub_download(repo_id=repo_id, filename="global_2000_birds_catalog.json")

# 1. Load catalog & flat Float32 vector index
catalog = json.load(open(cat_path, encoding="utf-8"))["species"]
photo_index = np.fromfile(f32_path, dtype=np.float32).reshape(len(catalog), 768)

# 2. Preprocess image to [1, 3, 160, 160] in [0, 1]
img = Image.open("bird.jpg").convert("RGB").resize((160, 160))
x = (np.asarray(img, dtype=np.float32) / 255.0).transpose(2, 0, 1)[None, ...]

# 3. Run distilled ONNX encoder & cosine similarity search
sess = ort.InferenceSession(onnx_path)
emb = sess.run(None, {"image_rgb": x})[0][0]
sims = photo_index @ emb
top1 = int(np.argmax(sims))
print(f"#1: {catalog[top1]['en_name']} ({catalog[top1]['kr_name']}) — similarity={sims[top1]:.4f}")

한국어 안내 (Korean Overview)

본 모델(jaichang/embeddinggemma2-global-birds-2000, 740M)의 768차원 임베딩 공간을 지식증류(Knowledge Distillation)하여, Flutter 모바일 앱(tflite_flutter)과 Flutter Web 브라우저(WebGPU / WebGL / WASM)에서 서버 없이 100% 온디바이스로 구동할 수 있도록 경량화한 전용 레포지토리입니다.

지식증류(Knowledge Distillation) vs 무증류 원본 타워(Un-distilled 740M Tower)

1.2GB에 달하는 전체 model.safetensors 중 티어 1(초경량 지식증류 학생 모델, 합계 23.9 MB)과 티어 2(무증류 원본 740M 파인튜닝 서브타워 INT8 직변환, 합계 53.3 MB) 두 가지를 모두 제공합니다: - 티어 1 — 지식증류 학생 모델 (embeddinggemma2_{vision,audio}_768d.onnx/.tflite, 합계 23.9 MB): 비전 Top-1 100.0%, 오디오 Top-1 99.1%, WebGPU 초고속 추론(~30-80ms) - 티어 2 — 무증류 원본 740M 파인튜닝 타워 (embeddinggemma2_undistilled_{vision,audio}_int8.onnx/.tflite, 합계 53.3 MB): model.safetensors 내 실제 Gemma4Vision Transformer(22.38 MB) 및 Gemma4Audio Conformer(30.90 MB) 가중치를 지식증류 없이 그대로 추출하여 INT8 양자화한 원본 서브타워 모델

파일 구성