Search public pages, research tools, and SAVRN solutions.
jef-v0 is an open-weight model for text classification from Lee, released under Apache License 2.0. Its published files total 438.5 KB.
The trained part of JEF, an open System One decision engine: typed questions in, probability distributions with calibrated confidence out, no text generated. This repository is not a standalone model.
By Lee, published under apache-2.0, revision c2e46a52ca51.
The trained part of JEF, an open System One decision engine: typed questions in, probability distributions with calibrated confidence out, no text generated. This repository is not a standalone model. It holds a bilinear decision head and a fitted calibrator that sit on top of a frozen jhu-clsp/mmBERT-base. That split is why it is ~400KB instead of ~800MB, and why it trains on CPU. Out-of-sample, on a split grouped by evidence rather than by sample — one CVE yields several questions over an identical description, and splitting per sample puts that text on both sides of the line. Test set: 1397 samples. On CTI-Bench VSP, with every training CVE excluded, attack vector reaches 0.840 against a…
The trained part of JEF, an open System One decision engine: typed questions in, probability distributions with calibrated confidence out, no text generated.
This repository is not a standalone model. It holds a bilinear decision head
and a fitted calibrator that sit on top of a frozen jhu-clsp/mmBERT-base. That split is
why it is ~400KB instead of ~800MB, and why it trains on CPU.
pip install "jef[torch]"
from jef_sdk import Jef, choice
jef = Jef(
"jhu-clsp/mmBERT-base",
head="hf://dennislee928tw/jef-v0/head.npz",
calibration="hf://dennislee928tw/jef-v0/calibration.json",
)
jef.evaluate(alert, {"team": choice("誰處理？", soc="監控", infra="基礎設施")})
Out-of-sample, on a split grouped by evidence rather than by sample — one CVE yields several questions over an identical description, and splitting per sample puts that text on both sides of the line. Test set: 1397 samples.
| bucket | n | accuracy | macro-F1 | ECE | temperature | P(correct) ECE |
|---|---|---|---|---|---|---|
choice:14 |
64 | 0.688 | 0.592 | 0.153 | 1.16 | unavailable |
choice:4 |
423 | 0.787 | 0.520 | 0.072 | 1.13 | 0.052 |
noul:2 |
487 | 0.813 | 0.808 | 0.035 | 1.15 | 0.041 |
score:5 |
423 | 0.570 | 0.460 | 0.094 | 1.01 | 0.099 |
soar_zhOn CTI-Bench VSP, with every training CVE excluded, attack vector reaches 0.840 against a 0.250 random baseline and user interaction 0.805 against 0.500. Conformal coverage holds at 0.895 against a nominal 0.900 there.
On TMMLU+ it scores 0.254 against a 0.250 baseline — chance. JEF judges evidence you supply; it does not know things. That boundary is stated because finding it in production is worse.
confidence is not correctnessconfidence is (n × peak − 1) / (n − 1): how peaked the distribution is. A
model can be decisive and wrong, which is why most of this ecosystem ships the
disclaimer "confidence measures concentration, not correctness".
calibration.json carries three things fitted on a held-out split: temperature
scaling per (kind, n_options) bucket, split-conformal quantiles, and an
isotonic map from confidence onto observed correctness. That last one is what
makes p_correct meaningful — and it is absent for buckets with too few
calibration points, which is deliberately distinguishable from a low value.
choice:4 at alpha=0.10: 0.861 against a nominal 0.90Split conformal guarantees coverage under exchangeability. Grouping by evidence puts different concepts in calibration and test, which breaks that assumption. Do not gate automated actions on prediction sets from these buckets.
Labels come from taxonomies, not from a language model: MITRE ATT&CK states which tactic each technique serves, and a CVSS vector states the attack vector, the privileges required and the severity band. Nothing here derives from any commercial model's output.
p_correct is
unavailable, the conformal set excludes nothing, and every scene gate routes
to a human. That is the design, not a failure.Apache-2.0. Full methodology, including a contaminated run that scored 1.000 and
why it was wrong, in
docs/RESULTS.md.
5 files, 438.5 KB in total. The weights are 1 file totalling 394.5 KB in npz.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| head.npz | Weights | 394.5 KB | 202b98571b09 |
| calibration.json | Configuration | 4.6 KB | — |
| report.json | Configuration | 33.8 KB | — |
| README.md | Documentation | 4.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
Released by Lee through its official repository on Hugging Face. Read the license.
| Precision | Weights in memory |
|---|---|
| As published | 394.5 KB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Yes. jef-v0 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
This is a RoBERTa-base model trained on ~124M tweets from January 2018 to December 2021, and finetuned for sentiment analysis with the TweetEval benchmark. The original Twitter-based RoBERTa model can be found here and the original reference paper is TweetEval. This model is suitable for English. 0 -> Negative; 1 -> Neutral; 2 -> Positive This sentiment analysis model has been integrated into TweetNLP. You can access the demo here.
FinBERT is a BERT model pre-trained on financial communication text. The purpose is to enhance financial NLP research and practice. It is trained on the following three financial communication corpus. The total corpora size is 4.9B tokens. More technical details on FinBERT: Click Link This released finbert-tone model is the FinBERT model fine-tuned on 10,000 manually annotated (positive, negative, neutral) sentences from analyst reports. This model achieves superior performance on financial tone analysis task. If you are simply interested in using FinBERT for financial tone analysis, give it a try. If you use the model in your academic work, please cite the following paper: Huang, Allen H.…
https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-6-v2 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).
This is a multilingual XLM-roBERTa-base model trained on ~198M tweets and finetuned for sentiment analysis. The sentiment fine-tuning was done on 8 languages (Ar, En, Fr, De, Hi, It, Sp, Pt) but it can be used for more languages (see paper for details). This model has been integrated into the TweetNLP library.
The huggingface models currently give different results to the detoxify library (see issue here). For the most up to date models we recommend using the models from https://github.com/unitaryai/detoxify Trained models & code to predict toxic comments on 3 Jigsaw challenges: Toxic comment classification, Unintended Bias in Toxic comments, Multilingual toxic comment classification. Built by Laura Hanu at Unitary, where we are working to stop harmful content online by interpreting visual content in context. Score not directly comparable since it is obtained on the validation set provided and not on the test set. To update when the test labels are made available. It is also noteworthy to mention…
With this model, you can classify emotions in English text data. The model was trained on 6 diverse datasets (see Appendix below) and predicts Ekman's 6 basic emotions, plus a neutral class: 1) anger 2) disgust 3) fear 4) joy 5) neutral 6) sadness 7) surprise The model is a fine-tuned checkpoint of DistilRoBERTa-base. For a 'non-distilled' emotion model, please refer to the model card of the RoBERTa-large version. a) Run emotion model with 3 lines of code on single text example using Hugging Face's pipeline command on Google Colab: b) Run emotion model on multiple examples and full datasets (e.g.,.csv files) on Google Colab: Please reach out to [email protected] if you have any…