JEV COMPARISON · JEVBENCH V1.3.0 · UPDATED SEPTEMBER 22, 2026
Jev vs SemIf
The strongest open rebuild on the board, and where it still falls short.
On this site
Jev 1.13.0
TypeSafe · System One model
Nothing for you to run, and still ahead where the cases get hard.
Alternative
SemIf
TheoLeeCJ · Open logit reader on Qwen3.5-4B
Gets within 1.3 points of Jev on a graphics card you already own.
The benchmark
JevBench v1.3.0, measured the same way
One benchmark ran all 52 systems under one method, so these bars are comparable with each other in a way that vendor-published figures are not.
Composite score
Higher is better
JevBench v1.3.0
The top three finish within 1.4 points of each other, and then the board falls away sharply. The composite hides where systems actually differ, so the two charts below break it apart.
Accuracy by difficulty
Where the decisions separate
Easy
72 straightforward cases
Standard
96 everyday cases
Judge
146 evaluation-style calls
HardHard
220 genuinely ambiguous cases
Easy and standard decisions separate almost nothing. The hard tier is where these systems stop agreeing.
The missing fourth axis: cost
The composite also weighs cost, scored from each system's public list price. Jev places last on that axis, but the benchmark does not turn a single price into a recommendation: bursty traffic and idle GPU time change the economics of self-hosting.
See Jev pricingThe board
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and SemIf are highlighted.
JevBench v1.3.0 scored 52 systems on the same 534 decisions (72 easy, 96 standard, 146 judge, 220 hard) on September 21, 2026, one request at a time.
Feature by feature
Jev and SemIf side by side
What each is better at
The practical trade-off
SemIf is remarkably close in the aggregate. The meaningful difference is less about a leaderboard and more about who owns the infrastructure, the data, and the calibration work.
Where Jev wins
- Hard-tier accuracy: 74.1% against 59.5%.
- Calibration is stronger out of the box: 82.7 against 72.6.
- No GPU, serving stack, capacity planning, or idle cost to operate.
- 64k context per request and published production rate limits.
Where SemIf wins
- Judge-tier accuracy: 95.2% against Jev's 94.5%.
- MIT code you can read, fork, audit, and run on controlled weights.
- No per-call price once a busy GPU is already paid for.
- Offline operation, including a WebGPU browser demo.
The analysis
The gap is concentrated, not universal
Give SemIf its due: 73.1 against Jev's 74.4 in JevBench v1.3.0, second on the board, and ahead on the 146-decision judge tier.
The gap is concentrated in two places. On 220 hard decisions Jev scores 74.1% and SemIf 59.5%, a 14.6-point spread. On calibration Jev scores 82.7 against 72.6 because its probabilities are trained for the decision interface rather than read directly from a frozen model.
The third gap is operational. SemIf's low per-decision estimate assumes a rented GPU that stays busy. Idle GPUs bill the same as busy ones, so bursty traffic moves that number in a hurry.
Data boundary
SemIf stays inside your network; Jev sends requests to a hosted API.
Operating model
SemIf trades API spend for GPU ownership, serving, monitoring, and calibration.
Decision shape
Both are strongest when software needs a typed choice, score, or no/yes judgment.
Open implementation
What SemIf actually is
SemIf takes an unstructured state, criteria written at request time, and a set of typed options. It runs a forward pass through an open model and reads native logits for those options instead of generating an answer sentence, parsing JSON, and validating it afterward.
The primary benchmark configuration uses Qwen3.5-4B. The project also supports Qwen3-0.6B, MiniCPM5-2B, and Qwen3-Reranker-4B. The code is MIT; upstream model weights keep their own licences.
Read SemIf on GitHubSelf-hosted
Run the model on your own GPU or rent one when volume justifies it.
Browser demo
Try a WebGPU build in the browser without a waitlist.
Direct logits
Return typed option probabilities without an autoregressive output loop.
Private by design
Keep state and decision criteria inside your network.
SemIf was previously called OpenJev. It is an independent project and is not affiliated with or endorsed by TypeSafe or Jev AI.
So which should you use?
Choose around your constraints
Pick Jev if
- Traffic is bursty, or you do not want to run a GPU serving stack.
- Inputs include long, noisy, or contested cases where the hard tier matters.
- You want thresholds that work on day one instead of after a calibration exercise.
Pick SemIf if
- You already own idle GPU capacity or can keep a rented one busy.
- The data legally cannot leave your network.
- Decisions are mostly easy, standard, or judge-tier and you can calibrate locally.
The honest comparison
Try Jev on the cases you are actually arguing about.
A scoreboard is useful for orientation. Your own awkward cases are the better test.
FAQ
Jev vs SemIf FAQ
Is SemIf the same as OpenJev?+
SemIf is the current name of the project formerly called OpenJev, by TheoLeeCJ. Other projects also use the openJev name and are unrelated.
Is SemIf as accurate as Jev?+
Close on most tiers and ahead on one. In JevBench v1.3.0 it matches Jev on easy decisions, trails slightly on standard, wins the judge tier 95.2% to 94.5%, and falls behind on hard decisions, 59.5% to 74.1%.
What hardware do I need to run SemIf?+
The published numbers use an RTX 3090 with a 4B model. Smaller supported models can run on less, and there is a WebGPU browser demo, but production throughput assumes GPU capacity.
Does SemIf give calibrated probabilities?+
It returns probabilities conditional on the options you supply. The project includes temperature calibration, and its README recommends fitting and validating probabilities on the workload where decisions will be made.
What does it cost to run SemIf?+
JevBench estimates roughly $0.022 per 1,000 decisions on a GPU that stays busy. That excludes idle GPU time and the engineering cost of running the stack yourself.
Figures are quoted from the public sources above as read on September 22, 2026. SemIf and every other product named on this page belongs to its respective owner.