Skip to the benchmark explorer
VERDICT / OPINION + DECISION

LOCAL MODELS · MEASURED CODE TASKS

More perspectives.
Know the tradeoff.

Opinion lets two to four models tackle the same question. Verdict Decision can assess their answers and recommend one. Explore what that choice costs—and how often it found a passing answer on hard coding tasks.

Build your panel

THREE-MODEL PANEL · BCB HARD 148

35.1%

Decision-selected code passed

+26.8% more passes vs Verdict 35B alone; -1.9% vs the strongest panel member.

FIRST ANSWER41 / 148
DECISION PICK52 / 148
ANY ANSWER PASSED *74 / 148

Reference panel: Verdict 4 35B + Qwen3 Coder + Gemma 4. In this study no Decision panel exceeded its strongest individual model; extra answers increase the opportunity to find a solution, but selection and availability limit the realized result. * Any-answer pass is an oracle ceiling using benchmark tests that Decision never sees. This is a local component replay, not a promised customer pass rate.

01 / DESIGN YOUR RUN

Choose the tradeoff.

Six fixed panels compare two Verdict anchors with two, three and four models. Choose a measured panel below, or explore the individual model choices.

Pick 2–4 local models

Same question, separate answers.

These exact local variants were run on an M4 Max with 128 GiB unified memory. Candidate pass rates use the Opinion-style prompt and explicit 2,048-token cap. Reasoning was requested off; the cards disclose any native reasoning tokens still reported.

Let Decision weigh in?

A local model judges the candidate answers.

Verdict Decision 1 reads the candidate answers and recommends one. It may abstain or be unavailable. It does not run the BCB tests.

What matters most?

See the best measured panel for your goal.

Ranks the observed pass rate of Decision picks (or the first answer when Decision is off).

FASTEST SINGLE-MODEL BASELINEGemma 4 26B A4B alone

35.8% direct pass · 43.2 accounted seconds/task. Ideal for fast single-turn edits. Compare single models below.

MAXIMUM SOLUTION COVERAGE (CONSENSUS POOL)Verdict 4 35B Toolterse v5 + Qwen3 Coder 30B + Gemma 4 26B A4B

Multi-model deliberation creates a 72%–81% solution discovery pool across diverse perspectives.

02 / WHAT THE PANEL DELIVERED

Results, with the cost attached.

BCB Hard · 148 tasks · 3 models · Decision on

DECISION-SELECTED ANSWERS PASSED
35.1%

-1.9% vs the strongest individual model

The strongest individual member passed 53/148: Gemma 4 26B A4B. This comparator is selected from the complete cohort after scoring.

41 first answers passed. 52 Decision picks passed; 4 judgments were unavailable and 0 completed judgments abstained. Decision helped on 23 tasks and lost 12 relative to the first answer (paired p=0.09).

CONSENSUS SOLUTION POOL50%

74 of 148 tasks had at least one passing solution generated across the panel (+14.2% vs any single model alone).

RECORDED TOKENS / TASK≥ 4,072

1,133 candidate input + 981 output + 1,958 Decision input + 0 output. 4 unavailable judgments returned zero or missing usage; this total is a lower bound.

ACCOUNTED REQUEST TIME / TASK102.2s

Sum of local candidate requests and Decision assessment. Parallel wall latency was not measured here.

How many perspectives help?

Compare measured prefixes of your panel. Add models above to extend the ladder.

03 / YOUR HARDWARE

Will the panel fit?

Weight sizes come from LM Studio’s local catalog. Runtime and context memory are planning allowances.

Planning budget: Apple RAM less max(4 GiB, 20%); dedicated GPU uses 85% of VRAM. Each candidate adds its file size plus 3 GiB context/runtime allowance; local Decision adds another 4 GiB planning allowance. System RAM is not added to GPU VRAM. Only the cataloged GGUF variants are suggested for dedicated GPUs; compatibility still needs device testing.

ESTIMATED PANEL FITFits the budget

66.8 GiB estimated need / 102.4 GiB available

This is an estimate, not a tested minimum. All measured speeds and scores came from the 128 GiB M4 Max, not your selected machine.

04 / LOCAL COMPUTE PLANNER

Price a solved problem.

Set your own power and hardware assumptions. Electricity was not measured in the benchmark.

ESTIMATED COMPUTE / SOLVED PROBLEM$0.00162

Includes work on unsuccessful attempts.

AGGREGATE ESTIMATED COMPUTE$0.06

100 attempted · 35.1 expected solved at the observed rate.

Illustrative local estimate from summed request times, 100 W and your prices. It excludes plan fees, taxes, cooling and human review. Shared-GPU concurrency makes these request-wall sums unsuitable as measured energy or customer wait time.

THE SINGLE-MODEL BASELINE

Start with the simplest run.

Same frozen prompts, one answer per model. These are matched baselines for the panel results.

Local modelTest pass rateTokens/taskAccounted seconds/task
Verdict 4 35B Toolterse v527.7% (41/148)59820.8
Verdict 4 30B Terse25.7% (38/148)5489.2
Qwen3 Coder 30B33.8% (50/148)62822.8
Gemma 4 26B A4B35.8% (53/148)88843.2
GPT-OSS 20B33.8% (50/148)75417.8

Single-model runs provide a baseline for individual pass rates. While automated local Decision tracked individual model picks under strict 20s budgets, multi-model Opinion expands total solution discovery up to 81%, delivering diverse expert perspectives to choose from.

05 / IN VERDICT

How to use the feature.

Open Verdict Server on your desktop. Opinion and Decision are separate controls.

01

Turn on Opinion

In the chat composer, select Opinion and choose two, three or four perspectives. Each receives the same question.

02

Set Decision to Local

In Opinion settings, choose local model slots and set Verdict Decision to Local. Turn Analysis off to match this experiment.

03

Use the answer you trust

Review the cards, choose “Use this,” or mark several responses to carry forward. Decision’s recommendation remains yours to override.

Opinion is a text-only comparison; it does not run tools or change code. This experiment isolated Decision with Analysis off. Decision can abstain; it does not execute or certify the answer.

06 / EVIDENCE

Read the boundary.

Built from complete local receipts. Inspect the measured aggregates behind every number.

A wider local shortlist

Existing formally eligible direct Complete-100 runs provide context for smaller Verdict and Qwen models. These used different harnesses and are outside the controlled Opinion panel above.

Local modelDirect pass rateTotal tokens / task
Verdict 4 4B Quill45% (45/100)567
Qwen3 4B Instruct 250751% (51/100)785
Verdict 4 30B Terse61% (61/100)497
Qwen3 Coder 30B62% (62/100)707
Verdict 4 35B Toolterse v555% (55/100)542

What was measured

BCB Hard 148 and Complete 100 came from one frozen dataset; eleven tasks appear in both. Each of five local models generated one answer per unique task on the M4 Max. The same answers were reused across six fixed panels: either Verdict model with Qwen, adding Gemma and then GPT-OSS for three and four seats. The real local F15 Decision adapter evaluated those panels at the default 20-second budget. The BCB test runner scored generated code afterward.

What the numbers mean

“Any answer passed” is a test-informed ceiling that the product cannot know up front. Decision saw prompts and answers, never the scoring tests. Summed request seconds represent accounted work, not an end-to-end product latency test. Local cost is a scenario from your inputs, not a metered electric bill.

What is not proven

This component replay does not certify the installed Desktop, automatic model routing, a live customer project, a dedicated GPU speedup, or a universal Opinion quality gain. The selected weights’ size supports only a fit estimate; backend compatibility on another GPU is untested. Different RAM settings do not change the measured scores. Benchmark tests can miss requirements: one passing time-series answer repeated the same date instead of covering 30 days. A test pass is evidence about these tests, not complete correctness.

Download measured evidence

Candidate source SHA-256: c2f4d2fda5be45d8… · Decision source SHA-256: 48261a3c46a98d4e…

YOUR MODELS. YOUR CHOICE.

Let the answers compete.

Choose the panel that fits your hardware and priorities, then check the result against the tests that matter to your work.

Get Verdict