Veylaro
Sign in Download
Benchmarks

Evidence follows the artifact.
No inherited scores.

Results belong to an exact model, runtime, and task set. Historical runs remain available for inspection; they do not establish scores for today's downloadable bundles. We separate internal project checks from standardized benchmarks.

Historical runs are preserved below with their recorded model identities. They are not scores for the current downloadable GGUF bundles.

Historical HumanEval · 164 tasks · one completion per task
mlx-community/gemma-3-text-4b-it-4bit (base)113/164 passed · per-task report · historical
68.9%
gemma4:12b via Ollama149/164 passed · per-task report · historical
90.9%
Current downloadable models · qualification pending
Laro lite: unsloth/gemma-3-4b-it-GGUF · Q4_K_M. historical MLX run exists; equivalence to this GGUF/runtime has not been established.
Laro med: unsloth/gemma-3-12b-it-GGUF · Q4_K_M. no complete release-attributed standardized score.
Laro max: unsloth/gemma-3-27b-it-GGUF · Q4_K_M. no complete release-attributed standardized score; 24 GB minimum, 32 GB recommended.
Internal project checks · August 2026Recorded checks include 40/41 and 38/41 across four small projects, 16/16 CRM DOM checks, and one 15/15 snake-game app/harness comparison. Several project runs reached their 1,500-second limit.

Passing individual checks is not proof of prompt completion, production-ready tenant security, or parity across every app task. These small internal courses are not SWE-bench or Terminal-Bench scores.

Frontier comparison: no complete, release-attributed SWE-bench or Terminal-Bench result is available for these bundles.
Systems improvements: simulations are design hypotheses, not measured capability gains. Improvements require executed, controlled comparisons.

Model hashes, historical report hashes, and evidence limitations are in the evidence manifest.

A pending score is not zero and it is not an estimate. It means the exact release artifact has not yet produced reproducible task-level evidence on supported hardware.

The local advantage

Capability is measured.
Privacy is architectural.

Raw capability is half the story. The other half is what it costs you — in privacy, availability and control. Here, it isn't close.

0
bytes of project code sent by local inference — optional online services are separate
∞
usage on Pro — cloud plans throttle you weekly
0
cloud inference outages — your local runtime has no provider dependency
0
network round trips for local inference — generation latency still depends on hardware
VeylaroClaude CodeCodex / CopilotGemini CLI
Local model inferenceBuilt inCloudCloudCloud
Project code sent to model providerNoRequiredRequiredRequired
Core coding chat works offlineYesNoNoNo
Usage on Veylaro ProUnlimitedSee providerSee providerSee provider
Agentic multi-file editsYesYesYesYes
Verified precedent memory on deviceOpt inProvider dependentProvider dependentProvider dependent
First monthUnlimited, no cardSee providerSee providerSee provider
Your GPU does the workYes———

Product terms change. This table describes the intended Veylaro architecture and broad product categories, not a benchmark. Verify current competitor limits before purchasing.

Methodology

Measured, not marketed.

The release gate requires generated code to execute, incomplete output to count as a failure, and every result to retain model, runtime, evaluator, and artifact identity. Historical HumanEval runs satisfy only part of that lineage, so they remain research records and are not presented as current Laro capability.

How we label every number

  • Measured — the exact release artifact ran here and the raw task-level output is retained. Only this can become a Laro result.
  • Vendor — the lab's own published figure on their own harness.
  • Not published — no number exists we can cite. It stays blank rather than guessed.
  • Historical — a different checkpoint or runtime was measured. It informs engineering but is not inherited by a release tier.
  • Pending — the release artifact has not yet completed the required run. All three current tiers are here.

Run the benchmark that matters.

Your codebase, your tickets, your Friday-afternoon bug. Free tier included.