# Browser LLM Bench

Compare browser LLM inference speed, latency and resource usage. Explore measured reports and run a private benchmark on your own device.

Canonical page: https://browserllm.dev/

## Interpretation

Measurements describe an exact model artifact, quantization, runtime, browser, device, workload and cache state. Performance medians exclude warmup. Tokenizers and actual output lengths differ across models. These tests do not measure intelligence or task quality; no overall model score is assigned. Missing telemetry is unavailable, never zero. CPU is dedicated Chrome process-tree capacity; Chrome GPU time is experimental driver-accounted time including rendering, not GPU utilization. Power is whole-host subsystem power, not model-only power. Browser memory is summed process footprint, not deduplicated RAM.

## Run protocol

The primary comparison loads one model once and runs one warmup followed by 60 measured requests in three windows of 20. Each request starts immediately after the previous response completes. There are 90-second loaded-idle gaps after windows one and two. After the third window: 30 seconds loaded idle, unload, then 15 seconds observation. Conversation history resets between requests. The windows share one session and are not independent trials. Older reports can use different schedules: read each report's saved protocol before comparing.

## Published comparison

60 requests · 3 × 20 · 90 s idle

| Model | Encoding | Generation (tok/s) | First output (ms) | Response (s) | Peak Chrome footprint (GB) | Completed |
| --- | --- | --- | --- | --- | --- | --- |
| [Llama 3.2 1B](https://browserllm.dev/models/llama-3-2-1b/) | q4f16_1 | 163.55 | 26.25 | 0.42 | 3.82 | 1 |
| [Qwen2.5 0.5B](https://browserllm.dev/models/qwen2-5-0-5b/) | q4f16_1 | 113.83 | 40.15 | 0.60 | 2.70 | 1 |
| [Qwen2.5 1.5B](https://browserllm.dev/models/qwen2-5-1-5b/) | q4f16_1 | 79.06 | 40.00 | 0.84 | 4.05 | 1 |
| [Qwen2.5 3B](https://browserllm.dev/models/qwen2-5-3b/) | q4f16_1 | 56.78 | 55.55 | 1.17 | 5.25 | 1 |
| [Mistral 7B Instruct v0.2](https://browserllm.dev/models/mistral-7b-v0-2/) | q4f16_1 | 51.88 | 223.00 | 1.44 | 7.08 | 1 |
| [Mistral 7B Instruct v0.3](https://browserllm.dev/models/mistral-7b-v0-3/) | q4f16_1 | 51.11 | 164.95 | 1.41 | 7.12 | 1 |
| [Qwen2.5 7B](https://browserllm.dev/models/qwen2-5-7b/) | q4f16_1 | 50.37 | 94.70 | 1.35 | 9.08 | 1 |
| [Qwen3 1.7B](https://browserllm.dev/models/qwen3-1-7b/) | q4f16_1 | 47.15 | 55.85 | 1.30 | 4.78 | 1 |
| [Qwen3 0.6B](https://browserllm.dev/models/qwen3-0-6b/) | q4f16_1 | 38.70 | 52.85 | 1.35 | 3.73 | 1 |
| [Qwen3 4B](https://browserllm.dev/models/qwen3-4b/) | q4f16_1 | 29.63 | 102.55 | 2.10 | 6.63 | 1 |

[Measurement methodology](https://browserllm.dev/seo/guides/methodology.md)