Local LLM vs API cost: a 4B model on a 2019 Core i9
We ran Qwen3-4B on a 2019 Core i9 with no graphics card to sort 20 support emails: 5.8 seconds each, 17 of 20 categories right, a few cents per 1,000.
In short:
- A 4-billion-parameter model runs on a 2019 Intel Core i9 with no graphics card. It sorted a support email in 5.8 seconds on average.
- It put 17 of 20 test emails in the right category and got the order number right in all 20.
- How you build the software matters as much as the chip. The same processor wrote nearly three times faster with a build made for it than with a general-purpose package.
- On a 2020 MacBook Air M1 with 8 GB of memory, the same model ran on the built-in GPU and took 2.9 seconds per email, about twice as fast as the i9, with the same categories and urgency levels.
- Per 1,000 emails, the electricity costs at most about ₹2.4. Claude Haiku 5.5 would cost about 3.6 US cents for the same work.
- At this size, cost doesn’t settle it. Run the model yourself when the emails can’t leave your own hardware. Otherwise a cheap API model is faster and less work.
What we tested
We used one of our everyday work machines. It has no graphics card beyond the processor’s built-in one, so we wanted to know whether it could do useful AI work anyway.
| Machine | Intel Core i9-9980HK (8 cores, 16 threads, released 2019), 64 GB DDR4-3200 RAM, built-in graphics only, macOS 26 |
| Runtime | llama.cpp built from source for this CPU (commit e0949d0, 11 November 2026), CPU only |
| Model | Qwen3-4B-Instruct-2507, compressed to about 4 bits per weight (Q4_K_M), a 2.5 GB file |
| Task | Read a customer support email and return JSON: category, urgency, order number and a one-line summary |
| Test data | 20 support emails we wrote ourselves, each with the answer we expected |
Sorting the inbox before a person opens it is a common first workflow for small support teams. llama.cpp enforced a JSON schema, so every answer was valid JSON with one of five categories (billing, bug, feature request, account access, sales) and one of three urgency levels.
Twenty emails is a small sample, so treat the accuracy figures as a rough signal. The machine also stayed in normal use during the test, with a browser, a terminal and a local web server running.
How fast is a 4B model on a 2019 i9?
First, llama.cpp’s own benchmark, three runs each:
| Test | 8 threads | 16 threads |
|---|---|---|
| Reading a 512-token prompt | 80.2 tokens/s | 76.0 tokens/s |
| Writing 128 tokens | 11.2 tokens/s | 4.7 tokens/s |
With one thread per physical core, it writes about 11 tokens a second. Using all 16 threads made writing more than twice as slow, most likely because the extra threads share cores and compete for memory. On a CPU, set the thread count to the number of physical cores and test from there.
On the real task, across the 20 emails:
- 5.8 seconds per email on average, 7.9 seconds for the slowest
- 116 seconds for all 20, which is about 620 emails an hour from one machine
- about 1.4 seconds to read each email and 4.4 seconds to write the answer
Most of the time goes to writing the answer, and the one-line summary is a large part of it. Dropping the summary should make each email noticeably faster. We haven’t measured that version yet.
Why the build made a nearly 3x difference
Our first run used llama.cpp from Homebrew, the usual way to install it on a Mac. That package is built to run on any Intel Mac, so it doesn’t target the newer vector instructions this chip has. With it, the same machine wrote 3.9 tokens a second and took about 17 seconds per email.
Building llama.cpp from source with native optimisations turned on raised writing speed to 11.2 tokens a second and cut the time per email to 5.8 seconds. The answers were identical. If a local model feels slow on your hardware, check how it was built before you buy new hardware.
The same test on a MacBook Air M1
We then ran the same test on a 2020 MacBook Air with Apple’s M1 chip, the smallest Apple silicon Mac we have. On Apple silicon, the Homebrew package of llama.cpp runs the model on the built-in GPU through Apple’s Metal, so we used it as it comes.
| Machine | MacBook Air, Apple M1 (8 CPU cores, 7-core GPU), 8 GB memory, macOS 27.3 (build 26C5035r) |
| Runtime | llama.cpp 0.9.0 from Homebrew (build 12110, commit cd959bae9), Metal, all layers on the GPU, 4 threads |
| Model and task | The same model file, system prompt, JSON schema and 20 emails, temperature 0 |
We set the context to 4,096 tokens so the model fits in 8 GB. Each email uses under 300 tokens. The i9 used llama.cpp’s default. The Air had other work open during the test, so these aren’t best-case numbers either.
| Core i9-9980HK, CPU only | MacBook Air M1, GPU | |
|---|---|---|
| Reading a 512-token prompt | 80.2 tokens/s | 204.4 tokens/s |
| Writing 128 tokens | 11.2 tokens/s | 22.0 tokens/s |
| Average time per email | 5.8 s | 2.9 s |
| Reading and writing each email | 1.4 s and 4.4 s | 0.6 s and 2.2 s |
| Slowest email | 7.9 s | 4.0 s |
| All 20 emails | 116 s | 57 s |
| Emails per hour | about 620 | about 1,260 |
| Category, urgency and order number right | 17, 15 and 20 of 20 | 17, 15 and 20 of 20 |
The M1 sorted each email in about half the time. It gave the same category and urgency as the i9 for every email, including the same mistakes.
Two more data points from the Air:
- With llama.cpp told to keep every layer off the GPU (
-ngl 0, 4 threads), it read 44.4 tokens a second and wrote 17.9. That’s slower than the i9 at reading and faster at writing. Writing speed depends mostly on how fast the chip can read the model from memory, which probably explains the M1’s lead there. - Apple’s MLX framework (mlx-lm 0.32.2) with a 4-bit MLX version of the same model read 162.1 tokens a second and wrote 23.8, with a peak memory use of 2.85 GB. We measured only speed with MLX, and its 4-bit format isn’t the same file as the one llama.cpp used.
llama.cpp reported that macOS lets the GPU use about 5.7 GB of the Air’s 8 GB, so this 2.5 GB model fits with room to spare. We didn’t try larger models on the Air.
How accurate was it?
| Correct | |
|---|---|
| Category | 17 of 20 |
| Urgency | 15 of 20 |
| Order number | 20 of 20 |
What it got wrong:
- It filed two login problems as billing: a password reset email that never arrived, and a team locked out by two-factor codes. In a live inbox, both would have gone to the wrong person.
- It filed an integration question as a feature request. A buyer comparing tools asked whether we connect to Xero, which we’d file as sales, though that one is arguable.
- It marked three sales enquiries as low urgency where we expected normal, and an upload stuck at 99% as high. That’s partly our fault: the prompt never said how urgent a sales enquiry is.
A few example emails in the prompt and a clearer urgency rule would probably fix most of these. We’ll test that next.
Does a bigger model get more right?
We also ran a model about twice the size on the i9, with the same settings and 8 threads. Qwen hasn’t released an 8B version of the 2507 instruct model, so we used the original Qwen3-8B at Q4_K_M, a 5.0 GB file. The JSON schema made it answer directly, without its thinking step.
| On the i9 | Qwen3-4B (2.5 GB) | Qwen3-8B (5.0 GB) |
|---|---|---|
| Reading a 512-token prompt | 80.2 tokens/s | 50.9 tokens/s |
| Writing 128 tokens | 11.2 tokens/s | 6.7 tokens/s |
| Average time per email | 5.8 s | 8.2 s |
| Category right | 17 of 20 | 19 of 20 |
| Urgency right | 15 of 20 | 16 of 20 |
| Order number right | 20 of 20 | 20 of 20 |
The 8B model filed both login problems correctly, and it got the urgency right on the sales enquiries and the stuck upload. It still filed the Xero question as a feature request. It also made four urgency mistakes the 4B model didn’t: an app crash marked normal instead of high, two feature requests marked normal instead of low, and a lost authenticator app marked high instead of normal.
It wrote shorter summaries, 39 tokens on average against 49, so each email took about 40% longer even though it writes much more slowly. We ran the 8B email test twice. The first run, while the i9 was busy with other work, averaged 10.0 seconds per email, so the table shows the second run.
The bigger model fixed the two login mistakes that would have sent emails to the wrong person. Urgency still needs a clearer rule in the prompt.
What does it cost per 1,000 emails?
On average the model read 119 tokens and wrote 49 for each email. Those counts come from Qwen’s tokenizer. Claude counts text slightly differently, so the API figures below are estimates.
| Option | Cost per 1,000 emails |
|---|---|
| Our i9, electricity, upper bound | about ₹2.4 |
| Claude Haiku 5.5 | about 3.6 US cents |
| Claude Haiku 5.5, Batch API | about 1.8 US cents |
| Claude Sonnet 5.5 | about 72 US cents |
API prices are from Anthropic’s pricing page, checked on 12 November 2026: Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, Claude Sonnet 5.5 costs $2 and $10, and the Batch API halves both.
For the electricity, 1,000 emails at 5.8 seconds each is about 1.6 hours of work. We haven’t put a power meter on the machine yet, so we used numbers that overstate the cost on purpose: the whole machine drawing 150 W (the processor is rated at 45 W) and electricity at ₹10 per kWh. That comes to 0.24 kWh, or about ₹2.4. The real figure is lower, and lower still if the machine is on for other work anyway, because only the extra power counts.
So when should you run AI on your own hardware?
At these numbers, cost doesn’t settle it. These things do:
- Where the data is allowed to go. If a contract, a regulator or your own policy says customer emails stay on your hardware, a local model is the straightforward answer, and an ordinary office machine is enough for sorting work like this.
- Speed. 5.8 seconds an email on the i9, or 2.9 on the M1, is fine for a queue that runs in the background. It’s too slow for someone waiting on a reply.
- Accuracy on your own emails. Run 20 to 50 of your real examples through both options before you decide. Our test showed the small model needs a clearer prompt for login problems.
- Your time. The hardware and electricity were the cheap part. Setting it up, keeping it running and updating the model cost more than both, and an API removes most of that work.
- Volume. A million emails at our token counts would cost about $36 on Claude Haiku 5.5. The same million would take this one i9 about 67 days, or the M1 Air about 33. The i9 tops out near 15,000 emails a day and the Air near 30,000, so past that you’d need a bigger GPU or more machines, and at today’s prices the API is hard to beat on cost unless the data rule above applies.
That’s also how we decide for clients. We suggest a self-hosted setup only when the data rules or the numbers call for it, and we measure it first.
How to repeat this test
You need the model file from Hugging Face, CMake and a C++ compiler. Build llama.cpp for your own CPU, with GPU backends off so the test stays on the processor:
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_NATIVE=ON -DGGML_METAL=OFF -DGGML_BLAS=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-bench llama-completion -j 8
build/bin/llama-bench -m Qwen3-4B-Instruct-2507-Q4_K_M.gguf -t 8,16 -p 512 -n 128 -r 3For the email test, run each email through llama-completion with the system prompt in a file, the JSON schema in another and temperature 0, so repeat runs give consistent answers:
build/bin/llama-completion -m Qwen3-4B-Instruct-2507-Q4_K_M.gguf -t 8 -cnv -st \
-sysf system.txt -jf schema.json --temp 0 -n 120 -p "Hi, I was charged twice for order #A-10482..."llama.cpp prints the reading and writing time for every run. We added those two together for each email and left out the time spent loading the model, which happens once.
On an Apple silicon Mac, the Homebrew package already runs the model on the GPU, so there’s nothing to build:
brew install llama.cpp
llama-bench -m Qwen3-4B-Instruct-2507-Q4_K_M.gguf -ngl 99 -p 512 -n 128 -r 3
llama-completion -m Qwen3-4B-Instruct-2507-Q4_K_M.gguf -ngl 99 -c 4096 -t 4 -cnv -st \
-sysf system.txt -jf schema.json --temp 0 -n 120 -p "Hi, I was charged twice for order #A-10482..."