Local models: Could Qwen 27B Replace My Claude Subscription?

Lukas Kaminskis · 10 min · October 14, 2026

What I wanted to know

Claude Opus 5.5 is the model I pay for and use every day for coding. Over three days in October I tested whether Qwen 3.8 27B, an open model running on my MacBook Pro, could do the same work. Qwen built the small app and passed all 11 checks. The larger job needed bug tickets from Claude Code and Codex.

Qwen runs locally. There are no per-token API charges. I used a 4-bit version, a 16.08 GB download. Quantisation stores the model's weights at lower precision, using fewer bits. That lets a 27-billion-parameter model fit on a laptop. Running it takes more memory than the download. The context and software need room too. I didn't measure the peak.

A token is a word or part of a word. Tokens a second measures how fast a model writes text. Inco AI, which makes the Splash app, says it runs Qwen at 144 tokens a second on an M5 Max. A Reddit user on an M5 Pro with 48 GB reports simple tasks taking 30 minutes to an hour or longer. They ran Qwen through Ollama and Claude Code.

Vendor claim: Inco AI on X, 19 Sep 2026.

One user's setup: u/Secret_Play9552 on r/LocalLLM, Sep 2026.

These are different measurements on different hardware and software, and they cannot be compared head to head. Speed of generation alone doesn't tell you whether the result works. I tested on an M5 Pro with 64 GB.

Runtime and harness

The runtime loads the model and generates text. I tested LM Studio, Splash and MTPLX, which uses Apple's MLX framework on the Mac's GPU. The harness is the agent software that manages files, commands and the loop around the model. I used Claude Code, OpenCode and a custom harness where Qwen writes small programs. Speed depended on the runtime and the task:

Test 1: a notes app from one prompt

Both models built a working notes app that passed all 11 checks on both runs. With default settings, Opus took 37 seconds and Qwen 557. After I raised three Claude Code timeouts, a rerun in a fresh folder took Opus 47 seconds and Qwen 162.

The job: build a notes app from one prompt, scored by 11 automated checks. One run per configuration.

Pointing Claude Code at a local model

Claude Code is Anthropic's coding agent. It sends the model your request and its instructions. The model picks the next step, and Claude Code carries it out. I ran Qwen behind a local server with a compatible API and set ANTHROPIC_BASE_URL to its address. The server has to be compatible with Claude Code.

What slowed the first run

My opening prompt was about 25,700 tokens, made up of the instructions and skills installed in my own Claude Code setup. Cold prompt processing delayed the first output long enough to trigger the stream timeouts. The retries lost the partly built cache. Long prompts got processed again.

I raised three stream timeouts to 10 minutes in Claude Code, the version I tested.

In a separate diagnostic run after the fix, the first request took about 42 seconds to its first token, with 8,192 tokens cached. Later requests reached their first token in 0.35 to 0.5 seconds, with 99.8% prompt-cache reuse. Complete replies still took longer. The notes-app rerun took 162 seconds against 557 before the fix. It also generated fewer tokens, and cache reuse could differ, so this is an observed improvement, not an isolated measure of the timeout fix.

Other costs on a laptop

Claude Code also asks the model small side questions, such as whether a command is safe to run. On one run, Qwen was busy with the main task. A side question timed out three times and cost about 18 minutes.

Every instruction in the prompt also costs processing time and memory. Claude's advertised context window is about a million tokens. Qwen's is 262,144. My runs used far less. The largest prompt was 71,427 tokens, about a quarter of Qwen's window. At around 65,000 tokens of context, Splash generated about 25 tokens a second. On short tool turns it managed 65 to 85. On a laptop, I don't think Qwen can use its full advertised window effectively.

If you try it: keep the opening prompt short, raise the stream timeouts for your Claude Code version and start a fresh session for each task.

Test 2: the real job

Opus passed 53 of 56 checks in one attempt in 9.4 minutes. No Qwen attempt on its own passed all 56, and the highest total was 46. Qwen reached 56 only when Claude Code and Codex wrote bug tickets for it to fix.

The job: an Evernote-style redesign of the notes app for desktop and phone. A script runs 56 automated functional and layout checks. Some checks are harder than others. The score counts checks passed.

Note on the ticket run: 132.5 minutes counts the initial attempt plus 13 accepted repair steps. It excludes a discarded 20.2-minute attempt, harness development and checks outside the accepted-step windows. It isn't total project effort. This assisted run happened before the autonomous loop experiments.

One attempt each

The short-prompt Qwen attempt used OpenCode and passed 38 of 56 checks in 39 minutes. Opus used Claude Code and passed 53 in 9.4 minutes. The different harnesses limit this comparison.

Qwen on its own

Each attempt below was a separate run with its own changes. One asked Qwen for the whole app in one big answer. After 80 minutes it had no saved app result and no score. A small-step loop on MTPLX reached 41 of 56. On Splash, an OpenCode loop reached 43 and the programs harness reached 46. The 56 checks come in three groups: 29 core, 13 extra and 14 review.

• Plan for hours. Both Splash runs hit their three-hour limit. One answer took almost 7 minutes to write.
• Keep the Mac awake. One healthy run died when the Mac went to sleep. Splash also crashed once and cost about 27 minutes. At the restart, only 4.3 GB of memory was free.
• Keep the checks out of reach. In one OpenCode run, Qwen ran 21 shell commands and read the test code. I threw away that 20-minute attempt and reran it cleanly.
• Expect progress in jumps. In the programs run, the harness kept 7 of the 14 steps Qwen made with thinking on. One step lifted the score from 23 to 36 in about 10 minutes.
• Each setup missed different checks. The OpenCode loop passed every core and extra check, and 1 of 14 review checks. The programs harness passed 7 of 14 review checks and dropped the Duplicate, Move and Delete menu items.
• Much of the time is reasoning. In the programs run, 83% of the tokens Qwen generated were reasoning. Model requests took about 116 of the run's 180 minutes. Qwen's own browser tests took another 28.

Qwen is very good if you have time

Claude Code and Codex reviewed Qwen's app and wrote 13 narrow bug tickets. Each described one problem the way a user would see it, such as "on a phone, tapping Create opens a blank screen". Qwen made the app edits for each ticket. I verified them by replaying the edits. After 13 accepted steps the app passed all 56 checks. Bugs outside what the checks cover could remain. Claude and Codex models also built the harness and the checks.

Opus missed two small-touch-target checks and the responsive-header check. It tried to test its own work in a browser. That attempt was blocked. Qwen ran browser tests too. Its autonomous attempts still missed checks. The ticket-assisted run passed all of them.

If you try it: break the work into small steps and run checks after each one.

The harness is where I'd work next

I think most of the improvement would come from the harness. The biggest differences in my tests came from the setup around the model. The OpenCode loop and the programs harness passed different sets of checks. Narrow bug tickets took the same model to 56 of 56.

The next version would need smaller steps, checks Qwen can't read, and gates that keep useful partial repairs. With that work, I think Qwen could produce good results consistently. On long-context tasks it would still trail Opus at the same time budget. Given more time, I think it could match Opus.

What I didn't test

I tested one Mac, one model size and no PCs. Text generation speed depends heavily on memory bandwidth. Apple lists up to 307 GB/s for the M5 Pro and up to 614 GB/s for the M5 Max. As a rough, untested estimate, an M5 Max might generate text about twice as fast. Whole tasks would speed up less. Prompt processing, tool calls, tests and retries take time too. Faster generation also wouldn't improve the results by itself.

At launch on 3 March 2026, Apple's US prices started at $2,199 for the 14-inch MacBook Pro with M5 Pro and $3,599 with M5 Max (Apple). Those base configurations differ from the 64 GB machine I tested. If you already have a Mac with plenty of memory, try it on small jobs you can check before buying anything.

The only full score came from cloud models writing tickets while Qwen made the edits on my Mac. That workflow sent the app to cloud models for review. It wasn't offline.

Should you cancel your Claude subscription?

Small jobs look promising. Qwen built the notes app and passed all 11 checks, more slowly than Opus. That's a small sample, but it's where I'd start.

Qwen needed cloud-written bug tickets to pass all 56 checks on this task. I'm keeping Claude.

How I measured

I ran these tests from 11 to 13 October 2026 on one MacBook Pro with an M5 Pro and 64 GB. I used Qwen 3.8 27B at 4-bit, with one run per setup. I changed harnesses and gates between attempts. Later loops could see the 56 checks as feedback. They weren't a held-out test. Wall-clock times include tool waits inside each timed window. They also include crashes, sleep or throttling where those happened. The records don't show the Mac was kept awake throughout. Later stage timings exclude server loading. These are experiments with my own workflow. A controlled model ranking would need fixed harnesses, held-out checks and repeated runs.

Want to start building with AI?

Ship real products with AI agents on our hands-on Building with AI programme.