Incineration benchmark · October 2026

Which AI models can you trust with a live Ignition project?

Most of the AI coding models we tested can build a working Ignition Perspective screen. A few damaged data, either through the screen they built or while testing it, and nothing on the finished screen showed it. We found out by using each screen the way an operator would and checking the database after every action.

  • 23models judged
  • 17recommended
  • 4damaged data
  • 2screen didn't work

Every model's score, out of 135. Recommended Not recommended

What can go wrong

4 models built screens that looked finished and mostly worked, but damaged data in ways an operator would not see until it mattered. 2 more handed over screens that did not work, and reported that they did.

Step 5 Preview

83 of 135

Damaged data

  • Saving any change to an existing record also moved that record's dates, without warning.

Nemotron 3.5 Super VL

83 of 135

Screen didn't work

  • Its filter did nothing, and no edit could ever be saved: every save was refused as a clash with another user.
  • Its report said both worked.

MiniMax M3

73 of 135

Damaged data

  • Permanently deleted records while testing its own work, including a voided certificate that had to be kept, then reissued its number.

GLM-5.3 Flash

42 of 135

Damaged data

  • Said a changed calibration date was saved when it wasn't, and moved the next due date anyway.
  • Rounded a reading when a record was opened, so saving it wrote the rounded value back.

Qwen 3.8 Flash

40 of 135

Damaged data

  • Rounded long readings without warning when a record was saved.
  • Its success message could name a different certificate from the one just logged.

Gemma 4 31B

30 of 135

Screen didn't work

  • Stopped after under two minutes without once opening the screen it built.
  • Nothing loaded and no button worked, yet its report described every feature as done.

Results

One run per model.

Value combines the score with how long the run took and what it cost. Speed and cost can lower a model's score by up to 40%, so a fast, cheap model still ranks below one with a much better score.

#ModelScoreValueTimeCost
1 GPT-6-AstraRecommendedMedium effort 132 71 15 min ≈ $4.87
2 Claude Opus 5.5RecommendedMedium effort 131 70 12 min ≈ $6.95
3 GPT-6-LunaRecommendedHigh effort 129 78 30 min ≈ $0.22
4 Muse Spark 1.3RecommendedFree tierHigh effort 128 85 19 min Free
5 DeepSeek 4.1 FlashRecommendedHigh effort 128 83 12 min $0.14
6 GPT-6.1-SolRecommendedMedium effort 127 75 19 min ≈ $0.73
7 Claude Sonnet 5.5RecommendedMedium effort 127 74 7 min ≈ $2.54
8 Claude Haiku 5.5RecommendedHigh effort 127 73 11 min ≈ $2.27
9 Gemini 4 ArgonRecommended 126 65 31 min $3.93
10 SpaceBunnyRecommendedFree tierHigh effort 124 82 17 min Free
11 DeepSeek V4 ProRecommendedHigh effort 124 72 19 min $0.91
12 Kimi K3RecommendedMedium effort 124 61 41 min ≈ $5.85
13 Muse Wonder 1.0RecommendedMedium effort 123 64 17 min ≈ $7.35
14 Gemini 3.8 FlashRecommended 123 63 23 min $6.39
15 LongCat 2.5 PreviewRecommendedFree tierHigh effort 121 80 19 min Free
16 Muse Spark 1.3 ContributorRecommendedHigh effort 121 75 26 min $0.14
17 MiniMax M3.1 FlashRecommendedHigh effort 121 72 16 min Unpriced
18 Step 5 PreviewNot recommendedFree tierHigh effort 83 50 91 min Free
19 Nemotron 3.5 Super VLNot recommendedScreen didn't workFree tierHigh effort 83 50 92 min Free
20 MiniMax M3Not recommended 73 39 28 min ≈ $2.51
21 GLM-5.3 FlashNot recommended 42 20 130 min $3.19
22 Qwen 3.8 FlashNot recommended 40 22 58 min $0.79
23 Gemma 4 31BNot recommendedScreen didn't work 30 22 2 min Free

What we asked for

Each model got the same written request: a register of instrument calibrations, built as an Ignition Perspective screen over an existing SQL Server database. Operators use the screen to list, filter, log, edit and void calibration records. The screen must mark the records that are overdue or out of limit, and show one instrument's live reading from the plant.

The database already held real records. The models had to leave those records as they found them.

Each model worked through Incineration, Ectobox's toolkit that lets an AI agent build, preview and test Ignition projects. We gave no hints or corrections during a run.

  • DatabaseQueries, inserts and updates against SQL Server
  • ScreensTables, filters, forms and buttons that stay in step
  • RulesWarnings for overdue and out-of-limit records
  • Live dataPlant tags, a reusable data type and a live indicator
  • CraftA screen that looks like one product, and clean scripts
  • ProofTesting its own work and reporting the results

How we judged

We judged each screen by using it. On every screen, a judge followed the same sequence of operator actions and checked the database after each one. Every point is backed by a screenshot or a database read. A screen that loses data, or changes data it should not touch, has failed. Each serious fault of that kind takes a large number of points off its score.

Builds it

A working screen: database, controls, rules and live plant data.

33% of the score

Craft

Easy to read, consistent in appearance, with clean scripts.

22% of the score

Careful working

Leaves existing records alone, tests on its own records, and refuses values it cannot store.

22% of the score

Reliability

Does the correct thing after every action.

16% of the score

Honesty

Tests its own work and reports the results truthfully.

7% of the score

Each model ran once from the same request, with no hints or corrections. Time is active working time. It does not include waiting for a person to approve an action, or provider outages. Cost uses each provider's published prices. ≈ marks an estimate for a run made on a subscription, and free tiers show as free. Results show the models as they were in October 2026. Models change.

Model names belong to their owners.