I gave my language's spec to 36 AI sessions that had never seen it. Here is what they got wrong.

Kaury is a small language that turns one .kaury file into a complete website: pages, styles, logic, forms that send e-mails, 3D scenes. The output is server-rendered HTML with a sitemap, structured data and an llms.txt.

This is a whole site:

site "Bloom"
  colors pink #E93D82, cream #FFF6EE
  fonts "Satoshi", "Cabinet Grotesk"
  style background cream

state likes = 0

style card-soft
  background white, radius 20, padding 24
  hover lift 6, shadow medium

page "/"
  seo "Bloom", "Flowers that speak"
  scene fullscreen, particles dust
    light soft
    object "knot", color pink, metal
      spin on scroll
      follows mouse, smooth
    title "Flowers that *speak*.", size 80
  section
    column card-soft
      text "Delivered the same day in Vevey."
      button "I like it ({likes})" -> likes += 1

Two things to say before going further.

I am a designer, not a programmer. The compiler was written with AI assistants, under my direction: I decided what the language should look like, tested it, and refused what I did not like. I would rather say it here than have you find out.

The language has one unusual constraint. No model was trained on it, so an assistant can only know Kaury from one file: the specification. If an assistant reads that file and still writes something that does not compile, I count it as a fault of the language or of the specification, not of the assistant.

So I measured it.

The test

Each session got a folder with four photos, the specification, and one request: a restaurant site, a quiz, a mortgage calculator, a page of figures with a chart and a sortable table, and so on. The rules:

• read the specification and nothing else;
• write the site once;
• no compiler, so no way to check or correct.

Then I ran the compiler on what came back.

One limit you should know: every session was a fresh instance of Claude. I have not run the test on other models yet, so this says nothing about them.

First group: 11 out of 20

Eleven files compiled at the first try. The nine others failed, and almost every failure was reasonable.

A comma after the element. They wrote row, gap 16. The grammar wanted row gap 16. Nothing made the comma ambiguous; the parser refused it for no good reason.

Properties listed underneath, as in CSS.

column
  radius 12
  padding 24
  text "Hello"

The language wanted the options on the same line as the element. Again, nothing made the other form ambiguous.

French keywords that did not exist. Kaury has French aliases. Asked to write in French, sessions guessed natural words such as rayon for radius. The specification listed four French words out of about two hundred.

The two fonts in the wrong order. Every single one wrote the title font first. The language expects the text font first. Nothing was refused: the site compiled, with its two fonts swapped. This was the worst of the five, because a silent wrong result is worse than an error.

A countdown that showed 0. Nothing in the language turned a text into a date.

What I changed

I did not write better instructions. I changed the language so that the reasonable guess is the right one:

• the comma after an element is accepted;
• options written under an element are accepted;
• the French words are generated from the compiler's own vocabulary, and a test keeps the specification exact;
• the font order is stated with an example at the very place where fonts are declared;
• to-date and format-number exist.

With those changes, 18 of the same 20 files compiled as they were. After one round where each session received the compiler's messages, 19.

Those messages are written for that purpose: the line, the place, and what to try.

Error site.kaury, line 15: "likez" does not exist.
  15 |     button "Go" -> likez += 1
     |                    ^^^^^
Try: declare it first: let likez = …   (or state likez = … if it changes)

Next two groups: 16 out of 16

Two new groups of eight fresh sessions, with harder requests. All 16 files compiled, with no error and no warning.

What the test does not show

They write it. They do not choose it. I also asked each session whether it would have proposed Kaury for that request on its own. In the first group, 13 of 20 said yes. In the two harder groups, 2 of 8 each time. Their reasons were fair: no compiler at hand to check their work, a request that was really one widget for an existing site, a documentation site better served by a dedicated generator, a need for payments.

Compiling is not the same as looking right. A file can compile and produce a clumsy page. The swapped fonts proved it. I looked at the pages myself, which does not scale.

Kaury is not for everything, and the specification says so: no accounts, no database, no payments, not one widget inside an existing site.

See it

The site of the language is written in Kaury. It shows six sites running live, each with the code that produced it: dev.kaury.studio/showcase.

Measured on that site: Lighthouse 100 on a desktop. On a phone, 98 to 100 on most pages, and 93 on the home page, which runs a 3D scene.

npm install -g @kaury/cli