Operated

Part 5 of five in StellarView Alpha, a series about delivering a solved application as an operated platform.

Our own tests passing is a laptop result. Somebody else’s suite is the step off it.

This is the application, running, with content created through its own published interface. Articles, authors, tags, comments, favourites. It is the part a hundred people have built.

The suite we did not write

Thirteen files. One thousand seven hundred and nine lines. One hundred and fifty four requests. Written by people with no interest in whether we pass.

It failed on the first run today.

Not on semantics. The suite makes twenty one anonymous registrations, and our anonymous-write limiter allows twenty per minute. It failed by a single request, on a production-shaped default, in a way no test we wrote would ever have found.

CI already knew. The comment in src/config.js records why:

// The anonymous-write limiter. These were literals in src/routes/users.js, so a
// conformance run — 154 requests, most of them registrations — tripped a
// production-shaped limit with no way to raise it for the run. Defaults are the
// former literals, so behaviour is unchanged unless somebody sets them.
ANONYMOUS_WRITE_WINDOW_MS: z.coerce.number().int().positive().default(60_000),
ANONYMOUS_WRITE_MAX_REQUESTS: z.coerce.number().int().positive().default(20),

You can count the other side off the stranger’s own files:

grep -h 'POST {{host}}/api/users$' /tmp/realworld/specs/api/hurl/*.hurl | wc -l
# 21

Twenty one against twenty. Run it the way CI runs it, with the limit raised for that job, and the result is 13 of 13, 154 requests, 912 milliseconds.

I got that wrong twice before I got it right, and both are worth admitting. The first attempt passed the wrong variable name and produced a false zero of thirteen that was measuring nothing. The second ran the files in parallel and measured our own rate limiter. Neither was the application’s fault.

The assertion that cost a schema change

The last of the thirteen files to go green failed on one line of somebody else’s test:

jsonpath "$.comment.id" isInteger

We were issuing uuids. Every client written against the specification, including the reference frontend, addresses a comment by that field. So the fix was not a serialiser tweak, it was rewriting the primary key of a populated table:

-- Rewriting the primary key of a populated table is the kind of migration that
-- deserves a rollback, so the old uuid is kept in `legacy_id` rather than
-- dropped. Any row that existed before this ran can still be found by it.

ALTER TABLE comments ADD COLUMN legacy_id uuid;
UPDATE comments SET legacy_id = id;

And then the part that is worth more than the fix. The migration missed a grant, and said so, in the file, permanently:

-- Without this, inserting a comment fails with "permission denied for sequence
-- comments_id_seq" — a 500, from a role that has INSERT on the table but cannot
-- draw the default value the column now depends on. Granting a table does not
-- grant the sequences its defaults use.
--
-- This is the third time a new object has been created without granting the
-- runtime role: 002 granted the tables, 005 exists because schema_migrations was
-- missed, and this migration missed the sequence. Any migration that creates an
-- object should be read with the question "what does aqueduct_app need on it".
GRANT USAGE, SELECT ON SEQUENCE comments_id_seq TO aqueduct_app;

“This is the third time.” A machine counting its own repeated mistake, in a comment, where the next person will read it.

What their suite does not test

Tenancy.

Counting users rather than articles: as tenant A, three rows. As tenant B, three others, with the same two names in both. With no tenant, nothing, because the database refuses rather than the code remembering to ask.

There is a real trap underneath this, and we hit it on camera. The single-tenant pin that makes the demo work at localhost also makes the isolation proof fail five of five. Every request then resolves to one tenant whatever host it carries.

The conformance run and the isolation proof need mutually exclusive configuration. That should be written down rather than discovered.

The seven phases a demo never reaches

That is the platform’s desk, not the application’s. A ticket raised in the app lands in JIRA, and StellarView reads it from there, classifies it, and enriches it against the repository before a human sees it.

And this is where the corpus lives: ninety four files across fourteen galaxies, on S3 with encryption and its own IAM roles. The demo application built its own version of this as phase seven. It exists because the platform showed it what one looks like.

A support desk holding real tickets. One came from a developer whose client broke when comment identifiers changed shape. It is escalated, because the desk would not answer it.

That refusal is the design. Ask it something its corpus does not cover and it says so:

I don’t have enough grounded information in this platform’s own repository to answer that confidently, routing to a human.

Take the provider away entirely and it returns 503 naming the missing key. It never invents an answer.

An operations dashboard reading the structured logs the observability phase produced. Insights over the data lake. Tenant administration. A generated wiki that fails its own drift check when the code moves underneath it.

And one more, which is the reason you can watch any of this:

That is the engine these five films were made with. You describe what a film has to get across, it drafts the beats and writes the narration, and you read the narration back before spending a render on it. Subject, type, target length, audience, the beats it must include, and the evidence it is allowed to state. Anything not in that evidence box will not be said, because the engine writes a weaker line that is true instead of a stronger one that is not.

It ships inside every platform this builds, so Aqueduct has one too. On this estate it reads Nothing drafted yet, which is the honest state of it: the films you are watching were rendered from the repository, not from that panel. A capability that exists and has not been used is worth more said out loud than quietly cropped out of frame.

Who keeps a spec alive for a decade

This comparison exists because other people built and maintained the control, unpaid, for ten years. That deserves more than a citation.

In October 2024, Gérôme Grignon, who had revived RealWorld in 2021 and carried it for three years, opened an issue titled “Stopping maintenance”:

While I had a great time joining the project a few years ago and learned a lot from it, it also has been mostly about negative feedback or criticism. Based on the API usage, I’d say a lot of people used it but most interactions were from the ones having issues with it.

Not only I don’t earn money out of it but spend money/free time on it, so I have reached the point where this project is a real burden for me.

But it’s always been a free time project, in direct concurrence with valuable time with family or personal hobbies.

The project had over eighty thousand stars. That issue received zero comments.

Four years earlier, Cameron Chapman had written the same thing. He was standing on the other side of the same problem:

Most of the maintainers of this project have slowly gotten busy with other things and haven’t been able to keep up as of late (this includes and is really referring to myself). I still think this project can provide a lot of value to people so it’s a bit of a shame to see the issues piling up.

And he named the mechanism precisely:

This is pretty time consuming though, as a lot of the time this requires setting up new environments on your machine to test out a new language/framework.

That is the step off the laptop, described by someone standing on it. Verifying work across a hundred implementations meant a human building a hundred environments by hand. It is the reason the project decayed, and it is the same reason the suite that checks us now is thirteen files a machine can run in under a second.

The project is alive today because of c4ffein. They first showed up as a contributor, then took ownership when maintainers were being sought. They restored the demo and the API that had gone dark, and rewrote the test suite.

They added their copyright line beneath Thinkster’s rather than replacing it.

What this was actually about

It was never about the application. A hundred people built the application.

The bar for what counts as delivered has moved, and it moved for reasons that have nothing to do with which model you use. Tenancy. Infrastructure as code. A rollback that has been run. Observability. Retrieval over your own corpus. A support desk. A wiki that fails when it drifts.

Fewer and fewer people can clear that bar, whatever platform they are holding. The ones who can have mostly left the editor behind and are working at the level of the lifecycle instead, because they have the vocabulary and the judgement, earned slowly. No tool supplies those. A tool multiplies whatever is already there.

Ten times has to go in for ten times to come out.

---

Conduit is the reference application of RealWorld, created in 2016 by Eric Simons and Albert Pai and funded in its early years by Thinkster. The project credits Dan Abramov and Addy Osmani with helping spark the idea, and TodoMVC as its model. It was revived in 2021 by Gérôme Grignon and Manuel Vila, who also built CodebaseShow. It is maintained today by c4ffein and Manuel Vila. MIT licensed, © Thinkster and © c4ffein.

We are not affiliated with the RealWorld project. Nothing here should be read as endorsement by its maintainers. Passing the spec’s suite demonstrates conformance to that specification and nothing more.

Thanks to everyone who kept this alive, unpaid, for a decade.

Back to the series, or start again at The plan asked 357 questions.