W31: Installed is not the same as proven
The week a filter meant to keep things tidy quietly held production code off the air for 33 hours — while every other project moved the opposite way: not reporting the problem, but fixing it.
- Sales-ai gained gates that repair the answer themselves instead of writing a human a report about what went wrong
- Price parity proven with numbers — search system against the offer actually sent: 6 of 8 exact, 2 within ±4%
- Radio Viens' new vocal version proven on air by measurement, not by ear — correlation 0.835 against a 0.157 control
- The Silver Fox portal now shows the client the input, the output and the work history — including where processing failed its checks
- Hotel descriptions rebuilt to the CEO's direction: benefits, not a spec sheet, and a claim may rest only on the hotel's own website
- A rule must be proven on both cases — that it holds back what should be held, AND that it lets through what should pass
- A silent failure costs more than a loud one: no alarm, no error message, just a system that keeps running wrong
- A system that repairs its own answer is a different class from one that reports its own error — in the second case the work stays with the human
- Tool semantics are not the same as an assumption about tool semantics — an unverified assumption costs hours that pass in silence
- A deploy filter with negative masks also stopped the real deploy — CEO-requested fixes were merged but not live for 33 hours, and nobody knew
- Monitoring reported as 'broken' something it had never seen working — a check on paper, not on the system
- Two of our own prematurely written rules had to be withdrawn — an audio-influence threshold and a conclusion about a department field; both were refuted by facts, not debate
Ecosystem overview
The past week across eight active projects circled one question: what separates “installed” from “proven”. The system answered it itself — a filter put in place so that automatic saves would stop restarting production also quietly stopped the real fixes. The code was merged but not live for 33 hours, and nobody noticed, because silence raises no alarm. Every other piece of the week ran the opposite way — from reporting to repairing, from judgement to measurement.
Vanilla Travel
The week went into ad attribution — which channel brings which customer. The view was rebuilt in three levels: total revenue first, then departments, then channels, so channels stop disappearing under the total. A channel × department cut was added — the question is no longer “how much did it bring” but “did it bring the right customer”. Two conclusions were withdrawn under our own review: direct entries turned out not to be a defect but 80% returning customers, and the department field was something entirely different from what was first assumed. At the end of the week the numbers were reconciled against an external source — the ad tool’s own export — so the check would not be the system talking to itself.
vt-internal-services
Thirty-five commits on sales-ai, and the main change is a matter of principle: the gates now repair the answer themselves rather than writing a report for a human. A price parity endpoint was built, comparing the search system’s price with the offer actually sent — and in the first full measurement 6 of 8 cases matched exactly, 2 within ±4%. On the way there an unpleasant gap surfaced: children were being dropped from the price request, and the client received a price for half the family. It was caught not by a test, but by comparing a manager’s reply with the system’s. Three more fixes: not knowing may no longer be presented as “no availability”, the passive “let us know” was banned from the prompt, and the gates stopped mistaking traveller count for a price.
Hotel Reviewer
Fifty-nine commits, all around one block — the hotel description the client reads. The CEO’s direction was clear: a longer summary, because the block is the page’s main text; benefits, not a spec sheet; no square metres. The most significant piece technically: the system now reads the hotel’s entire website and then checks the text against what is written there — a measurement may be vouched for only by a measurement on the site, not by a general claim. Mid-week brought a return to the June text version and a single-model approach; the two-stage variant was tried and rejected.
Radio Viens
Seventy-one commits, and two proofs. First: the artist’s new vocal version was proven on air by measurement — correlation 0.835 against a 0.157 control, rather than by ear and assumption. Second: a spoken-content line built on pre-rendered files — a new segment type means the server may no longer regenerate the text, which takes the pronunciation lottery off the table. A rescue programme was introduced for songs with a pronunciation defect: fix what is already on air, do not regenerate from scratch, and keep the old version in the archive. One rule we had written ourselves about an audio-influence threshold was withdrawn — refuted by listening, not by argument.
Silver Fox
Fifty-one commits toward the first client. The portal now shows the input, the output and the work history — including the cases where processing failed its checks. Every citation can be opened: the document and the right page, not a word with a promise. The acceptance chain runs from a clean clone with honest exit codes. Model selection was revised on our own testing rather than on announcements — a newer model for text tasks at the same price, the cheapest one for cheap tasks.
Infrastructure and ecosystem
Twenty-five commits in ecosystem monitoring, all about the same gap — a check that exists only on paper. The rescuer may no longer report as broken something it has never seen working. Six live production services that simply were not on the list were added to monitoring. One foreign service was removed from it. And after M-055, an auditor for a new class was installed — “merged, but not live”.
vt-blog
Six commits, one of which is a money question: chat product links had been going out without tracking parameters since 1 July, so the blog simply vanished from attribution reports. Not performing badly — not visible. Fixed. Alongside it, a monitoring canary that had been throwing false red alarms was rebuilt.
Insights
The week’s main lesson is simple and unpleasant: a rule must be proven on both cases. That it holds back what should be held, and that it lets through what should pass. The deploy filter was verified on one side only — that automatic saves no longer restart production. The other side — that real code actually goes live — was never tested once.
The second lesson comes from sales-ai and Silver Fox at the same time: a system that repairs its own answer is in a different class from one that sends a report about its own error. In the second case the work stays with the human, and the human does it late or not at all.
The third: measurement beats judgement. Price parity, on-air correlation, reconciliation against an external export — in all three cases a number ended the argument, not a discussion.
Mistakes and lessons
The week’s most expensive mistake is M-055. To stop automatic saves from restarting production, negative masks were added to the deploy filter. They worked well — too well: on 31 July the CEO-requested fixes landed in the repository and no deploy happened. The next deploy came only on the evening of 1 August. For 33 hours the code was merged but not live, and nobody knew, because silence raises no alarm. The masks have now been simplified to positive ones only, but — and this has to be said honestly — it is not yet proven that the new version works. The proof will be the next production push.
The second mistake is the same class in a different place: monitoring reported as “broken” a system it had never seen working. An alarm without a baseline is not an alarm, it is noise.
The third: twice we had to withdraw rules we had written prematurely ourselves. The audio-influence threshold was refuted by listening; the conclusion about the department field was refuted by the data. In both cases the rule was written down faster than it was verified.