The System Still Works: The Algorithmic Arms Race

Back in May, in The System Works, I wrote about the Sunday session where I finally set Antigravity loose to map out twenty years of gut feelings about the American healthcare system.

I wrote about the plastic card illusion: walking into a clinic with no insurance and paying a $115 copay, versus handing over an insurance card and watching the bill explode to $400. In consulting math, that meant an ordinary 15-minute consult jumped from $460 an hour to $1,600 an hour. Nothing about the clinical diagnosis had changed. Only the billing plumbing had.

The core thesis we landed on that Sunday wasn't that the system was broken. It was the opposite: the system is working perfectly. It is an autopoietic machine, an economic ecosystem designed to reproduce its own administrative complexity and extract maximum yield.

I ended that post noting that while the system hadn't changed, I could finally see the paths, the rules, and why people kept ending up in the exact same traps.

Well, last week, Blue Cross Blue Shield published an analysis that proved the machine has found its next gear.

The $1 Billion Query: Mining for Acuity

According to a new analysis of claims data by the Blue Cross Blue Shield Association, the rapid adoption of AI coding tools by hospitals added an estimated $942 million in commercial healthcare spending between 2023 and 2025 alone.

More than 60% of hospital systems are now running AI-enabled revenue cycle software. These tools continuously scan electronic medical records, lab reports, and doctor notes to find billable complexity.

And what did all those millions in new revenue buy in terms of actual medicine?

Zero.

BCBSA looked closely at patients undergoing major bowel surgery. The AI tools flagged a massive surge in secondary diagnoses, like anemia, derived from single post-operative lab floats. When a hospital attaches a secondary diagnosis of anemia to a surgical stay, the claim automatically bumps into a higher-severity, higher-reimbursement DRG (Diagnosis-Related Group) category. Over 70% of the entire $942 million increase (roughly $650 million) was tied strictly to these secondary diagnoses.

Yet when researchers looked at the clinical charts, there was no corresponding increase in treatments. No uptick in blood transfusions. No change in medication. No change in bedside recovery time.

Luke Chalker, BCBSA's senior vice president of product and data science, summarized the finding with brutal clarity:

"The disconnect between diagnoses and treatment suggests that AI is identifying more billable conditions, not sicker patients."

If you're a data person, read that sentence again. That is not a bug. That is an optimization routine running exactly to spec.

The 74,000-Code Moat

For decades, healthcare policymakers and insurers built a labyrinth of classification. We moved from simple fee-for-service to prospective payment systems based on 74,000 distinct diagnosis codes and 79,000 procedure codes. The stated goal was accuracy and granular clinical documentation.

The unstated reality was that human beings cannot hold 74,000 codes in their heads. Independent medical practices couldn't afford armies of certified coders, which accelerated the collapse of private practice and forced doctors into the arms of massive hospital conglomerates.

And now, the conglomerates have done what every well-capitalized enterprise does when faced with a complex rules engine: they automated it.

They pointed large language models and machine-learning classifiers directly at the EHR data lake. The algorithm doesn't ask if a patient feels better, or if a post-op hemoglobin dip is clinically meaningful. The algorithm asks a purely relational question: Does this float value in table A legally support a secondary modifier in table B that increases the payout in table C?

The code is no longer a proxy for care. The code is the product.

The Press Release: Setting the Narrative

What makes this entire situation so fascinating from a systems perspective is the delivery mechanism. This wasn't a leaked internal memo. This was a press release blasted out by the Blue Cross Blue Shield Association to major outlets like Reuters.

Why is a massive insurance conglomerate publicly complaining that hospitals are out-coding them?

Because they are setting the narrative.

As we traced in the blueprints of The Perfect Machine, Blue Cross was originally born in 1929 at Baylor University. It was created by hospitals, for hospitals, specifically as a pre-payment plan to guarantee hospital solvency. The hospital created the third-party payer to insulate itself from market forces.

For nearly a century, the two sides grew together. Now, the hospitals have turned loose algorithms that out-optimize the insurer's own rulebook. By blasting out a press release, BCBSA is establishing a public scapegoat for why your employer's health premiums are going to spike another 7% next year. They are pointing the finger squarely at the hospital bots.

But make no mistake: this is a two-sided bot war. Insurers aren't innocent victims. For the last five years, major health plans have deployed their own automated actuarial algorithms (like Cigna's PXDX and UnitedHealth's nH Predict) to batch-deny claims in 1.2 seconds without human review. The hospital buys an AI bot to maximize billable severity; the insurer buys an AI bot to mass-reject claims on protocol technicalities.

Two multi-billion-dollar machine-learning networks are now trading transactions back and forth across an opaque API perimeter. Neither machine cares about the human lying in the hospital bed. The patient is just the transaction payload passing between them.

The System Still Works

When you see headlines about AI adding $1 billion to hospital bills, the natural reaction is to throw up your hands and say, "Healthcare is completely broken."

It's not.

If you build a system where the customer does not pay, where the price signal is illegal or hidden, where payments are determined by an arbitrary 74,000-node taxonomy, and where survival depends on administrative volume, you will get autonomous code-mining bots every single time.

In May, I wrote that once you see the plumbing, you stop looking for villains and start understanding the incentives. The BCBSA report isn't evidence of a broken system. It is living, breathing proof that the machine is evolving right on schedule.

---

Architected by Chet, written by Gemini 3.1 Pro

When the Guardrail Stops Guarding

A few months ago, in You Can Point a Foreign Key Where?!, we looked at pointing foreign keys to unique candidate keys instead of traditional surrogate IDs.

In the comments, Tom offered an essential reminder from decades in the trenches:

"You and I have decades now of CREATE TABLE... and those decades have shown us the only thing that is truly immutable are the things stakeholders tell us are absolutely not immutable... business codes have a way of becoming less immutable across years. They get renamed, merged, generally f-d up all the time and when a new C-whatever comes in who likes SOON instead of ASAP, you're cooked."

Tom was arguing for surrogate keys to separate meaning from relational identity. He was right. Business vocabulary is fluid. What feels permanent during sprint planning rarely stays permanent across three fiscal years.

That brings us to a design choice many of us reach for when standing up a new feature: skipping the reference table entirely and putting business enums directly into a table CHECK constraint.

CREATE TABLE orders (
        order_id     NUMBER GENERATED BY DEFAULT AS IDENTITY,
        order_date   DATE NOT NULL,
        order_status VARCHAR2(20) NOT NULL,
        CONSTRAINT pk_orders PRIMARY KEY (order_id),
        CONSTRAINT ck_orders_status 
            CHECK (order_status IN ('OPEN', 'PENDING', 'SHIPPED', 'CANCELLED'))
    );

To be clear: this is not a post against CHECK constraints. CHECK constraints are indispensable. This is a post about which rules they can actually hold.

Putting an enum into a CHECK constraint looks tidy, declarative, and completely self-contained. It avoids spinning up another table, mapping another entity, or coordinating extra seed files. In the moment, it feels like disciplined engineering.

Until the business evolves, and the guardrail quietly stops guarding.

The Silent Failure: Enforcing Neither

The standard objection to putting enums in CHECK constraints is migration friction. We have all heard the complaint: adding a status requires an ALTER TABLE, table locks, and coordinated deployments instead of a simple INSERT.

That argument is true, but it is an ergonomic argument. A determined engineering team can easily wave it off as an acceptable deployment cost.

The fatal problem with an enum CHECK constraint is not operational friction. It is structural failure.

Consider Tom's scenario: leadership decides that 'PENDING' is too vague. From now on, operations needs to split future orders into 'AWAITING_PAYMENT' and 'AWAITING_STOCK'.

What happens to existing records? The business does not want to rewrite history. The thousands of closed orders that passed through 'PENDING' over the last two years need to remain untouched.

Now look at what happens to your constraint:

ALTER TABLE orders DROP CONSTRAINT ck_orders_status;
    
    ALTER TABLE orders ADD CONSTRAINT ck_orders_status
        CHECK (order_status IN (
            'OPEN', 
            'PENDING', 
            'AWAITING_PAYMENT', 
            'AWAITING_STOCK', 
            'SHIPPED', 
            'CANCELLED'
        ));

Because a table constraint evaluates every row equally across the entire table, 'PENDING' must remain in the allowed list forever just to keep historical rows valid.

The moment history and future diverge, the CHECK constraint is forced to permit both. And the moment it permits both, it can no longer prevent an application bug from inserting a brand new 'PENDING' order tomorrow morning.

You thought you had an automated guardrail protecting business integrity. What you actually have is a decorative plaque. The constraint that was written to enforce valid state transitions has quietly stopped constraining.

Temporal Boundaries Trump Hardcoded Strings

When we move business vocabulary out of DDL and into a proper reference table, we don't resort to a lazy is_active boolean flag either. As we saw in The Boolean is Lying to You, booleans erase history.

Instead, we use explicit temporal boundaries:

CREATE TABLE order_statuses (
        status_id            NUMBER GENERATED BY DEFAULT AS IDENTITY,
        status_code          VARCHAR2(20) NOT NULL,
        description          VARCHAR2(100) NOT NULL,
        display_seq          NUMBER NOT NULL,
        effective_start_date TIMESTAMP WITH TIME ZONE NOT NULL,
        effective_end_date   TIMESTAMP WITH TIME ZONE,
        --
        CONSTRAINT pk_order_statuses PRIMARY KEY (status_id),
        CONSTRAINT uq_order_statuses_code UNIQUE (status_code),
        CONSTRAINT ck_order_statuses_dates 
            CHECK (effective_end_date IS NULL OR effective_end_date >= effective_start_date)
    );

(Notice where the CHECK constraint lives: validating that an end date cannot precede a start date. A mathematical and calendar rule, exactly where it belongs.)

When the business retires 'PENDING', we don't touch the orders table. We update the lookup table:

UPDATE order_statuses 
       SET effective_end_date = SYSTIMESTAMP 
     WHERE status_code = 'PENDING';
    
    INSERT INTO order_statuses (status_code, description, display_seq, effective_start_date)
    VALUES ('AWAITING_PAYMENT', 'Awaiting Payment', 20, SYSTIMESTAMP);
    
    INSERT INTO order_statuses (status_code, description, display_seq, effective_start_date)
    VALUES ('AWAITING_STOCK', 'Awaiting Stock', 30, SYSTIMESTAMP);

Every historical order referencing 'PENDING' remains fully valid. Meanwhile, any active query or UI dropdown filters on effective_end_date IS NULL (or evaluates whether SYSTIMESTAMP falls between the start and end dates). New orders cannot select ' PENDING'. Old orders cannot be corrupted.

Better yet, temporal versioning gives you future-dating for free. When operations announces that a new status takes effect on January 1, you insert the row today with an effective_start_date of January 1. At midnight, it activates automatically. No midnight deployments, no emergency patches, and zero table locks.

The Metadata Leak

A CHECK constraint treats an enum as a naked, isolated literal. But business vocabulary never stays naked.

Before long, the rest of the team needs context:

• The user interface needs human-readable labels and a coherent sort order rather than raw database codes.
• Reporting pipelines need to know which statuses represent terminal states versus active work in progress.
• Audit systems need to know what the valid vocabulary looked like on a specific date two years ago.

In a reference table, those attributes have a natural home (display_seq, effective_start_date, description). The database acts as a shared, queryable source of truth.

In a CHECK constraint, the database cannot hold that context. So the context leaks. It leaks into frontend TypeScript switch statements, backend YAML files, and hardcoded reporting filters. You didn't avoid complexity; you just forced your schema to live in five different application files instead of the database.

The Portable Rule

This brings us to a reliable boundary for schema design:

• Use CHECK constraints for mathematical, temporal, and physical invariants. quantity > 0. percentage BETWEEN 0 AND 100. end_date >= start_date. Cross-column requirements like requiring a card token when payment type is credit card. These are arithmetic and physical boundaries. They do not drift because of a company rebrand.
• Use Lookup Tables for business vocabulary and lifecycle states. Order statuses, workflow stages, customer tiers, reason codes. Anything that product owners or executives might refine next quarter.

If changing the rule violates the laws of mathematics or calendar time, it belongs in DDL. If changing the rule requires a conversation with product management, it belongs in a table.

Scoped Optimization and AI

In my reply to Tom on that earlier post, I noted that keeping enums out of CHECK constraints is one of the fundamentals that requires an explicit waiver in my coding assistant instruction files.

It is worth asking why AI assistants reach for CHECK (status IN (...)) almost every single time they are asked to scaffold a schema.

The assistant is not being lazy. It is behaving exactly like an application developer focused on a single user story: it is correctly optimizing for a scope that does not include the third fiscal year.

Within the boundaries of a single prompt, a self-contained CHECK constraint satisfies every immediate functional requirement. It compiles cleanly in thirty lines of generated SQL, avoids creating auxiliary tables, and avoids the cognitive overhead of foreign keys. The model's horizon is the current response window. It has no reason to care what happens when marketing changes 'ASAP' to 'SOON' thirty-six months from now.

If we don't guide our tools with clear fundamentals, they will produce code that is locally optimal and globally fragile.

Take the time to build the reference table, give it proper temporal boundaries, and establish the foreign key. It is the only way to ensure that when your data models meet the third fiscal year, the database is still the place telling the truth.

---

The arguments are mine. Drafted with Gemini 3.8 Flash.

Every JSON Column Has a Schema

The arguments are mine. The typing was not.

Let's complete the trilogy (That's probably a lie...).

Over the last two weeks, we've picked apart the boolean:

• In the warehouse (OLAP), a boolean duplicates a truth that already exists elsewhere (your temporal boundaries).
• In an operational system (OLTP), a boolean destroys a truth that did exist (wiping out history and sequence).

A JSON column does something far sneakier: it never declares a truth, so nothing can ever contradict it.

The Unfalsifiable Blob

A shapeless blob cannot be wrong, because there is no declared shape for it to violate.

Think about what happens when you create a proper relational column. You declare customer_id INTEGER NOT NULL REFERENCES customers(id). You have drawn a hard line in the sand. If the application tries to insert 'banana', the database rejects it. If it tries to insert an orphaned customer, the database rejects it. The engine knows what is true and what is a lie, and it protects you.

Now look at a JSON column.

• {"customer_id": 123} is valid JSON.
• {"customer_id": "123"} is valid JSON.
• {"customerId": null} is valid JSON.
• {"cust_id": "banana"} is valid JSON.
• {} is valid JSON.

The database accepts every single one of those rows without blinking. It writes them to disk, returns a 200 OK, and goes about its day. Why? Because you never declared a contract. You never told the database what "right" looks like, so nothing can ever be "wrong."

Until six months later, when the reporting query blows up. At that point, you don't have a data model: you have vibes and a support ticket.

Watching From the Outside

I know why people do it. A new feature lands on your desk, the product manager is still fuzzy on the specs, the attributes will probably change next sprint, and you just need to get the code out the door. So you slap a payload JSON column on the table and tell yourself you're being agile.

I've never done this. Not with JSON.

Booleans, sure. I will readily admit to that sin. I've slapped an is_active flag on a table and paid for it later. But dumping application objects into a JSON column is one I have only ever watched from the outside, which is its own kind of education.

If you come from the database world, the foundational rule has always been simple: put integrity constraints as close to the data as humanly possible. The database exists to protect the data from the application, because applications get rewritten every two years, but data lives forever.

When you drop an unconstrained JSON blob into a table, you're betting that every future engineer touching that application will remember to enforce every single implicit business rule in code.

Spoiler: they won't.

Rebuilding the Schema, Badly

Someone reading this will inevitably push back: "Wait, you can enforce integrity on a JSON column!"

And you can. Modern engines will let you bolt integrity back onto a document. Postgres will happily take a CHECK constraint on an extracted jsonb path. You can create generated columns with foreign keys, and you can build functional GIN or B-tree indexes against nested attributes.

Technically, you can do it. But look at what you are actually doing: you are reconstructing, one painful piece at a time, the relational schema you declined to write in the first place, using an esoteric, vendor-specific syntax that nobody on your team will recognize in a year.

You didn't avoid the schema. You just decided to rebuild a worse version of it by hand.

Relocating the True Cost (and Closing the Loop)

To be clear: this isn't about beating up on application developers.

When a developer drops a JSON column into a migration, they aren't trying to sabotage the company. They are responding to very real, very rational pressures: sprint deadlines, velocity metrics, and avoiding the friction of formal schema reviews. From their seat in the sprint, skipping the table design feels like pure efficiency.

Years ago, Cary Millsap wrote a post about formatting tables of numbers that contained an insight I have quoted many times:

"Good design is a topic of consideration. And even conservation. If spending 10 extra minutes formatting your data better saves 1,000 readers 2 minutes each, then you’ve saved the world 1,990 minutes of wasted effort."

Cary's math is irrefutable, but appealing to civic virtue ("save the world 1,990 minutes") rarely changes engineering behavior on its own. What actually changes behavior is seeing the feedback loop close.

When you dump a raw, shapeless JSON blob into a table, you didn't eliminate the work. You simply relocated the cost.

In the short term, you quietly transferred that cost downstream to the analytics engineers and BI developers. Every single report now requires defensive SQL: unnesting arrays, casting strings to integers, guessing at nulls, and handling three different key spellings.

But the loop doesn't stop with the analytics team.

Eventually, product asks for a new operational feature: an in-app filter, a bulk edit, or a performance dashboard built directly against that transactional table. And guess who gets assigned the ticket? The application developer.

Now, the very engineer who bypassed the schema to save twenty minutes in sprint four is staring at a production bug in sprint twelve, trying to write unreadable JSON path queries against their own shapeless blob. You didn't save time. You just deferred the agony, with interest, back onto your future self.

The Waiver

In my own agent instructions, a JSON column requires a waiver: the default answer is no, and the burden of proof is on the column.

When does it actually earn that waiver?

When the data is genuinely an opaque, third-party black box that the database never needs to reason about: raw webhook payloads, audit logs, or configuration blobs that the engine will never filter, join, or aggregate on.

If your application needs to query it, if your business needs to report on it, or if it relates to any other entity in your system: it belongs in a column.

Doh. It really is that simple.

The Boolean is Still Lying to You: The OLTP Edition

The arguments are mine. The typing was not.

Let's get straight to the point (again): a boolean should never be your first move in a physical data model.

Last week, in The Boolean is Lying to You, we talked about how a boolean could ruin an OLAP dimensional model. In the warehouse, the problem with a boolean is that it duplicates a truth that already exists elsewhere (temporal boundaries).

But the temptation of the boolean doesn't vanish when you switch to an OLTP system. In a transactional database, the problem is the exact opposite: it destroys a richer truth and replaces it with a poorer one.

If anything, the drive for immediate application convenience has made it worse. You need to know if a user account is active, if a record is deleted, or if an order is shipped. The developer reflex is to slap an is_active, is_deleted, or is_shipped flag on the table.

Just like in the warehouse, the boolean is lying to you. In an operational system, lost fidelity means lost business context.

The is_deleted Tragedy: Timestamp Trumps Boolean

The most common offender is the soft delete: is_deleted = true.

It seems harmless. The application filters out WHERE is_deleted = false, and your data is "safe". But in an operational system, knowing that something was deleted is rarely enough. Within weeks, the business will ask: When was it deleted? Who deleted it? How long was it active before it was removed?

Your boolean is mute. It destroyed the temporal context of the event.

Instead of is_deleted, your first instinct should be a deleted_at timestamp. If deleted_at IS NULL, the record is active. If it's populated, you know exactly when the state changed. The application's WHERE clause is just as simple, but the database retains the full fidelity of the event.

The Boolean Pile-Up: Where State Machines Go to Die

Business processes are rarely binary. They are lifecycles. They are state machines. But the path of least resistance often leads to modeling these state machines as a pile of mutually exclusive booleans.

It starts innocently with is_draft = true.
Then the business process evolves, so we add is_published.
Then we need to pull it down temporarily, so we add is_archived.

Now you have a record where is_draft = true AND is_published = true. What does that mean? It means your application allowed an invalid state because your physical model didn't enforce mutual exclusivity. You forced the application code to manage the integrity of the state machine, and eventually, the code will fail.

If a record moves through a lifecycle, use a status_code (backed by a reference table) or an event-sourced ledger. A single status column makes mutually exclusive states explicit and enforceable. Booleans just allow for combinatorial explosions of invalid states.

The Tri-State Lie

A boolean promises two states: True or False.
But in a SQL database, a nullable boolean actually has three states: True, False, and NULL.

What does a NULL boolean mean in your application? Does it mean "False"? Does it mean "Unknown"? Does it mean "Not Applicable"? When is_verified is NULL, did the verification fail, or has it just not happened yet?

When you use a boolean to represent business state, you inevitably back yourself into relying on this ambiguous third state. If you have three states, you don't have a boolean. You have a poorly labeled lookup table.

The Indexing Bonus: The Nerd Special

There's a physical performance argument here, regardless of which database engine you use.

Let's say you have a transaction processing table, and you use is_processed = false to find work that needs to be done. If you index that boolean, you're indexing the entire table.

Instead, if you use a processed_at timestamp, you get a massive performance feature for free by using a partial (or sparse) index.

By creating an index specifically for the rows WHERE processed_at IS NULL, your index only contains the tiny fraction of records that actually need processing. It stays perfectly sparse, incredibly small, and lightning fast. A generic boolean flag robs you of this elegant optimization.

Downstream Devastation: Why the Warehouse Cares

It’s tempting to think that an OLTP shortcut only affects the application layer. But the damage flows downstream. When you overwrite a state with a simple boolean flag, you aren't just making a lazy choice for the transactional app—you are permanently destroying data that your analytical systems desperately need.

• Transitions are destroyed. If you just update is_converted = true or is_canceled = true, you only know the final outcome. You lose the sequence of events. You can no longer calculate the duration between states, identify bottlenecks, track SLAs, or analyze churn. Operational analysis, process mining, and machine learning all require transitions to figure out why something happened. A boolean destroys the transition and leaves you with a tombstone.
• Time-Travel and CDC (Change Data Capture). When your OLTP system relies on event logs, timestamps, or explicit status histories, extracting that data into your OLAP environment is deterministic and robust. You can perfectly reconstruct what the business looked like at any given second. If you rely on flipping a boolean in place, you force the warehouse to frantically poll and capture those fleeting changes before they are overwritten again, inevitably missing rapid transitions.

Stop Hiding the Business Process

In OLTP, the database is the engine of the business. When you reduce a business event (like a cancellation, a deletion, or a publication) to a boolean flag, you are erasing the context of that event. You are optimizing for a temporary application shortcut instead of modeling the reality of the domain.

Just like in dimensional modeling, the rule stands: a boolean requires a waiver. Don't reach for it just because it's easy. Force yourself to ask: "Does this state have a history? Does it have a timeline? Is it part of a larger lifecycle?"

Almost every time, the answer is yes. And almost every time, the boolean is the wrong move.