New: Governed AI for Critical Systems, on the controls regulators will ask for.New: Governed AI for Critical Systems. Read the post

Skip to main content

Assembled by Your Team, Owned by Your Team, Forever

BlogDay Two
Post 5 of 6

Thread AI Editorial · October 29, 2026

Monday, 7 a.m. A client’s HR lead writes in to say that the hiring-plan report sent to department heads over the weekend included salary ranges the board had approved for its incoming Chief Financial Officer and Chief People Officer, figures meant only for the compensation committee. The rule keeping the agent away from executive pay was a single line in a prompt, and a routine model update the week before had changed how the agent read it.

The firm’s engineers traced the failure and fixed the prompt in minutes, but the fallout is lasting: the new executives’ pay is now common knowledge inside the client.

A model update broke a limit, and executive pay leaked.

Why: Each client’s limits were set by hand, and this one lived in a prompt that a model update could change.

Working as intended until last week’s model update. The pay rule then failed unnoticed, executive pay was exposed from the weekend to Monday, and the damage lasted beyond the week.

  1. Last week

    A routine model update changes how the agent reads its prompts, for every client.

  2. The weekend

    A client’s hiring-plan report goes to department heads with the incoming CFO’s and CPO’s pay.

  3. Monday, 7 a.m.

    The client’s HR lead writes in. The firm fixes the problem, but the new executives’ pay is already common knowledge.

  4. That week

    A furious client handles the fallout. The firm scrambles to repair trust and confirm no other client’s limits broke.

  1. Last week

    A routine model update changes how the agent reads its prompts, for every client.

  2. The weekend

    A client’s hiring-plan report goes to department heads with the incoming CFO’s and CPO’s pay.

  3. Monday, 7 a.m.

    The client’s HR lead writes in. The firm fixes the problem, but the new executives’ pay is already common knowledge.

  4. That week

    A furious client handles the fallout. The firm scrambles to repair trust and confirm no other client’s limits broke.

This is the third time this quarter a limit has slipped in the firm’s hiring-plan reports, each at a different client. In all three cases, the methodology within the product worked as designed. Each failure came from how the limits were set, since they told the agent what not to touch instead of stopping it from reaching that data at all.

The consulting firm behind those reports advises companies on how to pay their people. From the beginning, the firm’s pay-benchmarking methodology lived with a few senior partners. These experts knew which peer groups held up for a given role and how to adjust when one job didn’t map cleanly to market data. Clients paid for this judgment, and demand for it eventually eclipsed availability on the partners’ calendars.

That prompted change. Two years ago, the firm encoded the methodology into an AI agent and packaged it as a product sold alongside its engagements. The product soon became a core offering, with 40 clients now using it to shape their growth and hiring plans. Each client sees the product’s suggested pay ranges for each role, refreshed against its own HR data, while partners spend their hours on engagements that still need them in the room. Product and people sell together.

Who Runs All This?

The firm built the product on a capable open-source framework and cloud services, with a separate environment for each client. Compensation data is sensitive client data, and walling each client off from the others was a necessary call.

Walling off the first client was a design decision. Walling off the 40th was a staffing one. Come planning season, the second-order question lands: Who runs all this? The firm has 40 environments to patch and 40 sets of limits to maintain, so every model update or patch could change how an agent behaves somewhere. Each client also has its own bounds. One client’s HR system can be read but never written to, and another’s agent is never supposed to touch executive pay.

That firm’s work piled up with every onboarding, since each new client meant a new environment, a new connection to its HR system, and a new set of limits defined and deployed. Most of this work fell to the handful of people who understood both the methodology and the setup, and some of them are the same partners the product was supposed to free up.

When the firm’s own hiring plan for next year circulates, it calls for more platform engineers than consultants. The firm set out to productize a playbook and ended up staffing itself like an infrastructure company.

The Question That Keeps Coming Back

The same week the hiring plan goes around, the client from that Monday morning follows up. Before renewing, its new CISO wants an answer to one question about its HR systems: “What can this agent access and do?”

The firm can’t answer quickly. Each client’s limits were set by hand at onboarding. For the Monday client, the rule against touching executive pay was a line in a prompt. For another, “read only” is a setting in a config file that only the engineer who set it up knows is there. It’s the Monday morning problem again, 40 times over. Client by client, nobody can say what the agent is allowed to touch, or what the next model update could change.

Assembled and maintained by the firm

What can this agent access?

Depends on who you ask

HR system
ask the engineer who set it up
executive pay
one line in a prompt
signed off by
unknown
last changed
unknown

Limits set by hand at onboarding, kept in prompts, config files, and people’s memory.

Built into the platform

What can this agent access?

Written into the system

HR system
read only
executive pay
no access
signed off by
client CISO and the firm
last changed
Day One, logged

Limits written into the system, where the firm and the client’s security team can both check them.

Before launch, a question like this decides whether a product ships. After launch, it comes back with every renewal and every new security lead. With 40 clients, that means the firm has to answer the question 40 times.

The Parts Are Free

The pieces were never the hard part. Servers, logins, permission tools, and agent frameworks are easy to get, and most are cheap or free. The cost comes after assembly, when the firm has to keep them running for every client, indefinitely. Each new client adds upkeep that never ends, and over time the upkeep becomes the business.

The framework and cloud services did what they promised, and the methodology passed every capability test the firm gave it. The harder questions are about what it takes to keep the product running and whether anyone besides the handful of people who built it can do that work.

A firm in this position should own its methodology without having to manage all the machinery. Two conditions make that possible:

  1. 01Each client’s limits are written into the system, where the firm and the client’s security team can both check what the agent can read, what it can change, and who signed off

  2. 02The methodology is saved in a portable format, so the firm can switch platforms or models without rebuilding the compensation advisory expertise the partners spent years refining

The first turns the CISO’s question into something the firm can look up. The second keeps the methodology the firm’s asset, one that grows more valuable with each client.

This is how a firm can own the playbook forever and leave the plumbing to the platform.

Built In

With both conditions in place, the firm’s 41st client is onboarded with bounds set and visible from Day One. Limits are enforced outside the prompt, where a model update can’t override them. And later, when a CISO asks what the agent can access and do, the firm answers by showing the record, and the partners get back to the methodology.

The model is updated. The limits stay put.

Why: Limits are enforced by the system, outside the prompt, so a model update can’t change what the agent can see.

Before: the model update broke the pay rule, executive pay was exposed from the weekend to Monday, and the damage lasted. After: the same week runs as intended throughout, and when the CISO asks what the agent can access, the firm shows the record.

Before

  1. Last week

  2. The weekend

  3. Monday, 7 a.m.

  4. That week

  1. Last week

  2. The weekend

  3. Monday, 7 a.m.

  4. That week

After

  1. Last week

    A routine model update changes how the agent reads its prompts, not what it can access.

  2. The weekend

    The client’s hiring-plan report goes to department heads with only what they’re allowed to see.

  3. Monday, 7 a.m.

    A normal Monday. Executive pay stayed with the compensation committee.

  4. That week

    The CISO asks what the agent can access. The firm shows the record.

  1. Last week

    A routine model update changes how the agent reads its prompts, not what it can access.

  2. The weekend

    The client’s hiring-plan report goes to department heads with only what they’re allowed to see.

  3. Monday, 7 a.m.

    A normal Monday. Executive pay stayed with the compensation committee.

  4. That week

    The CISO asks what the agent can access. The firm shows the record.

That is what productizing the playbook requires: people and product selling together, with the methodology reaching clients the partners will never have time to serve.

The Series

This is Post 5 of six, and the first of two about productizing the playbook. Read the thinking behind this series in the manifesto.

Post 1
Published

Your Customers' Regulators Will Ask What the Agent Did – and Why

For companies whose customers already depend on their product, adding AI means answering to those customers’ regulators too. This post follows a claims software company that ships an agent to review insurance claims, then can’t explain one of its denials to a state examiner because the reasoning was never recorded when the decision was made.

Post 2
Published

The Demo Was the Easy Part

Businesses that run their operations on a product hold every new feature to the same service commitments. This post follows an insurance software company whose overnight deploy leaves quote requests stalled mid-decision while every health check stays green, because nothing recorded where each request stopped or how to resume it.

Post 3
Published

What Can This Agent Access for Each of Your 200 Clients?

Turning an internal tool into a product for a new market pays off if it ships while the opportunity is still open. This post follows a company that plans to sell its internal dispute-reconciliation agent, then learns that keeping each client’s data, billing, and limits separate could take quarters to build, long enough to miss the opening.

Post 4
Published

A 97% Success Rate Is a Liability

Teams moving fast into unfamiliar territory often launch first and plan to iterate later. This post follows a company building its first logistics product, whose exception-handling agent slips from 99% to 97% success as volume grows, leaving 1,500 failed runs a day with nothing built to handle them.

Post 5
This Post

Assembled by Your Team, Owned by Your Team, Forever

Codifying expertise lets a firm reach clients its experts will never have time to serve. This post follows a pay-consulting firm that turns its partners’ methodology into an agent product for 40 clients, then finds that keeping each client’s limits in place has it staffing like an infrastructure company.

Post 6
Coming Soon

Productize the Playbook: Composite Two

The second playbook composite, closing the series. A different organization, the same tension: doubling down on human capital while turning part of the playbook into product.