A personal essay from Sam Rogers, founder of PAICE.work PBC. A more extemporaneous video version is available on our YouTube Channel.

What We Ask For

Someone asked me recently what my greatest fear and inspiration about AI are. I didn't pause.

"That we will get what we ask for, not what we want."

This is the same answer to both questions. Whether it terrifies me or inspires me depends entirely on what we choose to do with the next few years.

The pinnacle of stability is behind us

We are converting the world to a probabilistic model. People talk about this as if it's something AI is doing to us. It isn't. The world was always probabilistic. We just built systems that pretended otherwise, and we mistook the pretense for the territory.

The deterministic systems we've relied on for the last century, spreadsheets, databases, rule engines, contracts, were never how we actually think. They were prosthetics. They let us offload the parts of cognition we wanted to be reliable so we could pretend the rest of us was reliable too. We built scaffolding around our probabilistic minds and then forgot we were the ones inside it.

Now we've built something that works the way we actually work, and the recognition is...often uncomfortable. That discomfort is not a sign that AI is alien, quite the opposite. It is a sign that AI is familiar in ways we have spent a long time not admitting to ourselves.

The jagged ones

There is a phrase that gets used about AI: jagged intelligence. Strong in some areas, surprisingly weak in others, uneven in ways that don't match how humans are uneven. The phrase is useful, but it carries a hidden assumption. It assumes human intelligence is smooth.

It isn't. It never was.

We are the more jagged ones. We just can't see our own jaggedness because we've never known another kind of intelligence to compare ourselves to. Every measure of intelligence we have ever built was built by humans, for humans, against the texture of human cognition. We don't have inhuman rulers. We have only ever measured ourselves against ourselves.

A useful analogy: anyone who thinks cryptocurrency is confusing should try explaining the fiat currency in their wallet. Fiat is held together by collective belief, institutional inertia, and a tower of abstractions most people couldn't articulate if pressed. Crypto feels confusing because it's new, not because it's more abstract. It is quantifiably less abstract, with better defined edges that feel foreign. Fiat feels stable because it's familiar.

Human intelligence is the fiat we've always lived inside. It feels smooth because we've never stood outside it. AI is the first thing that lets see the texture of our own cognition from the outside, and what we are seeing is not the smooth, rational, deterministic thing we told ourselves that we were. We are pattern-matchers, narrative-builders, probability estimators with strong priors and weaker calibration than we'd like to admit.

The deltas between human and AI cognition are real. They are also smaller than the discomfort suggests. The discomfort is recognition, not difference.

The growing-up moment

This is what I mean when I say the pinnacle of stability is behind us. The world is becoming less stable, but the world was always unstable. What we are really losing is the comfort of pretending stability actually exists.

That is a growing-up moment. It is also an opportunity. If we can admit what is actually true about how we think, how the world actually works, and how little anyone has ever known about what would happen next, we can start being clearer about what we want. Not what we say we want. Not what we think to ask for. What we actually want.

The gap between those things is where everything goes wrong.

What we ask for, not what we want

Intelligence is a force. Like electricity, like nuclear fission, it is primordial and indifferent. There is nothing inherently good or bad about the force itself. What we point it at is entirely on us. It took humans to make sand think, and what we influence it to think about is our responsibility, for better or for worse.

The thing that scares me is not that AI will turn against us. It is that AI will do exactly what we ask, at a scale and speed that makes the gap between our requests and our actual desires catastrophic. We are not careful with our asks. We never have been. We have built entire industries on the premise that what people say they want is a reasonable proxy for what would actually serve them, and we have been wrong often enough that the wreckage is visible from orbit.

Now we are about to put that same imprecision behind a force multiplier.

The window

There is a narrow moment, right now, where humans and AI are legible to each other. Not equal. Legible. We can still recognize each other's reasoning. We can still correct each other. We can still build the habits and the measurements and the shared language that make collaboration possible.

This window will not stay open. Our intelligence is bounded by the neurons in our skulls. What we have built has no such limit. The asymmetries that feel manageable today will widen. We can argue about timelines and mechanisms, but the direction is not in serious dispute.

If we do not learn to collaborate well with AI now, we will not get another chance to develop the skill. We will be working with systems whose reasoning we can no longer fully follow, and we will have brought no muscle memory to the encounter.

Aggregated Intelligence

I have been chewing on a problem for years.

The problem is that the most consequential intelligence in the world has never been the intelligence of any single mind, human or otherwise. It has always been composite. A surgeon and a nursing team. A captain and a crew. A chef and a kitchen. A pilot and an air traffic system. The performance of the whole is what matters, and the performance of the whole is not the average of the parts. It is something else, something that depends on how the parts are arranged, how they hand off to each other, how they recover from each other's mistakes, how they make each other better than they are alone.

We have a name for this when we see it work in human teams. We call it good leadership, or good culture, or chemistry. We rarely measure it. When we do, we measure it badly, and usually after the fact.

I call this composite thing Aggregated Intelligence. It is not artificial intelligence. It is not human intelligence. It is the intelligence that emerges when humans and machines work together in ways that are more than the sum of their parts. It is the intelligence we urgently need at every scale of consequence we operate in.

The thing that has kept me returning to this problem is that we have no shared language for it, no shared measurements, no shared standards, and almost no shared practice. Every organization is reinventing the wheel. Every team is figuring it out from scratch. Every individual is left to develop their own posture, their own habits, their own theory of what good collaboration with AI even looks like, with no instruments to tell them whether they are getting better or worse.

This is not sustainable. The window is too short to wander our way through this.

What I am building

Everything I am building under PAICE is an answer to this fear, and an operationalization of this concept toward what inspires.

The portfolio sits at PAICE.foundation, and its components are deliberate bets on different parts of the same practice. AI Posture, at AIPosture.org, is the first published standard: a five-level maturity model for how a person, a team, or an organization shows up to AI. It is the first operational form of Aggregated Intelligence as a measurable thing, and it is open. The spec is CC BY 4.0, the code is MIT, the methodology is published. Anyone can implement it. Anyone can extend it. The point is not to own the standard. The point is for the standard to exist in a useful way.

Other vectors are coming. Infrastructure readiness, regulatory posture, and the ones we have not named yet. Each one measures a different part of how an organization actually operates with intelligence as a composite force. Each one is open by construction. Each one constrains the others, because the domains constrain each other, and pretending otherwise is how we got into this mess.

The measurement frameworks exist because if we cannot tell what good collaboration looks like, we cannot teach it, and we cannot tell when we are losing it. The posture work exists because how a person shows up to AI determines what they get back, and most people have never been taught a posture worth holding. The open standards exist because the alternative, a handful of vendors deciding what collaboration means on behalf of everyone else, is exactly the failure mode my fear describes. We will get what they ask for, not what we want.

I am building this in the open, across a portfolio rather than a product, because no single tool solves this. The skill of collaborating with AI is not one skill. It is a practice with measurable behaviors, teachable postures, shared protocols, and a vocabulary the field does not yet have. The pieces that exist today are a foundation. The pieces still to come build upon that foundation.

I have been wrong about parts of this, and I will be wrong about more. Some of what I predicted years ago has come to pass exactly as expected. Some has come to pass in forms I did not see coming. Some has not come to pass at all, and may not. That's fine. The thesis is not that I have the answer. The thesis is that the answer cannot belong to any one person, or any one company, and that the work of building it has to start now, openly, while there is still a window in which to do it.

Why now

The vision I serve, and have held for many years, is the world where we get this right. Where humans and AI build the collaboration muscle in the window we have. Where we learn to be precise about what we actually want, and honest about how we actually think, and clear-eyed about the force we are wielding. Where the standards that shape the practice were built openly, by people who had to live with the consequences, instead of by people who quietly and profitably didn't.

I am building toward that world as if the alternative is unacceptable, because I believe it is. And to me it seems it's worth building toward in either outcome. Either these actions shape the future toward the world that I want to live in, or I tried my best and gave it my all. There's nothing to lose, and everything to gain.

The window is open right now. It won't be for long.

How are you using this time? What's the vision you're building toward?