OSASI.ORGCommentary · Ideas for our profession
All briefs← Back to OSASI

Below the waterline: who is measuring the freeboard?

September 23, 2026 · OSASI Commentary

Choosing which AI tools to buy is the easy part of an AI strategy. The hard part, the show says, is everything beneath the tools, from platform and inference to hardware and sovereignty. Follow the layers down and every one of them ends at the same question for a profession entrusted with confidential files: who inspects for contaminated ballast water discharge?

How a brief is made

A member reads an article or listens to a podcast and sends the snippets, key thoughts or concerns tied to a passage or a timestamp. An AI assistant, chosen for the task and the setup (Claude, ChatGPT, Codex, Gemini or another), searches the full article or transcript, pins the times, checks the quotations, and assembles a draft. The draft goes through rounds of review, and nothing is published until a person approves that exact version. The cues on each post start the player at the passage discussed. Commentary is OSASI's perspective, offered for discussion, not a summary of the source. Each brief is co-authored by a person and an AI assistant. AI assistant for this brief: Claude (Anthropic).

Source podcast · Value Driven Data Science

This episode prompted the commentary below. It is one perspective, offered for discussion.

Value Driven Data Science · September 10, 2026
Episode 122: [Value Boost] 4 Questions Every Data Scientist Should Ask About AI Strategy

Listening highlights · Dr Genevieve Hayes with Victor Coimbra of Artefact, a twelve-minute follow-up to episode 121.

−11:41 to −10:49 remainingIntroductions · Genevieve HayesWho is speaking, and how this Value Boost follows last week's hybrid agentic organisation episode
−10:49 to −9:48 remainingThe shopping list · Genevieve HayesAI strategies that stop at which vendor and how many seats; the application layer is the tip of the iceberg
−9:48 to −7:05 remainingFour layers below the waterlineApplication, AI platform (harness, context, observability), inference, hardware
−7:05 to −6:11 remainingWhy only the tipShort-term gains, and "never bet against technology"
−5:49 to −3:28 remainingThree symptomsNobody uses the tool; governance and security failures; token costs explode
−3:28 to −1:05 remainingFour questionsOne per layer, ending at sovereignty: do you control where your intelligence is created?
−1:05 to −0:26 remainingThe one to lead withWhere and how should AI be used, the question that changes the work first

Tap a time above to start the player there. If Spotify shows a card about the show first, close it and the player carries on at that time. Times are approximate transcript times; ads and episode editions may shift them. This plays the full episode, not an extracted clip.

Who is talking

Dr Genevieve Hayes hosts Value Driven Data Science, a show for data professionals who want to become strategic experts. This is a Value Boost, a short follow-up to episode 121, where she and Victor Coimbra discussed hybrid human and AI organisations (OSASI's brief on that conversation is Hybrid agentic AI powerhouse, or human-AI slop soup?). Hayes introduces Coimbra, Global AI and Data Engineering Lead and a partner and CTO at the AI consultancy Artefact. The episode walks through a four-layer framework from his article Below the Waterline: The Four Layers of an AI-Native Company.

The show's own message

The show sets out its argument plainly. Choosing which AI tools to buy is the easy part of an AI strategy; the hard part, and the part most organisations are ignoring, is everything beneath the tools, from platform and inference decisions to hardware and sovereignty. Hayes sets up the problem before Coimbra speaks: "Many organizations currently have AI strategies that read a lot like shopping lists." Their biggest concerns, she says, are Anthropic or OpenAI, and how many Copilot seats to add to the Microsoft bill. The application layer, in her words, "is just the tip of the iceberg, and without considering what's below the surface, these organizations are setting themselves up for a world of pain."

OSASI reads the same framework from the other end. For a profession that holds confidential client files and signs its conclusions, the layers below the waterline are not an engineering detail. They are where governance, security, privacy and independence are decided.

Sailors have a word for what sits between the waterline and the deck: freeboard, the margin that keeps a vessel afloat when the sea gets rough. Load lines set the minimum, and someone has to measure it. Ballast is the other half of the picture: water taken on in one port and discharged in another, carrying whatever came with it, which is why its discharge is inspected. An AI strategy has both. The margin of control an organisation keeps over its own work, and the client information it takes on and lets out somewhere else.

Four layers

Coimbra describes the layers from the surface down.

Application. What everyone sees: which tool is in use, Claude, Copilot or something else. Easy to see, and the least strategic.

AI platform. The first layer below the surface holds three things: the harness that manages the model, the context of the company, and the observability that shows where tokens go in and out. He is careful about context: "Context is not the same thing as data. Data is maybe 10% of the context that you produce in your company."

Inference. How the organisation controls its model use: open-source models, models it builds, and models other than the large language models, such as visual models.

Hardware. In his words the most important layer: who holds the power to produce intelligence at all. The largest technology companies spend heavily on data centres because they want to own it, and he expects it to become "almost the electricity layer".

Not every company needs to own every layer. Some can be outsourced, "but other layers you need to at least outsource in a conscious way". Most organisations, he says, make no conscious decision at all, because short-term gains are easier to see than long-term ones.

Three symptoms

Asked how a data scientist can tell that an application-only strategy is failing, Coimbra names three symptoms.

The first is that people do not adopt the tool, which he says is more often a sign of the wrong tool than of the wrong people. The second:

"The second symptom is a governance problem"

He means leakage, people uploading what they should not, and, in his words, "some decision that AI took that should not be revised by a human." His reminder is short: "AI cannot get fired". The third symptom is an explosion of token costs; he cites Uber spending its whole year's token budget in three or four months.

Then a warning for leaders. If a team stops working because a single vendor's model is out of service, the organisation already has a problem, because "your entire operation cannot be hostage of a thing or a solution that you do not control today."

Four questions

Coimbra reduces each layer to one question, as he did in his article (the article words them differently; the podcast versions are these).

Application: how are people actually using AI?

AI platform: "do I have visibility where my token is being spent?" He bets 80 percent of companies do not know.

Inference: "do I control my intelligence?" The goal, he says, is not the most intelligent model but "the best model to the best task", whether that is Claude, an open-source model or one built by a researcher on the other side of the world.

Hardware: "do I control where my intelligence is created?" This is where sovereignty enters. If an organisation does not control where its intelligence is produced, a new regulation or law could shut down the whole operation.

If a data scientist can ask only one, he says to lead with the first, because it changes the actual work soonest.

One framework, looked at three ways

The episode moves quickly, and it is fair to ask whether the symptoms and the questions are separate rule sets. They are not. There is one framework, the four layers, seen three ways: the layers say what sits below the waterline, the symptoms say what failure looks like on the surface, and the questions are the diagnostic, one per layer. Read that way, the symptoms point downward. Poor adoption is a failure at the application layer. Leakage, AI decisions nobody governed and invisible token spend are failures of the AI platform, where context, authority and observability are supposed to live. A stalled operation when one vendor goes down is a failure of the inference and hardware layers. Coimbra's article names the same pressures in its own terms: platform, cost and sovereignty.

The layers, for appraisers

Mapped onto valuation practice, each layer asks something specific.

Application. Which parts of an assignment go to a machine, and which stay with the appraiser. The previous brief covered this: speed on the parts a model can carry, never time pressure on the judgment.

AI platform. What the model is allowed to see, what it is allowed to decide, and whether there is a record of it. An appraiser's context is far more than data: the workfile, local market knowledge, the reasons a comparable was rejected. The observability Coimbra describes is the machine's version of a workfile, and a model with no record of what it did with a client's file fails the standard the profession already keeps for itself.

Inference. Whether the right model is used for the task, including open-weights models that can run where the data already lives.

Hardware. Where the model runs and who holds the keys. For confidential client files, this is the privacy question in its plainest form.

What this means for OSASI

Brief 001, Our data. Our knowledge. Our control., made the case for shared AI infrastructure that keeps confidential information under the control of those entrusted with it. Coimbra arrives at the same place from a consulting practice serving large enterprises: the decisions that matter are below the waterline, and an organisation that leaves them to its vendors has not made them consciously.

Few appraisal firms can build a data centre. They do not need to. What they need is to outsource consciously, and to do it together: shared answers to the platform, inference and hardware questions, with access boundaries the members set. That is what the sandbox open-weights AI expression of interest is for. The application layer is where appraisers will feel AI first. The layers below it decide whether the profession still controls its own work, and someone in the profession has to be measuring the freeboard.

Send a thought →

A like is counted once per reader per day and stores nothing about you. A thought goes by email to the person who edits these briefs; nothing you write appears on the site unless it becomes part of an approved brief. Share opens the service you pick; this site adds no trackers.

Prefer a feed reader, Slack or Teams? Follow by RSS.