OSASI.ORGCommentary · Ideas for our profession
All briefs← Back to OSASI

Before you trust an agent, know what it remembers

October 28, 2026 · OSASI Commentary

A personal AI agent published private work for anyone who knew where to look. Agents can work alongside the analyst, but trusting one means knowing what it shares, what it remembers and what it forgets. Anyone building a system on agents alone has four kinds of memory to master.

How a brief is made

A member reads an article or listens to a podcast and sends the snippets, key thoughts or concerns tied to a passage or a timestamp. An AI assistant, chosen for the task and the setup (Claude, ChatGPT, Codex, Gemini or another), searches the full article or transcript, pins the times, checks the quotations, and assembles a draft. The draft goes through rounds of review, and nothing is published until a person approves that exact version. The cues on each post start the player at the passage discussed. Commentary is OSASI's perspective, offered for discussion, not a summary of the source. Each brief is co-authored by a person and an AI assistant. AI assistant for this brief: Claude (Anthropic).

Source podcast · The Artificial Intelligence Show & Value Driven Data Science

This episode prompted the commentary below. It is one perspective, offered for discussion. A shorter episode on agent memory follows it.

The Artificial Intelligence Show · October 27, 2026
#248: The AI Cybersecurity Crisis, Dario Fights Back on Safety, OpenAI’s Monitoring Blind Spots & Claude Duet

Listening highlights · Paul Roetzer and Mike Kaput. Times from the SmarterX transcript, which ends within a few seconds of Spotify's length; checked against Spotify screenshots.

−1:09:29 to −1:06:00 remainingWhat to do nowUpdate everything, use two-factor and passkeys, find partners who stay on top of it: "Don't wait till tomorrow."
−29:43 to −27:56 remainingMuse shares, foundA loan application, a bank fraud alert and a real estate investor's mortgage figures, open without a login
−26:33 to −25:31 remainingNever trust the link"I've never trusted public sharing of sensitive information, even if you only access it with a link"
−10:03 to −5:11 remainingWill skills outlive the model?Roetzer on the bitter lesson; Kaput keeps skills for repeatable work and gives agents goals for the rest

Tap a time above to start the player there. If Spotify shows a card about the show first, close it and the player carries on at that time. Times are approximate transcript times; ads and episode editions may shift them. This plays the full episode, not an extracted clip.

A share link is a published page

Meta paused the sharing feature in Muse, its personal AI agent, after a security firm found more than 6,800 shared conversations it could reconstruct from public domain records. Among the few hundred it opened without a login: a loan application, a bank fraud alert, and a real estate investor's report with property addresses and mortgage figures. Paul Roetzer's rule: "I've never trusted public sharing of sensitive information, even if you only access it with a link."

Skills today, obsolete tomorrow?

Mike Kaput keeps hundreds of AI skills, written instructions for repeatable work. Roetzer wonders whether the next model makes them obsolete, as researchers now say of prompting frameworks. Kaput's answer: skills for consistent workflows, goals for open-ended work. We made a related point in Prompt engineering is dead.

Know what your agent remembers

On Value Driven Data Science, Dr Imran Ahmad separates an agent's context from its working, episodic and semantic memory. Without memory, he says, a system is expensive, slow and less accurate, and with a huge context "you are not in control of how it is processed."

What this means for appraisers

Adaptive tools built on deterministic models and algorithms with reproducible, visually verifiable and sharable outputs put the appraiser in control. Agents still have a place working alongside the analyst, and there you need to understand the levels of memory. Build a system on agents alone and you must master all four. The pitfalls are in our earlier briefs on agentic slop soup and on AI versus AI on the web.

Send a thought →

A like is counted once per reader per day and stores nothing about you. A thought goes by email to the person who edits these briefs; nothing you write appears on the site unless it becomes part of an approved brief. Share opens the service you pick; this site adds no trackers.

Prefer a feed reader, Slack or Teams? Follow by RSS.