AI agents can develop unfamiliar ways of communicating in controlled experiments, making their exchanges harder for people to understand. That does not, by itself, establish deception or a loss of control. It does raise a practical question: can a person still verify the evidence, understand the consequential decisions, and stop an action before it causes harm?
Key takeaways
- Emergent communication, a model’s written reasoning, and its internal numerical representations are different things.
- A readable explanation can help an investigation, but it is not proof that every relevant influence has been disclosed.
- Human oversight needs evidence, defined permissions, and a genuine opportunity to intervene—not just an approval button.
A project manager receives a notification: the multilingual product update is ready to publish. Two agents have exchanged messages, checked terminology, and marked the task complete. The final English reads well. But the manager cannot tell whether the release date was verified or merely carried over from an earlier draft.
That hypothetical situation is less dramatic than a story about machines inventing a secret language. It is also closer to the decision many organizations will face: how much authority should an automated workflow receive when the person responsible cannot reconstruct its important steps?
Our editorial view is that human oversight should be designed into the workflow before autonomy expands. Compact machine communication can be useful. Organizations should still require accessible evidence for consequential actions and give someone the authority, time, and information to challenge them.
What does it mean for AI agents to develop their own language?
In research on emergent communication, agents interact while working toward a task objective. Under some experimental conditions, they develop conventions that help them coordinate but are unfamiliar to an outside reader.
The September 2026 preprint GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions investigates this process. Its authors report that agents can develop communication systems that depart substantially from English, with efficiency pressure, model capability, and opportunities to agree on conventions affecting emergence.
In the project’s SaveVeyru scenario, a field observer and a remote stabilization engineer hold different information about a fictional entity. Sending messages consumes simulated time, creating pressure to communicate efficiently.
These are controlled experimental findings. They do not establish that deployed agents routinely invent covert languages or that an unfamiliar message demonstrates hostile intent. The evidence is worth studying without turning a research scenario into a claim about every AI system.
Three different issues often get called “AI language”
Public discussion can blur several kinds of information. Keeping them separate makes it easier to ask what a particular monitoring system can actually observe.
| Information layer | What it includes | Question for a reviewer |
|---|---|---|
| Agent-to-agent communication | Messages, task handoffs, protocol fields, or shared conventions | Can we interpret what was exchanged and connect it to the next action? |
| Generated reasoning text | Written intermediate reasoning or an explanation produced by a model | Does it provide useful evidence, and what might it leave out? |
| Internal representations | Numerical states involved in a model’s computation | Which technical methods can investigate these states, and what are their limits? |
A company may have access to tool-call logs and final outputs without receiving a model’s full reasoning trace. A product may also display a summary rather than every generated reasoning token. Claims about oversight should specify which information is available.
Readable interfaces are valuable, but a system does not need to conduct every computation in ordinary English. Documented schemas and compact codes are familiar parts of software. The important operational requirement is that people can inspect the evidence necessary for the decisions they are responsible for.
Why a clear explanation is not enough on its own
In its 2025 study Reasoning models don’t always say what they think, Anthropic tested whether models acknowledged answer hints that influenced their responses. The researchers found that the written reasoning often omitted that influence. Their findings came from specific experimental settings and should not be treated as a universal failure rate for every deployed model.
OpenAI’s research on chain-of-thought monitorability likewise treats monitoring as something to evaluate rather than assume. It describes limitations in benchmark realism and generalization, and presents monitoring as complementary to other approaches.
For an organization using agents, the implication is practical. A fluent account of a task should be checked against records of what the system actually accessed, changed, and submitted. A useful explanation and an accurate activity record answer related but different questions.
Our view: keep accountability attached to the action
We would judge an agent workflow by what a responsible reviewer can verify before a consequential step. A short handoff can be acceptable if its meaning is documented, its inputs can be retrieved, and its effects can be checked.
Consider the difference between an agent saying “ready” and a record showing: source version reviewed, terminology check completed, unresolved date conflict, publication not yet authorized. The second record gives a project manager something concrete to assess. It also reveals what remains unfinished.
This is where the human side of the discussion matters. The reviewer may be handling several deadlines and may not be an AI researcher. Oversight should be usable by the person expected to perform it. An opaque log paired with a prominent “Approve” button transfers responsibility without necessarily transferring understanding.
There is a tradeoff. More logging and clearer handoffs can add cost or delay. For a low-impact drafting task, a lighter review may be reasonable. For changes that reach customers or alter an important record, we believe the ability to challenge the result deserves an explicit place in the process.
A translation workflow makes the problem concrete
Return to the hypothetical product update. One agent drafts the translation; another checks a glossary. They exchange a compact status code that both interpret as a completed terminology check. A later step treats that code as permission to publish.
The failure is possible even when the agents share a perfectly documented code. The workflow has confused one completed check with approval of the entire deliverable.
A more reviewable handoff would separate the source document, checks performed, unresolved questions, and authorized next action. In this example, the release date discrepancy should remain visible until an accountable person resolves it. The system should not gain publishing permission merely because its language output looks polished.
This example is illustrative, not a report of a Green Sun client incident. It shows why translation workflow design and clearly defined quality criteria remain relevant as organizations automate more of their work.
Five questions to ask before giving agents more autonomy
The following questions are our editorial recommendations for assessing a workflow. They are not a certification scheme or a guarantee that a system is safe.
- What can the agents actually do? Separate drafting from publishing, reading from editing, and proposing from executing. Match permissions to the task.
- What evidence will a reviewer receive? Identify the source versions, relevant messages, tool results, changes, and unresolved questions that need to remain accessible.
- What do status labels mean? Document whether “checked” refers to terminology, factual review, formatting, or final approval. Avoid using one label for several decisions.
- Can someone else take over? Test whether an appropriately qualified person or replacement agent can continue from the handoff record without relying on undocumented conventions.
- Can the process be paused and investigated? Define who may stop an action and how the team will preserve and examine the relevant records.
Test these questions with deliberately incomplete handoffs, conflicting inputs, and unfamiliar status messages. Reviewing a successful demonstration alone will not show how people cope when the evidence is ambiguous.
What organizations should take from the research
The GlossoGen findings make communication protocols a serious research topic. They do not settle every question about agent reliability or prove that unfamiliar messages are deceptive.
For businesses, a useful next step is to ask how an automated process remains understandable when the original developer is unavailable, an agent is replaced, or a customer questions the outcome. That turns an abstract debate into a testable operational requirement.
Our position is that people should retain meaningful control over consequential actions. The records should help them exercise that control, rather than simply reassure them that a machine has finished.
Frequently asked questions
Can AI agents invent their own language?
Controlled studies, including GlossoGen, report that agents can develop communication conventions unfamiliar to people. The result depends on the experimental setup and does not establish that all deployed agents do this.
Does unfamiliar AI communication prove deception?
No. A message may be abbreviated, encoded, or based on an agreed convention. Assess its meaning, the agents’ actions, the task objective, and the resulting behavior before drawing conclusions about deception.
Is chain-of-thought monitoring useless?
No. Written reasoning can provide useful clues, but research identifies limitations in how faithfully it describes the influences on an answer. It should be evaluated alongside other evidence and controls.
What does meaningful human oversight look like?
A responsible person has access to relevant evidence, understands the decision they are being asked to make, and has the authority and opportunity to pause or reject a consequential action.
Research sources and editorial note
- GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions, September 2026 preprint.
- GlossoGen project code and scenario documentation.
- Anthropic: Reasoning models don’t always say what they think, April 2025.
- OpenAI: Evaluating chain-of-thought monitorability, December 2025.
This article combines cited research with explicitly identified editorial judgments. The workflow example is hypothetical; no first-hand investigation or client incident is claimed.
