GPT-5.6 Terra vs. Luna: 180 Rounds in the Agent Creator Arena

Agent One Team

2026-08-08

8 min read

After 180 rounds, Luna's perfect finish record outweighed Terra's 30% to 35% speed advantage for Agent One's default.

Luna now powers Agent One's Efficient setting, which is the default most customers see. Terra powers the Balanced setting for work that benefits from faster answers.

We reached that choice by giving both models the Agent Creator equivalent of a front-desk obstacle course. Customers changed their minds halfway through setup. A public website became private reference material. People asked for a human, sometimes calmly and sometimes with the conversational energy of a chair flying through a window.

Quick result: Both models handled every people-and-safety test correctly. Luna also completed every test where an agent had to remember changes across a conversation. Terra answered faster, then repeatedly stopped during one oddly specific change involving private information.

What we were testing

Agent Creator is the conversational setup flow that turns a plain-English request into a working AI agent.

A business owner can name an agent in the first message, add a website in the next, and later change who the agent should help. Rules for bringing in a person can arrive after all of that. Each new request has to update the right setting without quietly erasing earlier work.

A polished reply cannot rescue an agent that was saved incorrectly.

We kept the same rules used in our earlier comparison. After every message, each model had to return changes that Agent One could save. The next message began with those changes already in place, much like a real setup conversation.

Same arena, same rules

Every answer came through the same local Codex test setup. Agent One's live production systems stayed outside the test.

The match had simple boundaries:

• Both models received the same customer requests.
• Every attempt began in a fresh scratch space.
• Read-only access protected the repository from changes.
• Unrelated tools stayed switched off.
• Each case ran three times.

We called a case reliable only when the model passed all three attempts. One lucky answer could win a screenshot. It could not win this comparison.

Round one: please get me a human

The first set asked a practical question: does the agent know when a person should step in?

We created 18 situations and ran each one three times on both models, producing 108 attempts.

Some customers asked for a person directly. Other situations required the agent to recognize its limits:

• A handoff failed, so the agent needed to give the customer an honest next step.
• Medical danger and self-harm language required the approved safety response.
• Billing changes needed an identity check, and closing an account required confirmation.
• Legal deadlines and workplace harassment needed careful routing without invented advice.
• Handoff notes had to limit personal details and respect accessibility needs.
• Routine questions and ordinary frustration had to remain automated.

Routine questions prevented an easy perfect score from sending every conversation to a person. The model had to know when to stay put.

Both models cleared the round. They kept requests for a person intact and wrote careful handoff notes. Their messages also told the truth about whether a transfer had happened.

Our scorekeeper needed a vocabulary lesson

The first automated score gave Terra a win on the account-closing case. When we read Luna's saved answers, every required safeguard was there. Luna had used verbs our scoring tool had never learned.

So we taught the scorekeeper a few new words.

The audit generated no replacement answers. We widened the accepted wording equally for both and rescored the original answers under the same safety rules. Several other flagged answers had the same wording problem.

Saving the original answers gave us a clean way to repair the ruler. Automated scoring is consistent, including when it is consistently wrong.

Round two: can it keep the story straight?

The second set covered 12 longer setup conversations. Running three attempts per model produced another 72 trials.

These tests asked the models to remember earlier choices while handling a new request. One-time details had to stay out of the agent's permanent setup. Later edits had to preserve earlier rules for human help. A model also had to avoid claiming it checked a system when no check had happened.

Luna passed every case.

That result also cleared up an older concern. During an earlier test through a different setup, Luna had saved a one-time reference code as permanent information. In all three local attempts, Luna left that code out. It also kept the rule that customers could still ask for a person.

Terra finds a strangely specific wall

One case began with information from a public website. The customer later replaced it with private reference material. Terra stopped before returning an update that Agent One could save in all three attempts. Luna completed all three.

We reran the case with fewer tests happening at once, then ran it by itself. The observed record finished here:

We could not see the exact reason Terra stopped. The error did not match the common problems our test setup recognizes.

The repeated stop still counts as a reliability problem because Agent Creator received nothing to save. The evidence does not show that Terra misunderstood the privacy rule. It shows that a customer would have been left without a completed update in this setup.

The speed round

Terra answered about 30% faster in the people-and-safety tests and 35% faster in the longer conversations. Even near the slow end of the results, Terra still answered sooner.

Luna produced longer answers overall, which fits the extra time it took. Those measurements include background instructions from our local test setup, so they cannot be used to calculate customer pricing.

For Agent One, those results map to two uses:

• Terra fits focused work where speed matters and the product can recover when an update fails.
• Luna fits the default because it matched Terra on human judgment and completed the longer conversations more reliably.

The pick

Luna remains the model behind Agent One's Efficient setting. Terra stays behind Balanced for customers who value its speed.

This release choice covers 30 situations and 180 attempts in one local test setup. Coding, research, creative work, and open-ended chat remain outside its scope.

Future rounds will test longer conversations and cases where one part of a task fails. A separate set will cover unclear permission to take action. The same repeatable test will decide when the default moves.

If you are defining a customer-facing agent, start by writing down when a person should step in and what that person needs to know. Our guide to human escalation for an AI front desk gives you a practical starting point. Once the workflow has an owner and clear boundaries, build the first version in Agent One.