Signal · The review load

The more your AI gets right, the less closely your team checks it

AI lets a marketing team turn out far more work, and most of it comes out right. Every piece still needs someone to approve it, and studies of people checking automated systems find they check less closely the more often the system has been right. This page follows one example week of AI-drafted work to show what that does to the few pieces that are wrong.

This link was made on an earlier version of the page, so some of the settings in it no longer apply.

Pick the example closest to your team's work

Each is a kind of work a team signs off, shown with a typical mistake. If yours isn't here, pick the nearest.

A wrong piece looks like the right ones

One line in this piece is wrong, and it's written the same way as the lines around it. Turn on Show the mistake, under the piece, to see which one.

Told there's a mistake, you might suspect a line or two. The reviewer isn't told, and this is one of twentya hundredtwo hundred pieces that week. Finding it takes checking each line against the facts it depends on, like the brief, the price list or the rules.

Point 1. A wrong piece looks as finished as a right one. Only a line-by-line check against the facts finds the mistake.

After a run of right ones, the check gets lighter

Next comes the week this piece came from, with one square for each piece. The fill in a square is how much of that piece the reviewer checks. The first few are filled to the top, because every line of them gets checked. After a run of right ones the reviewer starts to skim, and the wrong piece arrives in the middle of that.

The example starts just after the reviewer last caught a mistake. From there the check gets lighter with every approval, down to a level that depends on how often the AI has been right, and only a caught mistake brings back a full check. It spaces the mistakes evenly, which a real week won't.

How often is the AI right? example

Choose how reliable the AI is. A more reliable AI puts more right pieces before each wrong one.

How many pieces reach your team in a week? example

Choose how busy the week is. At the same rate, more pieces mean more mistakes.

At 20 a week and 3 in 4 right, 5 are wrong (20 × 1 in 4) and 15 are right.At 20 a week and 9 in 10 right, 2 are wrong (20 × 1 in 10) and 18 are right.At 20 a week and 19 in 20 right, 1 is wrong (20 × 1 in 20) and 19 are right.At 100 a week and 3 in 4 right, 25 are wrong (100 × 1 in 4) and 75 are right.At 100 a week and 9 in 10 right, 10 are wrong (100 × 1 in 10) and 90 are right.At 100 a week and 19 in 20 right, 5 are wrong (100 × 1 in 20) and 95 are right.At 200 a week and 3 in 4 right, 50 are wrong (200 × 1 in 4) and 150 are right.At 200 a week and 9 in 10 right, 20 are wrong (200 × 1 in 10) and 180 are right.At 200 a week and 19 in 20 right, 10 are wrong (200 × 1 in 20) and 190 are right. example

The wrong pieces are made-up examples, and so are the counts.

At 3 in 4 a mistake turns up every few pieces, so the reviewer keeps checking closely and catches the ones that show in the piece. At 9 in 10 and 19 in 20 each mistake arrives after many right pieces in a row. Now and then one gets a close look and is caught, which brings the full check back for a few pieces. The rest get the same light check as their neighbours. An AI that's right more often makes fewer mistakes in all, which is the gain you want from it, but the review catches a smaller share of them than at 3 in 4, and the ones it misses ship unnoticed.

Open any of the first five squares with a dashed outline, or use Another wrong piece on the card, to see what each would do if it shipped.

Point 2. The more often the AI is right, the longer the run before each wrong piece, and the lighter the check it gets.

Why each right piece makes the next check lighter

Three findings explain the pattern. Turn on Show where each applies to mark each one on the week: A on the last right piece before the wrong one, B on the wrong line, C on the wrong piece's square.

  1. A. Reliable systems get checked less. In lab studies, people watching an automated system caught fewer of its failures the more reliable it had been, and when it stayed reliable they checked on it only now and then.1
  2. B. People see what they expect. Airline pilots described looking at a checklist item that was set wrong and seeing it as set right, because it had been right on so many checks before.2
  3. C. AI work looks finished either way. In an experiment with 373 consultants, on a task built so the AI's answer was wrong, those working without AI got it right 84.5% of the time. Those using AI got it right 70.6% of the time, and 60% when they'd also been given guidance on using it. The answers written with AI were rated higher for quality, right or wrong.3

None of these studies looked at marketing teams. They show which way the effect runs, not how big it is in marketing work.

See them on the week

Point 3. Each right piece makes the next one easier to approve, and the wrong one arrives looking like the rest.

Checking works when nothing else competes for it

In the lab studies, people whose only task was watching the system caught nearly all of its failures. When they had other work to do as well, they caught markedly fewer.1 On a team, the nearest thing is dedicated time for each check, with nothing else competing for the reviewer's attention. Switch on Dedicated time to check to redraw the week that way.

With it on, every piece is checked line by line and the mistakes that show in the piece are caught. A mistake that doesn't show, like an offer that only works in one countrya claim legal never cleareda report that counted some orders twicea code that has expired, still needs the facts in front of the reviewer.

A check like that takes time on every piece. Most of it goes to pieces that turn out right, and that time is how the wrong ones get found, because they look like the rest until someone reads them closely.

See the week

Point 4. With dedicated time, the line-by-line check comes back. A mistake the piece doesn't show still needs the facts it depends on.

What it takes to catch the wrong ones

Reaching this step redraws the week with both: a reviewer with nothing else to do in those minutes, and the facts each piece depends on at hand. Every wrong piece is caught. The lab studies tested only the first of these, and even then people caught nearly every failure, not every one. They didn't test marketing work.

What the week showed

  1. A piece with a mistake looks as finished as the others.
  2. The more reliable the AI, the longer the run before each mistake, and the lighter the check.
  3. Research on people checking automated systems points the same way, though none of it looked at marketing work.
  4. Dedicated time and the facts at hand bring the full check back.

Every right piece is work AI added to your team's week. As the AI gets more reliable, the check on them tends to get lighter, unless someone decides it shouldn't.

Not every piece needs that check, and a system can test for some of these mistakes before anyone sees the piece. Which pieces get a person's full check is a choice about how your team oversees its AI work, best made on purpose.

So for your team's segmentsadsbudget movesjourney changes, or whatever AI-drafted work you sign off, find out who checks each one, whether checking is all they're doing in those minutes, and whether the brief and the send rulesthe approved claims, prices and datesthe reports the figures came fromthe current offers, codes and send times are open beside them.

See the week
Example
Audience segmentsAd copyBudget movesJourney changes
How often right
3 in 49 in 1019 in 20
Pieces a week
20100200
Switched onDedicated time, facts at hand
In this example week, 5 of the 20 pieces have a mistakeIn this example week, 2 of the 20 pieces have a mistakeIn this example week, 1 of the 20 pieces has a mistakeIn this example week, 25 of the 100 pieces have a mistakeIn this example week, 10 of the 100 pieces have a mistakeIn this example week, 5 of the 100 pieces have a mistakeIn this example week, 50 of the 200 pieces have a mistakeIn this example week, 20 of the 200 pieces have a mistakeIn this example week, 10 of the 200 pieces have a mistakeNone of them ship1 of them ships1 of them ships3 of them ship3 of them ship2 of them ship2 of them ship2 of them ship2 of them shipNone of them ship1 of them shipsNone of them ship2 of them ship1 of them ships2 of them ship2 of them ship1 of them shipsNone of them shipIt doesn't shipIt shipsIt doesn't shipIt shipsIt shipsIt shipsIt shipsIt shipsIt doesn't shipNone of them ship5 of them ship5 of them ship15 of them ship15 of them ship10 of them ship10 of them ship10 of them ship10 of them shipNone of them ship6 of them ship2 of them ship8 of them ship6 of them ship7 of them ship4 of them ship7 of them ship4 of them shipNone of them ship4 of them ship1 of them ships4 of them ship3 of them ship4 of them ship2 of them ship4 of them ship2 of them shipNone of them ship10 of them ship10 of them ship30 of them ship30 of them ship20 of them ship20 of them ship20 of them ship20 of them shipNone of them ship12 of them ship4 of them ship16 of them ship12 of them ship14 of them ship8 of them ship14 of them ship8 of them shipNone of them ship7 of them ship2 of them ship9 of them ship6 of them ship8 of them ship4 of them ship8 of them ship4 of them ship example5 of 54 of 54 of 52 of 52 of 53 of 53 of 53 of 53 of 52 of 21 of 22 of 20 of 21 of 20 of 20 of 21 of 22 of 21 of 10 of 11 of 10 of 10 of 10 of 10 of 10 of 11 of 125 of 2520 of 2520 of 2510 of 2510 of 2515 of 2515 of 2515 of 2515 of 2510 of 104 of 108 of 102 of 104 of 103 of 106 of 103 of 106 of 105 of 51 of 54 of 51 of 52 of 51 of 53 of 51 of 53 of 550 of 5040 of 5040 of 5020 of 5020 of 5030 of 5030 of 5030 of 5030 of 5020 of 208 of 2016 of 204 of 208 of 206 of 2012 of 206 of 2012 of 2010 of 103 of 108 of 101 of 104 of 102 of 106 of 102 of 106 of 10wrong pieces caught before they ship1 needs a fact from outside the piece.1 needs a fact from outside the piece.3 need a fact from outside the piece.3 need a fact from outside the piece.2 need a fact from outside the piece.2 need a fact from outside the piece.2 need a fact from outside the piece.2 need a fact from outside the piece.Add the facts, and all are caught.Caught by an occasional close look. The rest get only a light check.Every mistake here shows in the piece.Each is checked as lightly as its neighbours.1 needs a fact from outside the piece.Each is checked as lightly as its neighbours.2 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.Every mistake here shows in the piece.Add the facts, and all are caught.Each is checked as lightly as its neighbours.Every mistake here shows in the piece.Each is checked as lightly as its neighbours.1 needs a fact from outside the piece.Each is checked as lightly as its neighbours.1 needs a fact from outside the piece.Each is checked as lightly as its neighbours.Every mistake here shows in the piece.Add the facts, and all are caught.5 need a fact from outside the piece.5 need a fact from outside the piece.15 need a fact from outside the piece.15 need a fact from outside the piece.10 need a fact from outside the piece.10 need a fact from outside the piece.10 need a fact from outside the piece.10 need a fact from outside the piece.Add the facts, and all are caught.Caught by an occasional close look. The rest get only a light check.2 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.6 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.4 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.4 need a fact from outside the piece.Add the facts, and all are caught.Caught by an occasional close look. The rest get only a light check.1 needs a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.3 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.2 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.2 need a fact from outside the piece.Add the facts, and all are caught.10 need a fact from outside the piece.10 need a fact from outside the piece.30 need a fact from outside the piece.30 need a fact from outside the piece.20 need a fact from outside the piece.20 need a fact from outside the piece.20 need a fact from outside the piece.20 need a fact from outside the piece.Add the facts, and all are caught.Caught by an occasional close look. The rest get only a light check.4 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.12 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.8 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.8 need a fact from outside the piece.Add the facts, and all are caught.Caught by an occasional close look. The rest get only a light check.2 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.6 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.4 need a fact from outside the piece.Caught by an occasional close look. The rest get only a light check.4 need a fact from outside the piece.Add the facts, and all are caught.

Dedicated time to check is on, so every piece here is checked line by line.

Each square is one piece, filled by how much of it gets checked:line by lineskimmedglanced athas a mistakethe one on the cardthe 3 right ones before itthe 9 right ones before itthe 19 right ones before itcaughtneeds a fact from outside the piece

Open a wrong piece

Spring win-back, lapsed membersSpring win-back, by emailSpring win-back, not in autumn'sSpring win-back, $20 offSpring win-back, high spendersSpring sale, variant 7 of 24Spring sale, variant 12 of 24Spring sale, variant 15 of 24Spring sale, variant 18 of 24Spring sale, variant 21 of 24Shift $40,000 into search, out of paid socialShift $25,000 into connected TV, out of displayShift $30,000 into paid social, out of searchShift $15,000 into email, out of affiliatesShift $20,000 into brand search, out of generic searchWin-back journey, branch BWin-back journey, branch CWin-back journey, branch DWin-back journey, branch EWin-back journey, branch F

If it ships. Members who ordered last week get a win-backMembers who opted out of email get oneAutumn's win-back list gets the spring one tooCanadian codes fail at checkoutThe offer reaches far more members than meantPromises a warranty nobody approvedPromises a price the store won't chargeMakes a claim legal never clearedOffers free returns the policy doesn't coverEnds the sale two weeks early$40,000 moves on an inflated search figure$25,000 moves on a test a store opening may have lifted$30,000 moves on two returns counted differently$15,000 moves on a return worked out wrong$20,000 moves on sales the ads may not have addedPast customers get a first-order offerThe reminder repeats every day, with no endFrench and German customers get the US offerEveryone in the branch gets an expired codeSome texts go out in the middle of the night

Account created more than 180 days agoEmail consent is AnyIn the autumn win-back is YesCountry is United States or CanadaLifetime spend more than $50Every frame now comes with a lifetime warranty.Frames from $49 until Sunday.Shown to reduce eye strain.Free returns on every order, worldwide.Sale ends March 31.Search figure from the March channel reportBased on a two-week test in one city, March 3 to 16Paid social credited for 28 days after a click, search for 1 dayEmail brought in $26,000 on $10,000 of spendCounts every sale after a brand-search clickSend the welcome offer, 20% off a first orderSend the reminder, then repeat dailySend the English offer to every marketSend the offer with code WINBACK20Send a text right after the wait, at any hour

Reading the brief, then checking which date the rule counts from, catches it.Reading the consent line catches it.Reading the rule word by word against the brief catches it.Catching it needs the list of countries the offer works in.Reading the brief's $500 against the rule catches it.Catching it needs the brand's warranty terms.Reading it against the price at the top catches it.Catching it needs legal's list of cleared claims.Catching it needs the returns policy.Reading it against the dates at the top catches it.Catching it needs a look at how the March report counted orders.Catching it needs a look at what else happened in that city.Reading the two credit windows catches it.Dividing out the figures on the line catches it.Reading the goal against what the figure counts catches it.Reading the journey's name catches it.Following the branch to its end catches it.Catching it needs the offer each market has.Catching it needs the list of codes that still work.Reading when the text can go out catches it.

What's wrong with it

The mistake. The rule counts six months from the day the account was opened, not from the last order, so members who ordered last week are still in it.Any takes in members who opted out of email, so the send would reach people who asked not to get it.The rule is the wrong way round. It keeps only the members the brief leaves out, so the members who had the autumn win-back get another one.The $20 offer only works at the US store, so Canadian members get a code that fails at checkout.The brief sets the line at $500, so this takes in far more members than it should, and the offer goes to people it wasn't meant for.The brand's warranty is one year. Nobody approved a lifetime promise, and the line reads like every other benefit in the set.The ad's own price line says frames start at $59. $49 was last season's price, so this line promises a price the store won't charge.The brand has no study behind an eye-strain claim, and legal never cleared one.Free returns cover the US and Canada only, so the ad promises more than the returns policy gives.The ad's own dates say the sale runs to April 14. March 31 came from last year's copy, so this line ends the sale two weeks early.That report counted an order twice when a customer clicked two search ads, so search's $3.10 is too high, and the gap the move rests on may not be there.Those were the two weeks the city's new store opened, so some of the $2.40 may be sales the opening brought in, and the test can't tell the two apart.Paid social gets credit for sales up to four weeks after a click and search for one day, so the two returns aren't counted the same way and can't be compared as they stand.$26,000 on $10,000 is $2.60, not $5.20, the same as affiliates, so the reason given for moving the money no longer holds.Counting every sale after a click takes in people already on their way to buy, so the $8.00 isn't a measure of the sales the ads added, which is what the goal asks for.The branch is for customers who have ordered before, so they get an offer for a first order they can't place.The repeat has no end, so a customer who doesn't order gets the reminder every day and never moves on to the next step.The English offer is the US one, so customers in France and Germany get it in English, priced in dollars.WINBACK20 was autumn's code and stopped working on January 1, so everyone in the branch gets a code that fails at checkout.The wait ends at whatever hour it started, so some texts go out in the middle of the night.

Reading the brief, then checking which date the rule counts from, catches it.The line shows it to anyone who knows an email send needs opted-in consent.Reading the rule word by word against the brief catches it.Catching it takes knowing where the offer can be used.Checking the rule's figure against the brief's $500 catches it.Catching it takes knowing the warranty the brand actually offers.Reading this line against the $59 at the top of the ad catches it.Catching it takes the list of claims legal has cleared.Catching it takes the returns policy.Reading this line against the dates at the top of the ad catches it.Catching it takes knowing how the March report counted orders.Catching it takes knowing what else happened in that city those two weeks.The line shows it to anyone who knows the two windows have to match.Dividing the line's own figures and comparing the result with the $5.20 above catches it.Reading the goal against what the $8.00 counts catches it.The journey's name gives it away, since a win-back goes to people who have ordered before.The branch shows it, followed step by step to the end.Catching it takes knowing which offer each market has.Catching it takes knowing which codes still work.The line itself shows it, read for when the text can go out.

With a mistake every few pieces, the reviewer still checks this segment rule by rule and stops on the wrong one.After 9 right ones in a row, the reviewer checks the segment's name, brief and size, and only skims the wrong rule.After 19 right ones in a row, its name and size are enough to approve it, and nobody checks the rules.With a mistake every few pieces, the reviewer still reads every line of this ad, the wrong one included.After 9 right ones in a row, the reviewer checks the ad's headline and format, and only skims the body copy.After 19 right ones in a row, its headline is enough to approve it, and nobody checks the body copy.With a mistake every few pieces, the reviewer still reads this move's goal, its figures and the line they come from.After 9 right ones in a row, the reviewer checks the move's amount, goal and figures, and only skims the line they come from.After 19 right ones in a row, its amount and figures are enough to approve it, and nobody checks where they come from.With a mistake every few pieces, the reviewer still follows this branch step by step and stops on the send step.After 9 right ones in a row, the reviewer checks the branch's condition and wait, and only skims the send step.After 19 right ones in a row, the branch's shape is enough to approve it, and nobody checks the send step.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.This one happened to get a close look, so the reviewer caught it, and the next few pieces get a full check again.With dedicated time, the reviewer reads every line of this piece, and of every other piece this week.At 3 in 4 every piece already gets a line-by-line check, so the switch changes nothing here.With the facts in front of them and dedicated time, the reviewer reads every line of every piece. In this example all 5 wrong pieces are caught, and the other 15 go out checked.In this example both wrong pieces are caught, and the other 18 go out checked.In this example the 1 wrong piece is caught, and the other 19 go out checked.In this example all 25 wrong pieces are caught, and the other 75 go out checked.In this example all 10 wrong pieces are caught, and the other 90 go out checked.In this example all 5 wrong pieces are caught, and the other 95 go out checked.In this example all 50 wrong pieces are caught, and the other 150 go out checked.In this example all 20 wrong pieces are caught, and the other 180 go out checked.In this example all 10 wrong pieces are caught, and the other 190 go out checked.

The fills are drawn to show the direction the research found, not measured on marketing work.

An audience segment drafted by AI, waiting for someone to approve itAn ad drafted by AI, one of 24 variants waiting for someone to approve themA budget move drafted by AI, waiting for someone to approve itA change to a customer journey drafted by AI, waiting for someone to approve it

An example segment
Spring win-back, lapsed membersSpring win-back, by emailSpring win-back, not in autumn'sSpring win-back, $20 offSpring win-back, high spenders
For members who haven't ordered in six monthsAn email to members who haven't ordered in six monthsFor members who haven't ordered in six months and weren't in the autumn win-backFor members who haven't ordered in six monthsFor members who haven't ordered in six months and have spent more than $500
Loyalty member is Yes
Email consent is Opted inLast order more than 180 days agoLast order more than 180 days agoLast order more than 180 days agoLast order more than 180 days ago
Account created more than 180 days agoEmail consent is AnyIn the autumn win-back is YesCountry is United States or CanadaLifetime spend more than $50
Has an email address
48,210 people
Approve segment
An example ad variant
Spring sale, variant 7 of 24Spring sale, variant 12 of 24Spring sale, variant 15 of 24Spring sale, variant 18 of 24Spring sale, variant 21 of 24
Feed, square. Spring sale, March 17 to April 14. Frames from $59.
Lighter frames, built for every day
Every frame now comes with a lifetime warranty.Frames from $49 until Sunday.Shown to reduce eye strain.Free returns on every order, worldwide.Sale ends March 31.
Shop the sale
Approve variant
An example budget move
Shift $40,000 into search, out of paid socialShift $25,000 into connected TV, out of displayShift $30,000 into paid social, out of searchShift $15,000 into email, out of affiliatesShift $20,000 into brand search, out of generic search
The goal is to spend where the ads add the most sales
Search returned $3.10 for every dollar in March. Paid social returned $1.80.Connected TV returned $2.40 for every dollar in its test. Display returned $1.30.Paid social returned $2.90 for every dollar in April. Search returned $2.10.Email returned $5.20 for every dollar last quarter. Affiliates returned $2.60.Brand search returned $8.00 for every dollar in March. Generic search returned $2.20.
Search figure from the March channel reportBased on a two-week test in one city, March 3 to 16Paid social credited for 28 days after a click, search for 1 dayEmail brought in $26,000 on $10,000 of spendCounts every sale after a brand-search click
Starts Monday, runs four weeks
Approve move
An example journey change
Win-back journey, branch BWin-back journey, branch CWin-back journey, branch DWin-back journey, branch EWin-back journey, branch F
If there's no order in 90 days
Wait 2 days
Send the welcome offer, 20% off a first orderSend the reminder, then repeat dailySend the English offer to every marketSend the offer with code WINBACK20Send a text right after the wait, at any hour
Then rejoin the main journey
Approve change

The mistake. The rule counts six months from the day the account was opened, not from the last order, so members who ordered last week are still in it.Any takes in members who opted out of email, so the send would reach people who asked not to get it.The rule is the wrong way round. It keeps only the members the brief leaves out, so the members who had the autumn win-back get another one.The $20 offer only works at the US store, so Canadian members get a code that fails at checkout.The brief sets the line at $500, so this takes in far more members than it should, and the offer goes to people it wasn't meant for.The brand's warranty is one year. Nobody approved a lifetime promise, and the line reads like every other benefit in the set.The ad's own price line says frames start at $59. $49 was last season's price, so this line promises a price the store won't charge.The brand has no study behind an eye-strain claim, and legal never cleared one.Free returns cover the US and Canada only, so the ad promises more than the returns policy gives.The ad's own dates say the sale runs to April 14. March 31 came from last year's copy, so this line ends the sale two weeks early.That report counted an order twice when a customer clicked two search ads, so search's $3.10 is too high, and the gap the move rests on may not be there.Those were the two weeks the city's new store opened, so some of the $2.40 may be sales the opening brought in, and the test can't tell the two apart.Paid social gets credit for sales up to four weeks after a click and search for one day, so the two returns aren't counted the same way and can't be compared as they stand.$26,000 on $10,000 is $2.60, not $5.20, the same as affiliates, so the reason given for moving the money no longer holds.Counting every sale after a click takes in people already on their way to buy, so the $8.00 isn't a measure of the sales the ads added, which is what the goal asks for.The branch is for customers who have ordered before, so they get an offer for a first order they can't place.The repeat has no end, so a customer who doesn't order gets the reminder every day and never moves on to the next step.The English offer is the US one, so customers in France and Germany get it in English, priced in dollars.WINBACK20 was autumn's code and stopped working on January 1, so everyone in the branch gets a code that fails at checkout.The wait ends at whatever hour it started, so some texts go out in the middle of the night.

If it ships. Members who ordered last week get a win-backMembers who opted out of email get oneAutumn's win-back list gets the spring one tooCanadian codes fail at checkoutThe offer reaches far more members than meantPromises a warranty nobody approvedPromises a price the store won't chargeMakes a claim legal never clearedOffers free returns the policy doesn't coverEnds the sale two weeks early$40,000 moves on an inflated search figure$25,000 moves on a test a store opening may have lifted$30,000 moves on two returns counted differently$15,000 moves on a return worked out wrong$20,000 moves on sales the ads may not have addedPast customers get a first-order offerThe reminder repeats every day, with no endFrench and German customers get the US offerEveryone in the branch gets an expired codeSome texts go out in the middle of the night.

Reading the brief, then checking which date the rule counts from, catches it.The line shows it to anyone who knows an email send needs opted-in consent.Reading the rule word by word against the brief catches it.Catching it takes knowing where the offer can be used.Checking the rule's figure against the brief's $500 catches it.Catching it takes knowing the warranty the brand actually offers.Reading this line against the $59 at the top of the ad catches it.Catching it takes the list of claims legal has cleared.Catching it takes the returns policy.Reading this line against the dates at the top of the ad catches it.Catching it takes knowing how the March report counted orders.Catching it takes knowing what else happened in that city those two weeks.The line shows it to anyone who knows the two windows have to match.Dividing the line's own figures and comparing the result with the $5.20 above catches it.Reading the goal against what the $8.00 counts catches it.The journey's name gives it away, since a win-back goes to people who have ordered before.The branch shows it, followed step by step to the end.Catching it takes knowing which offer each market has.Catching it takes knowing which codes still work.The line itself shows it, read for when the text can go out.

Where the judgment sits
What's holding your operating model backA marketing operating model in six rows. The last, the human-agent boundary, is where this page's question lives: what an agent may do on its own, and what a person checks first.
References
  1. 1Parasuraman, R. and Riley, V. "Humans and automation: use, misuse, disuse, abuse." Human Factors 39(2), 1997, pp. 240 to 241. Lab studies of people monitoring automated systems. ↩
  2. 2Degani, A. and Wiener, E. L. Human Factors of Flight-Deck Checklists: The Normal Checklist. NASA Contractor Report 177549, 1990, pp. 38 to 39. Interviews with airline pilots and incident reports. ↩
  3. 3Dell'Acqua, F. and others. "Navigating the jagged technological frontier." Harvard Business School Working Paper 24-013, 2023, pp. 14 to 15 and Figure 7, p. 33. 373 BCG consultants on a task chosen to sit outside the AI's capability. ↩