New arrivals: 14 Scout case studies

Internal exploration · Merchant agent

Linking ad spend to a store's sales without relying on the platforms alone

An ad campaign can multiply clicks without bringing in buyers, and a platform can show good results that cannot be linked to the store's orders. For the merchant agent we are building, we followed by hand the path from spend to sale: what the campaigns aim for, the visits the store actually received, the journey on the site, then the orders. Above all, this exercise helped us define what the agent can propose, and what it must leave to the merchant and their team.

AIAdvertisingE-commerce

The online store of a brand we work with invests in Meta and Google Ads every day. In the run-up to Black Friday, the team in charge of the campaigns had changed several Meta campaigns during the week. On a November day when sales stayed at about 37% of the daily target, two advertising facts stood out: a Meta campaign that brought half of the Meta clicks, and a Google acquisition campaign paused during the day. We needed to understand what they meant for sales, without taking the platforms' reports for the reality of the store.

Look at what each campaign is trying to achieve

An ad platform delivers ads according to the objective it is given. A campaign optimised for purchases looks for people likely to buy; a campaign optimised for page views looks for people who will load the page, which is far less demanding. Comparing their clicks without accounting for this difference means comparing two different things.

That day, about 83% of Meta spend went to campaigns optimised for purchases. The campaign optimised for page views, launched three days earlier, accounted for only about 17% of spend, but half of the clicks. Meta's overall click-through rate was clearly higher than over the previous seven days. A single ad, out of some fifteen running, produced about a third of the clicks on its own. Reading this rise as renewed interest in the store would have been risky: it went hand in hand with the weight taken by this cheap campaign, and we could not isolate the cause.

Follow the visits after the click

The next step is to find these clicks on the site. We did this with Signal, the measurement platform we are developing, keeping only visits that were probably human. Bots and our technical tools are excluded. Collection had been interrupted overnight and for part of the morning: only visits from the rest of the day could be observed, and we flagged this.

Over that period, the campaign optimised for page views brought 117 sessions and no observed add-to-cart. Visits from Google produced 8 for 138 sessions, those from emails 6 for 63, and the Meta retargeting campaign, aimed at recent visitors, 3. More than 90% of the page-view campaign's sessions stopped after one page. Since its launch, Meta had not attributed a single purchase to it either.

This finding is measured, but it has clear limits. It covers a partially observed day and small numbers. It does not say whether this campaign's visitors will buy later. The add-to-carts reported by Meta also cannot be compared directly with the sessions observed on the site, because they do not follow the same counting rules.

What the sales results say, and what they do not

Meta credited itself with no purchases that day, compared with a little over two a day on average over the previous week. We did not present this as an alert. Meta's attribution can be revised for seven days, and a single day with small numbers is not enough to conclude. The figure had to be read again a week later, with the date of each reading.

On the Google side, the account history dated the manual pause of the acquisition campaign to around midday, from the web interface. Its clicks fell to zero in the afternoon, when it had received 170 the day before over the same hours. The account's clicks were down about 40% on the average of the previous seven days. The conversions reported by Google, however, did not fall. The reason for the pause was not established. We neither asked for nor recommended reactivating the campaign: the first step was to ask the team whether the pause was intentional.

Two limits weigh on any reading of profitability. Conversions from Meta, Google and the email tool do not add up, because the same order can be claimed by several tools. And the products' gross margin was not available: without it, it is impossible to say how far ad spend remains profitable.

The trade-offs to examine

Our investigation does not conclude that a budget should be cut. It puts forward a proposal that the team can accept, change or reject. For the Meta campaign, we proposed a comparative test limited to seven days: reduce the share of the campaign optimised for page views, without changing the total budget or the ads, then compare the store's carts and orders rather than the return shown by Meta. This test guarantees no gain. It answers a precise question: do these clicks bring in buyers?

We also defined in advance what would change our minds. If, over the week, visits from this campaign produce carts at a rate comparable to the campaigns optimised for purchases, the hypothesis of poorly qualified traffic loses its force. If their add-to-cart rate stays below half that of the purchase campaigns, the signal deserves to be raised with the team. Either way, the decision is theirs.

Finally, we set aside several reflexes. Cutting brand campaigns because they seem to capture demand created elsewhere would be premature: their real contribution has not been measured, and it would take a dedicated test to do so. Sharply reducing advertising just before the Black Friday peak would also be risky: the following week does not look like the days we observed.

What the agent can do, and what it will not do

This work was carried out semi-manually: a language model did the reading and the analysis under our direction, with read-only access. It defines the role we give the merchant agent in advertising. The agent will have to read each campaign's objective before its clicks, follow visits through to the cart and the order, date every reading of the platforms, separate what is measured from what is assumed, and put forward bounded tests along with their refutation criterion.

The agent will not change a campaign, a budget or an ad. Its access to the ad accounts is planned as read-only, and its proposals will go to the merchant and to the team that runs the campaigns. We claim no gain: the proposed test has not been run yet. What we want to make possible is a discussion about spend that rests on the store's orders, and not only on what each platform displays.

All case studies →

Something to dig into? Add this article to your office hours discussion.

See my discussion