How the machine moves
10 animated diagrams. Each plays by itself when you scroll to it; hover to pause, or step through with the arrows.
The 100-millisecond auctionWaterfall vs header biddingWhere the advertiser’s dollar goesHow invalid traffic gets caughtads.txt, sellers.json & schainHow a CTV ad reaches your TVFrom ad to sale: attributionIdentity: cookies, IDs and clean roomsAnatomy of a bot fraud schemeThe retail media loop
The 100-millisecond auction
What happens between you opening a web page and an ad appearing: a real-time auction that finishes faster than a blink.
The 100-millisecond auction
1/8
You open a page
Your browser starts loading an article. The page has a space for an ad, but nobody has decided which ad yet.
- You open a page: Your browser starts loading an article. The page has a space for an ad, but nobody has decided which ad yet.
- The slot goes up for sale: Code on the page (often Prebid) calls the publisher’s SSP: “I have a 300×250 slot, on this article, seen by this device, in Mumbai.” That message is a bid request.
- Is anyone real here?: Before the slot is offered, traffic checks look for signs of invalid traffic: known data-center IPs, bot user agents, spoofed domains. Suspicious requests are dropped here, pre-bid.
- Bid requests fan out: The SSP sends the bid request to many DSPs at once, each buying for different advertisers. Every DSP now has a few dozen milliseconds to decide.
- Bids come back: Each DSP checks its campaigns and data: is this someone our advertiser wants? What is this impression worth? It answers with a price, per thousand impressions (CPM).
- Highest bid wins: In today’s first-price auction the highest bid wins and pays what it bid. DSP A wins at $4.20 CPM, which is $0.0042 for this single impression.
- The ad is delivered: The winner’s ad server sends the creative (the image or video) to the page. Tracking pixels fire so the impression can be counted, billed and measured.
- You see an ad: All of that happened before the page finished loading. It repeats billions of times a day, which is why fraudsters target it: fake “visitors” can collect real bids.
Waterfall vs header bidding
Why publishers moved from asking buyers one at a time to asking them all at once, and how it raised their revenue.
Waterfall vs header bidding
1/6
The old way: a waterfall
In the waterfall, the publisher offers the slot to one partner at a time, in a fixed order, each with a floor price.
- The old way: a waterfall: In the Waterfall (daisy-chaining), the publisher offers the slot to one partner at a time, in a fixed order, each with a floor price.
- Pass… pass…: Network 1 has nothing worth $3 for this visitor, so it passes. The slot trickles down to Network 2, then Network 3. Every hop adds delay.
- Money left on the table: Network 3 takes it for $1.20. But Network 2 had a buyer willing to pay $2.80. It was never asked the right way. The order, not the price, decided who won.
- The new way: ask everyone at once: Header bidding puts a wrapper like Prebid in the page. It sends the request to every partner simultaneously.
- Real prices come back: Everyone answers with a real price within a timeout (typically 1–1.5 seconds on the web): $2.10, $2.80, $1.20.
- The ad server decides on price: The best header bid ($2.80) is passed to the publisher ad server, which compares it with direct deals and other demand. The highest price wins. Publishers typically earned noticeably more after switching.
Where the advertiser’s dollar goes
Follow one dollar of programmatic spend from a brand to a publisher, and see who takes a cut along the way.
Where the advertiser’s dollar goes
1/7
A brand spends a dollar
A brand sets aside $1.00 for programmatic display. It hopes that dollar buys attention from real people on quality sites.
- A brand spends a dollar: A brand sets aside $1.00 for programmatic display. It hopes that dollar buys attention from real people on quality sites.
- Agency and platform fees: The agency and the DSP each take a share for planning, buying and running the technology. The DSP’s cut is its take rate.
- Data and verification: Audience data, Ad verification and brand-safety tools charge per thousand impressions. These can be small individually and add up together.
- The sell side takes its cut: The SSP or exchange keeps a fee before paying the publisher. Resellers in the chain can add more hops.
- The mystery slice: In the ISBA/PwC 2020 study, about 15% of spend could not be traced to any party: the unknown delta. Later studies found better matching data shrinks it a lot.
- What reaches the publisher: In that study, publishers received about half of the advertiser’s spend. The rest is the ad tech tax: the price of automation, data and intermediaries.
- And was it even a person?: Even the part that reaches a site only works if a human sees the ad. MFA sites and invalid traffic can soak up spend without reaching anyone real.
How invalid traffic gets caught
The layers of defence between a bot and an advertiser’s budget: lists, signals, behaviour and after-the-fact audits.
How invalid traffic gets caught
1/7
A visit arrives
Every impression starts as a visit from a device. Some are people; some are crawlers, scripts or hijacked phones pretending to be people.
- A visit arrives: Every impression starts as a visit from a device. Some are people; some are crawlers, scripts or hijacked phones pretending to be people.
- Layer 1: known lists: The easy catches: declared crawlers on the IAB/ABC Spiders & Bots List, known data-center IP ranges, non-browser user agents. The MRC calls this GIVT.
- Layer 2: device and network signals: Next, deeper checks: does the device claim to be an iPhone but behave like a Linux server? Is it a headless browser or a residential proxy? Inconsistencies are red flags.
- Layer 3: behaviour: Bots can fake a fingerprint, but it is hard to fake human behaviour at scale: timing, scrolling, click-to-install gaps, impossible volumes. Behavioural analysis catches SIVT.
- One score to act on: Signals are combined into a score buyers can act on in real time. ClearTrust’s TQI Score™, for example, rolls 150+ filters into a single number.
- Pre-bid: don’t buy it: With pre-bid filtering, bad traffic is never bought. Clean requests go through to the auction.
- Post-bid: prove it and claw it back: Post-bid measurement audits what was bought. Invalid impressions are reported, excluded from billing and, where contracts allow, refunded.
ads.txt, sellers.json & schain
How a buyer checks that the seller really is allowed to sell a site’s ads, and why it defeats domain spoofing.
ads.txt, sellers.json & schain
1/6
The publisher declares its sellers
news.example posts a public file, ads.txt, listing every company allowed to sell its ads and the account ID it uses, marked DIRECT or RESELLER.
- The publisher declares its sellers: news.example posts a public file, ads.txt, listing every company allowed to sell its ads and the account ID it uses, marked DIRECT or RESELLER.
- A bid request arrives: The DSP receives a request “for news.example”, sent by SSP account 1234, with a SupplyChain object listing every hop it passed through.
- Check 1: is this seller authorised?: The buyer checks news.example/ads.txt: is SSP account 1234 listed? If not, the request is unauthorised and is not bought.
- Check 2: who is account 1234?: The SSP’s sellers.json names the business behind 1234 and whether it is the publisher itself or an intermediary. Hidden sellers are a warning sign.
- A spoofer tries the same trick: A fraudster sends requests claiming to be news.example from its own junk site. This is domain spoofing. Its seller ID is not in news.example’s ads.txt.
- Only verified paths get bought: Requests whose seller, ads.txt entry and schain all line up get bought; the spoofed ones fail. Simple public files closed one of the biggest fraud loopholes of the 2010s.
How a CTV ad reaches your TV
Server-side ad insertion stitches ads into streams. The same trick, faked, is one of the costliest frauds in streaming.
How a CTV ad reaches your TV
1/6
An ad break is coming
You are watching a show in a streaming app. An ad pod (a break of several ads) is due.
- An ad break is coming: You are watching a show in a streaming app. An ad pod (a break of several ads) is due.
- The server asks for ads on the TV’s behalf: With SSAI, a server, not the TV, requests the ads. It passes along the TV’s IP address and device details in the VAST request.
- The pod is sold: Buyers bid for the slots. CTV prices are high because the screen is big and the viewer is often a whole household.
- Stitched into the stream: The SSAI server stitches the ads into the video so they play smoothly, like broadcast TV. Great for viewers, but the buyer never talks to the TV directly.
- The fraud: SSAI spoofing: Because requests come from servers, a fraudster can run a server that pretends to be SSAI for millions of TVs that are not watching anything. This is SSAI spoofing. Schemes like this have faked huge volumes of requests.
- The defence: Buyers verify that SSAI servers are known and certified, check that device signals are consistent, and use standards and signed identifiers to prove a real device requested the ad.
From ad to sale: attribution
Which ad “caused” a purchase? Last-click, multi-touch and incrementality give very different answers.
From ad to sale: attribution
1/6
Three ads, one person
A shopper sees a video ad, then a social ad, then clicks a search ad and buys.
- Three ads, one person: A shopper sees a video ad, then a social ad, then clicks a search ad and buys.
- The sale: They place an $80 order. Now every ad platform wants the credit.
- Last-click says: search did it: Last-click attribution gives 100% of the credit to the final click. Simple, but it ignores everything that built the desire to buy.
- Multi-touch spreads the credit: Multi-touch attribution shares credit across the video, social and search ads using rules or models. Fairer, but it needs to track people across sites, which privacy changes have made harder.
- Incrementality asks the real question: Would they have bought anyway? A holdout group that sees no ads shows the baseline. The difference is the incremental effect, the sales the ads actually caused.
- Why fraud loves attribution: If credit goes to whoever touched the sale last, fraudsters can fake that touch. Click injection and cookie stuffing steal credit for sales that would have happened anyway.
Identity: cookies, IDs and clean rooms
How advertisers recognise the same person across sites, and what replaces third-party cookies when they are not available.
Identity: cookies, IDs and clean rooms
1/6
You visit two sites
You browse a shoe store, then read the news. Each site, and the ad tech on it, gives your browser its own ID in a cookie.
- You visit two sites: You browse a shoe store, then read the news. Each site, and the ad tech on it, gives your browser its own ID in a cookie.
- Different IDs for the same person: Cookie A and cookie B do not know they are the same browser. On their own, the shoe ad cannot follow you to the news site.
- Cookie syncing joins them: Third-party cookies and cookie syncing let ad platforms swap IDs in the background, so A and B are matched. That is how retargeting works across the web.
- Where cookies fail: Safari and Firefox restrict third-party cookies, and apps never had them. Chrome still allows them (Google dropped its plan to remove them), but many people still cannot be matched this way.
- Logged-in IDs: When you log in, your email can be turned into a one-way hashed ID such as UID2. It is stable across sites that use it, and you can opt out.
- Clean rooms: match without handing over data: In a data clean room, a retailer and a brand match their customer lists in a controlled space and only see aggregated results, never each other’s raw data.
Anatomy of a bot fraud scheme
How organised fraud rings turn fake websites and hijacked devices into real advertising money, and where they get caught.
Anatomy of a bot fraud scheme
1/7
The goal: real money for fake views
An ad fraud ring earns money every time an ad is “shown”. It doesn’t need people, just traffic that looks like people.
- The goal: real money for fake views: An ad fraud ring earns money every time an ad is “shown”. It doesn’t need people, just traffic that looks like people.
- Build the audience: It runs a Botnet: malware-infected phones and PCs, or rented servers running automated browsers that load pages and scroll like humans.
- Hide where it comes from: Data-center traffic is easy to spot, so traffic is routed through residential proxies: home internet connections, often from people who don’t know their device is involved.
- Send it to “premium” pages: The bots visit sites that are fake, or that spoof famous publishers’ names in bid requests so the inventory looks valuable.
- Launder it through the supply chain: The impressions are sold through resellers and exchanges, sometimes mixed with real traffic (sourced traffic), until they look ordinary.
- Cash out: Advertisers pay for impressions nobody saw. The money flows back up the chain to the ring. At its peak, the Methbot operation was estimated to earn millions of dollars a day.
- Where it breaks: Scale is the fraudster’s weakness. Identical behaviour, impossible device mixes, ads.txt mismatches and IP patterns expose SIVT. Detection firms, platforms and law enforcement have taken down many such rings.
The retail media loop
Why retailers became ad companies: they know what people actually buy, and can prove an ad led to a sale.
The retail media loop
1/5
Retailers know what you buy
Every loyalty card swipe and online order is first-party data: what was bought, when, and by whom.
- Retailers know what you buy: Every loyalty card swipe and online order is first-party data: what was bought, when, and by whom.
- Brands pay to reach those shoppers: A cereal brand pays the retailer to advertise to people who buy breakfast foods. The retailer runs a retail media network.
- Ads on the store itself: Sponsored products appear right where people shop, at the moment of decision.
- And off the store: With off-site retail media, the retailer’s audiences are used to target ads on other websites, apps and streaming TV.
- Closing the loop: Because the retailer sees both the ad and the purchase, it can report sales driven by the campaign: closed-loop measurement. That proof is why budgets moved here fast.