Under the hood of Real-Time Bidding

Originally written for boom, the latest in performance display advertising.

Within the display advertising environment, Real-Time Bidding has evolved from curiosity to mainstay in the last 18 months.  Though real-time bidding can appear to be a pretty simple value proposition – advertisers bid, win and display on an impression-by-impression basis –don’t make the mistake of thinking it’s unsophisticated.   The technology behind RTB is extraordinary.   At Criteo, we’ve built out our real-time capabilities from scratch in the last two years, to handle 7 billion daily RTB requests on 16 exchanges covering more than 30 countries.

Big challenge, big solution.  Let’s take a look under the hood, shall we?  

In order to participate on a RTB exchange, media buyers first need to implement proper bidding protocols. Apart from the nascent OpenRTB initiative, there is not yet an industry-wide standard, so development work is required in each case.  Getting everything set is usually quite straightforward, especially with the right software architecture (and the right team … another story in itself).  However, since the ecosystem is made up of many companies figuring this out on the fly – no names – things can occasionally get tricky and surprises can happen.

Now, what about the bidding process?  In a trivial case, where an advertiser might simply enter a bid ten cents across the board, the process is not a big deal.  But in the non-trivial case, a bid should be the end result of the system applying sophisticated algorithms to a combination of data sent in the request as well as the media buyer’s own data.  This data can include things like:

• User properties: user id (cookie matching required), user data stored by your platform

• Zone properties: size, positioning e.g. above or below the fold

• Website properties: domain, URL, site content.

• Business rules: block lists, ad verification for brand safety.

For the winning bidder, it’s then time to deliver a banner, which could again range from a simple, static banner to a unique, fully personalized ad incorporating multiple products, offers, layouts, etc. 

Up to now, all of this is actually the easy part!  And here’s the real catch: once a request has been sent by the exchange, the bid must be received within a very short amount of time - generally under 100 milliseconds. Since typical one-way coast-to-coast latency in the US being 75 milliseconds, infrastructure becomes the alpha and omega of RTB. Get it wrong and your most clever algorithm won’t even get a chance to bid! 

The next challenge is then to introduce as much algorithmic intelligence as possible in whatever precious time is left. This leads to another problem: namely, storing and retrieving data quickly enough within the bidding process.  Then there are the issues of logging and crunching huge amounts of data, and detecting and fixing technical issues.   Quickly.  Not to mention the unannounced specification changes breaking the protocol, but I said “no names.”   Oh, and did I mention 24/7/365 availability?  The list goes on and on. 

RTB is far from mundane, and it represents a tremendous technical challenge.  But it’s also the future of media buying, and so we keep working at it.