Inside Google’s plan to make the whole web as fast as AMP

In a blog post today, Google is announcing that it’s formally embarking on a project to convince the group in charge of web standards to adopt technology inspired by its Accelerated Mobile Pages (AMP) framework. In theory, it would mean that virtually any webpage could gain the same benefits as AMP: near-instantaneous loading, distribution on multiple platforms, and (critically) more prominent placement on Google properties.
 
This sounds impenetrably dense and boring, but please don’t click away yet! This is important, a little tricky to understand, and critical to how the web and Google interact in the future. In many ways, Google’s success or failure in this endeavor will play a major role in shaping how the web works on your phone.
 
If you’re unfamiliar, AMP is Google’s attempt to make webpages as fast and portable as other “instant articles” (like what you might read on Facebook or Apple News). The idea is that when you click a link on those other platforms, you don’t have to wait for the article to load because it’s already preloaded in an app. AMP’s goal is to bring the same performance to the web itself.
 
By creating AMP, Google blithely walked right into the center of a thicket comprised of developers concerned about the future of the web. Publishers are worried about ceding too much control of their distribution to gigantic tech companies, and all of the above are worried that Google is not so much a steward of the web but rather its nefarious puppet master.
 
All that angst has metastasized in the past few months, with a widely circulated open letter to Google asking it to fix AMP, more Medium blog posts than can be read in a week, Twitter screeds, and arguments in the comments of AMP’s own GitHub code repository. And that’s only the stuff coming from web developers. (I keep a folder of bookmarks I call “AMPhole” to try to keep up, and that hole gets deeper nearly every day.)
 
The whole situation is slightly frustrating to David Besbris, VP of search engineering at Google. Earlier this week, I went to Mountain View to talk with Besbris and Malte Ubl, engineering lead for AMP. “This is honestly a fairly altruistic project from our perspective,” says Besbris.
 
”It wasn’t like we invented AMP because we wanted to control everything, like people assume,” he says. Instead, he argues, go back and look at how dire the state of the mobile web was a few years ago, before AMP’s inception. It sucked — in fact, Nilay Patel published a story on this very website titled “The mobile web sucks” in 2015. He was right. Apple and Facebook dealt with that problem by creating proprietary formats and then convinced publishers to distribute their news in those formats on their platforms. As Nilay wrote:
 
Taken together, Apple News and Facebook Instant Articles are the saddest refutation of the open web revolution possible: they are incompatible proprietary publishing systems entirely under the control of huge corporations, neither of which particularly understands publishing or media.
 
Besbris saw things the same way: “The trend in the industry at the time was the simple way of solving these problems, where you guaranteed that you could control the experience … but that comes at the cost of the web.” So Google’s solution was AMP, a framework that was designed to make the web as good as those platforms so that the web would actually have a shot at competing with Apple and Facebook.
 
Making webpages instant and portable required a very clever — and very kludgey — hack.
 
The hack involved a combination of existing web technologies (like iFrames), stringent standards on webpages themselves (so they’d be guaranteed to load fast), and — critically — a different kind of infrastructure for how webpages get from a publisher’s server to your phone. These kludges are often the things that you see people complain about when they complain about AMP. iFrames have weird scrolling behavior, URLs don’t match, and AMP results often look anemic when compared to full webpages. (Fixes for all of those issues either exist or have been proposed.)
 
Despite all those problems, here’s what is impressive about AMP: when you publish a webpage, it can be served from any caching server. But that’s not what really makes it fast; what truly makes a difference is that it can load nearly instantly because it’s already been preloaded in the background. And yet, despite that preloading, you don’t count as a visitor and the publisher doesn’t get to set any cookies or do any tracking until you click. And you can trust that the cached, instantly loaded page that’s sitting inside Google search or Twitter is faithful to its canonical source, even if that source was updated after it was first published.
 
”Back at the beginning, it was pretty much assumed that the web couldn’t do any of this,” Besbris says. Ubl and his team came up with that combination of technologies that made it possible, but it required technology that isn’t really built into the web right now. So Google faced a choice: take the time to try to convince the web standards body to adopt it and browser makers to support it or just go ahead and put it out in the world as a mostly Google-backed project on Google’s own products, primarily search.
 
”We need a vehicle to actually figure it out. You can’t just know it without trying stuff,” Besbris argues. Google had to prove that the web could be as good as Instant Articles. More importantly, it had to get as good quickly — before people abandoned it for a million different custom apps and article formats. Besbris says Google couldn’t wait for the committees that help craft web standards to get it done. “If you start by trying everything through the standards process, we would still be talking about it,” he argues.
 
Whether or not AMP counts as “the web” is actually one of the points of contention — though as you’d expect, Besbris and Ubl believe strongly that it is, and they make a compelling case for it. Accelerated Mobile Pages, they contend, do not need to use Google’s servers or serve Google ads; they can be published and distributed completely independently of Google.
 
Regardless of what Google’s engineers think, outside of Mountain View, AMP is associated much more strongly with Google than it is with the web — even though it’s been adopted by Bing, Twitter, Baidu, and others. Part of the problem is that AMP was Google’s reaction to Facebook and Apple, so it fell into the easy thought bucket of “proprietary article formats.” But most of it is that Google is huge and has pushed AMP in a big way with its biggest product: search. Publishers that support AMP show up in Google’s top stories carousel, which means a flood of traffic. It’s a huge incentive to support AMP.
 
And so, finally, we are ready to talk about today’s blog post by Ubl. You need all that backstory to understand the following, a seemingly simple sentence that explains what Google is doing:
 
Based on what we learned from AMP, we now feel ready to take the next step and work to support more fast-loading content not based on AMP technology in areas of Google Search designed for this like the Top Stories carousel.
 
What Google is proposing is not to turn the entire web into AMP, but rather to take some of the ideas behind the clever hacks that made AMP work, clean them up, and then make them a universal standard that has nothing to do with Google. That way, nearly any webpage could be distributed as easily and loaded as quickly as ones that are supported by AMP.
 
Google is not blind: it knows that other companies aren’t likely to up and adopt AMP as a universal solution to fixing the web. Although Ubl will happily tout how many non-Googlers contribute to AMP code, at the end of the day, he works at Google and is the BDFL (Benevolent Dictator For Life) on the AMP project. The new standards — many of which are already in development — seem like they would be genuinely good for the web. But just as importantly, they’re genuinely more likely to get adopted by competing companies if they’re seen as fundamental new web technologies and not just a Google project.
 
”Our intention has always been to take AMP — get the good bits and the lessons that we’ve learned from building AMP — into the standards track,” says Besbris.
 
This new format doesn’t have a name yet — Google is obviously reticent to propose one — but whatever it is, it’s likely to encompass a mélange of proposed technologies. Ubl got deep into the weeds of some of these with me, but I’ll spare you and instead just point to the links in his blog post:
 
Google’s goal is to extend support in features like the Top Stories carousel to AMP-like content that (1) meets a set of performance and user experience criteria and (2) implements a set of new web standards. Some of the proposed standards in the critical path are Feature Policy, Web Packaging, iframe promotion, Performance Timeline, and Paint Timing.
 
The next steps are going to probably involve a process that, at my most optimistic, I would guess will take months — more likely it will be years. Various standards bodies have to hash out proposals, try them out, and agree to them. And it’s not just the W3C body that standardizes the web, either. The Web Packaging technology that allows webpages to work offline and be redistributed may actually need to be agreed upon by another body.
 
And after all that, the companies that make web browsers and apps need to implement it all. When I dared Ubl to guess at what the timeline for all this might be, he described what he expected the process might look like for quite awhile before admitting, “in that respect, no, no timeline.”
 
Meanwhile, Google is absolutely going to continue to develop AMP and promote its use. But to speed along the standard process, it’s offering the same carrot that originally supercharged AMP’s growth: prime placement on Google’s own properties. Google is promising that any webpage that matches the performance of an AMP page will get the exact same treatment in Google search.
 
Google isn’t able to provide a “punchlist” of features it is going to require before a page can get the same treatment as AMP — at least not yet. It took this long for the team behind AMP to figure out that the technology behind it could be universalized (or, if you prefer, de-Googlized), so it’s going to work with the rest of the web community to actually do it before it makes any firm decisions about what is going to get special treatment in search. Google has some ideas — including performance benchmarks for pages and support for instant-loading web packages — but won’t commit to a firm list just yet.
 
Once these standards are agreed upon and implemented, any page that conforms to them could appear in the Top Stories carousel, the Google news feed on Android, and even get the same little blue lightning bolt badge that appears next to AMP-supported results in search. These pages will not be guaranteed better placement in search results, however — just as AMP doesn’t guarantee search placement today. “The things you need to do in order to rank well in search will continue to be there and we will continue to not tell you what they are,” Besbris jokes.
 
I don’t think that Ubl’s post today is going to wash away the bad feelings that many have about AMP. There’s just too much wrapped up in it. AMP has served as a vessel for worries about Google’s power over the web, the dicey future of news publishing, and whatever general angst we all feel about the sorry state of most mobile webpages. Most of these concerns have been fair, but there hasn’t been enough momentum for a more open alternative. With this move, Google seems to be trying to create one.
 
Here’s how Besbris sums up today’s news:
 
The intention here is to be better about communicating our intent. We’ve always wanted to [make the technology behind AMP a web standard] and always said this, but not very clearly. … We just want to communicate very clearly that AMP is not anything other than trying to make the web better. The lessons learned from AMP are being put into the standard bodies so we can have other frameworks implement the same stuff too.
 
The discussion about AMP since its inception has been confusing and contentious. That’s usually the way with web standards. But I think that the stakes with AMP feel higher than previous battles because of a growing understanding that we can’t take the existence of the open web for granted anymore. It could very well fade away. It has to be maintained and, more than that, improved.
 
That’s not to say that everybody should just trust Google completely to have the open web’s best interests at heart. The company certainly benefits more than its competitors from a strong and vibrant web and is so influential over the web that it sometimes looks like domination. Google could stand to be a little more cognizant of those realities when it talks about the web.
 
As we start wrapping up our discussion about AMP, Besbris repeats one of my questions back at me: “Have we just been idiots around communicating this stuff?”
 
”Yes,” he answers.

---