Patents versus Accessibility — Again

I've ranted about frivolous patents more than once here. Others far closer to the issue do it daily, so my voice is but a drop in the ocean (and yet nothing happens).

This time it's a little different, and yet familiar (read my April post Patents versus Accessibility). Thanks to a tweet from WebAxe, I found out that somebody has patented (granted June 26, 2014) a tool that automatically makes a web site accessible. I find this a preposterous claim (at best), but here's the abstract from the patent:

A method for an accessibility solution provided as a software. The method includes approving or implementing by a website owner of a code into his website and receiving web format by a user device by “scraping” the data from the website pages or by other means of using client side plugin, server side plugin, browser or 3rd party server side or mobile app. The method also includes analyzing the data over the user device or on the server side and clicking a button by the end-user and the original code and the content and code that were collected from the website are rewritten which can also be done automatically as a suggestion to the end user, and the end-user sees or can use a new format according to the updated standard.

Unlike so many frivolous patents that are nothing more than an idea, this patent has some compelling claims. Unfortunately, the likelihood that all those claims can be realized seems pretty small.

Why I Am Wary

Just as anyone who practices accessibility understands that there is no automatic tool that can completely and truly evaluate a site's accessibility, there is also no tool that can automatically make a site accessible. To quote Jared Smith of WebAIM fame:

Accessibility is a continuum. It is based on the user experience. It cannot simply be defined as pass/fail or accessible/inaccessible.

While I don't want to attack what may very well be the well-intentioned best efforts of a dedicated person or team to improve the nature of accessibility on the web, I also have too many conversations with clients who are happy to accept a one-click solution at face value, unaware of the risks. More than anything, this post will serve as a quick reference for the inevitable conversation I'll have with a client or prospect about relying on a tool like this (or perhaps this tool in particular).

I am interested in how this tool can evaluate the value of alt text on images, verify whether an article or section is the correct sectioning element, make sure that placeholder text is not a barrier, understand when a button should be used, and so on. And then fix all these by re-writing HTML, JavaScript and CSS and not introduce more issues. These are just a tiny, miniscule slice of the kinds of decisions that require a person who understands the context in order to be successful.

The Patent Holder's Qualification

The evidence doesn't bear out that this tool can do what it advertises. The company that makes it, which promotes its accessibility services, has many issues on its own site. Which begs the question, did the company use its own tool and get such awful results, or does the company not see enough value in it to use it?

I wasn't the first to tear into the site. Folks on Twitter independently identified issues with color contrast, keyboard navigation (ok, that was me), image alt text, use of a carousel, a blank accessibility statement, incorrect ARIA role use, and misspelled ARIA attributes. All this feedback was unprompted.

To be fair, the tool may not exist yet. The patent was only just approved, though it was filed over two years ago. While I don't want to name and shame, there really is no way to make my point about the proposed tool without doing so.

The Patent Holder's Possible Example

Looking to the site of the registrant of the patent (I also made a Wayback permalink) I see an "Accessibility" box floating on its home page which may provide insight into how this patent might be applied. This "Accessibility" box allows some rudimentary control over the content. For example you can scale the text up or down, something I first built into sites about 15 years ago. You can change the text to white on black or black on white (yes, I also did that about 15 years ago). Oddly, you can convert the page to grayscale, removing all color information (it's not obvious how to turn that off, so it's still gray for me). You can toggle "alternate navigation," which just puts a big box around items that have keyboard focus (the site benefits since it wrongly uses outline:none). There is an option called "Speed control" which may adjust the rate at which its carousel/slider changes. There is an option to "Block blinking," though its affect was not immediately apparent to me.

I'm not sure that the patent would withstand scrutiny from someone who has worked in web accessibility. Some of the ideas outline, while not implemented in the tool on its own site, seem to lean on existing technologies that themselves are subsumed within patents. For example, one of the figures in the patent shows how optical character recognition (OCR) can be used to convert the text in an image into text that can be passed along to the user (figures 7a and 7b).

The Takeaway

As always with what feel like frivolous patents, I fear that they can be used to block truly useful and innovative solutions from coming to market. In this case, I worry that the already poor state of accessibility on the web could be stifled if the patent owners try to exercise any claims against those of us who've been doing accessibility work for years.

Patents versus Accessibility

Just one example of the many gesture patents rolling around in the USPTO database, this one for denoting something important by making a heart shape over it.

I haven't ranted about patents for a while, which doesn't mean there hasn't been any activity. In fact, there has been activity in precisely the space where I think patents can disproportionately hurt users — in accessibility technologies.

A couple weeks back Don Norman put together an overview of gestures as user interface behaviors in Gestural Control: The Good, the Bad, and the Ugly. His opening salvo sums up the concern with patent encumbrance:

I await the day when gestures become standardized. When systems combine the best of all worlds: gestures, voice, and menus, keyboards, and pointing devices. But before we can do this, we have a simple task to do: reform the patent system.

He rightly notes that while gestures can be powerful, they are not universal. Cultural differences alone can make gestures difficult to translate across users. Implementation of gestures is also inconsistent, even across devices from one manufacturer (looking at you, Apple, Microsoft, and everyone else). He does identify a scenario where gestures can be consistent:

Yes, some groups of people such as athletes and referees, military squads and others learn standard gestures to let them work efficiently where other means of communication are not possible, either because of the need to be silent or because of a high noise level.

I argue that this statement also applies to users with disabilities. For example, sign language is a common set of gestures that allows its users to communicate regardless of the ability to hear (whether by audio impairment or, as in the case of some of my friends, stealthily complaining about the server).

Patents on gestures is nothing new. The opening example of a heart-shaped gesture to allow Google Glass users to "like" things they see in real life dates back to 2011. That is from a very quick search in the US Patent & Trademark Office (USPTO) database for "gesture." If had a budget to search, I suspect I could find older.

The problem here is that gestures are already obtuse and complex enough in many cases, but patent-encumbered gestures will result in even more unintuitive and bizarre contortions to perform the same function across devices. This will also make it that much harder for users to switch platforms, let alone learn a first platform.

Think about this in the scope of accessibility. As I mused on Twitter, wouldn't it be nice if a user could enable and use accessibility features on any device with the same gesture (or set of gestures)? Just as I can pick up any device and read words on the screen in a generally consistent way, or just as I can expect a ramp to work that same way no matter the building to which it's attached, I think consistency removes a barrier to entry that is otherwise being built up more and more.

Example of a GUI with accessibility features in an Apple patent.

If you think accessibility features will be somehow immune to the patent process, consider Apple's patent for a graphical user interface for the visually impaired (there's a PDF of the entire application as well).

That Apple patent likely means if Samsung wants to use an interface that has even a passing resemblance to the one Apple has patented, you can expect Apple to sue. The same goes for a gesture. The same will go for gestures that are tied to accessibility features.

We've already got a patent system that's out of control, with even the most basic ideas receiving patents. It seems the same patent process is also building barriers to accessibility using the very ideals that should be using to tear barriers down. Ideally with enough people raising a stink we can slow or even stop the process.

Previous Patent Rants

• Frivolous Patents on the Web
• WebM, H.264 Debate Still Going
• More Frivolous Patents
• A Patent Trolling Primer
• Are Patents Killing HTML5 Video?

WHATWG as W3C Community Group in Name Only

As of Monday, April 23, The W3C has announced that it is looking for a new editor for the HTML Working Group specifically tasked with shepherding HTML5 through the process until it reaches a formal recommendation. Ian Hickson (Hixie) made the request for a call for his replacement so he could focus on ongoing HTML development, such as HTML.Next. With that announcement the W3C is also making the following changes:

• The W3C is beginning its replacement editor search specifically for HTML5 and HTML Canvas 2D Context to carry them both through the Recommendation phase of the specification process. The W3C expects the process to take 30 days, though it isn't clear if that means it will have candidates to consider at the end of the 30 days or a new editor by then.
• The W3C reminds us that W3C Community Groups and HTML WG Task Forces can continue to submit proposals to the specification.
• Hixie, the current HTML5 editor, will continue working on the WHATWG HTML specification. Though it will be versionless, the W3C expects it to spin off proposals to its HTML WG.
• The W3C states that editors of "Recommendation-Level" specifications (specs going through the W3C process, with versions) and authors of HTML.next (the WHATWG versionless specs) are free to make changes directly to their own specs, but the W3C is urging them to work together.
• The W3C is expanding the W3C HTML WG charter to allow it to work on HTML.Next at the same time the HTML5 specification is snaking its way through the official process. Once that happens, the W3C will be looking for more editors.
• The W3C is making clear that it plans to continue its partnership with WHATWG.

The W3C also created the Web Hypertext Application Technology Community Group on Monday. Given that W3C Community Groups allow open participation and can submit proposals for the specification, it looked like the W3C and WHATWG might be coming together and possibly opening up to more community feedback.

The potential and hope of moving all HTML5 (etc) development under one organization was quickly dashed when Hixie sent this message to the WHATWG mailing list:

[…] [I]t means we have a clear patent policy. Historically, we've relied on the W3C HTML working group for this, but as the W3C has focused on stablising their HTML5 snapshot, newer features have not enjoyed the same coverage. […] This will basically have no effect on how the WHATWG operates, except that when we make use of the W3C Community Group "Final Specification Agreement" (FSA) mechanism, anyone who wishes to co-sign the agreement [2] will be invited to do so.

The W3C Web Hypertext Application Technology Community Group echoes that message:

This is the W3C community group for the WHATWG, a mechanism through which the WHATWG can publish specifications with patent agreements.

At first glance it may have looked like WHATWG was coming together with the W3C, and by using a W3C Community Group allowing anyone to participate, but that does not appear to be the case. Instead this move seems to primarily be a method to bring WHATWG into the patent protection that the W3C enjoys through some of its partnerships, as suggested by Steve Faulkner:

but the Hypertext Application Technology Community Group appears to be nothing more than a ploy to get microsoft on board at the WHATWG via the provision of a patent policy.

Any hints of bringing WHATWG into the W3C fold appear to be gone. Any hope that anyone can participate appears to be dashed as well, since it doesn't look like the Community Group will have its own mailing list and the forum will likely be ignored.

Unfortunately, I was quick to sign up for the Web Hypertext Application Technology Community Group but took far too long to realize it wasn't going to change the process of participation. I suppose I could blame patent chaos for all of this, or I could just accept that the HTML5 process isn't going to change.

Patent Wars Sorta-Infographic

I'm giving in to the cool hip trend of infographics that has been popping up like pinkeye across blogging and tech sites lately. These infographics are typically nothing more than data points (sometimes just narrative) strewn about with mathematically suspect charts or somewhat-related design elements. But they seem to draw traffic, even when there isn't even data to graph (Browsers as Wrestlers "Infographic"). So I am using my lack of shame to power through this long weekend with three posts of three infographics from other sites.

In today's installment I am posting an infographic that has some (imprecise) charts and a couple process maps outlining the current state of patents called Patent Wars: A New Age of Competition. You can find the original image at Business Insurance Quotes site, where it has no accompanying explanation or background.

Related

• Frivolous Patents on the Web
• WebM, H.264 Debate Still Going
• More Frivolous Patents
• A Patent Trolling Primer
• Are Patents Killing HTML5 Video?

Are Patents Killing HTML5 Video?

You may recall from my post in February, WebM, H.264 Debate Still Going, that the H.264 video codec is considered patent-encumbered (which resulted in its dismissal from the HTML5 specification) and Google has argued that its own WebM / VP8 codec is made up of patents it owns, releasing it as royalty-free.

We all had to know it wouldn't be that simple. MPEG-LA, the licensing entity for multimedia codecs such as H.264, put out a call for patents related to VP8, the underlying technology in WebM, back on February 11, leaving it open for a month. Ostensibly MPEG-LA cast its wide net in the hopes that it could find existing patents that it can then use (perhaps by forming a patent pool or preparing for a lawsuit) to lay claim to rights over technology in VP8.

In April, Google started its own call for patent holders and formed the WebM Community Cross-License (CCL) initiative. Its description from its site:

The WebM Community Cross-License (CCL) initiative enables the web community to further support the WebM Project. Google, Matroska and the Xiph.Org Foundation make the various components of WebM openly available on royalty-free terms. By joining the CCL, member organizations likewise agree to license patents they may have that are essential to WebM technologies to other members of the CCL.

With no official announcement, MPEG-LA revealed in an interview (WebM Patent Fight Ahead for Google?) that twelve parties had come forward with patents they felt were covered in VP8. If those claims hold up then MPEG-LA's next step is to create a patent pool, which allows even more patents and patent holders to be added to the mix. It also means Google will be faced with either paying for licensing from that patent pool or defending itself in a lawsuit. Given Google's US $105 million outlay to acquire the VP8 codec (via its acquisition of On2) already, you can bet some number crunching will take place to evaluate the value of either approach.

MPEG-LA most likely doesn't care whether or not VP8 wins over H.264. All MPEG-LA is interested in is getting its licensing fees from both of them. While MPEG-LA doesn't necessarily fit the model of a standard patent troll (it isn't located in Texas), if you read my post from yesterday, A Patent Trolling Primer, you can see some parallels.

These are posts I have written both about patent abuse and the H.264 vs. WebM debate. Instead of a recap of each here, you can get more history in these posts:

• A Patent Trolling Primer, August 1, 2011.
• More Frivolous Patents, July 31, 2011.
• Frivolous Patents on the Web, October 15, 2010.
• WebM, H.264 Debate Still Going, February 21, 2011.
• H.264 Getting Dropped from Chrome, January 13, 2011.

Back in January, Moving Picture Experts Group (MPEG, a working group of ISO/IEC and not related to MPEG-LA) issued a statement that it plans to move forward with a royalty-free video encoding standard. Sadly, I don't have any news on its progress. If you have any, please comment below and let me know.

Other Media Types

As proponents of Ogg Vorbis will tell you on the We Want Ogg site, while they claim that it is a royalty-free and unpatented audio format, they cannot seem to get either Apple or Microsoft to support it in their browsers — even though the code to embed it natively in the browser is available (a better solution than a plug-in). In this case, owning patents related to audio technologies as a browser maker is motivation to not support an open format.

Google's WebP image format, which uses some of the still-image compression tricks from VP8, isn't being supported in Mozilla (Mozilla rejects WebP image format, Google adds it to Picasa). While patent issues around VP8 may certainly creep into WebP, in this case the argument to refuse support for the image format comes down to its lack of clear benefits to other image formats. If VP8 compression formats become patent-encumbered then you can bet this new format will die on the vine.

Related

• MPEG LA: 12 companies own patents essential to Google's VP8 codec
• Royalty-free MPEG video codec ups the ante for Google's WebM/VP8
• Google builds WebM patent pool of its own to fight back against MPEG-LA

Update: March 7, 2013

Today on Google's WebM blog:

Today Google Inc. and MPEG LA, LLC announced agreements that will result in MPEG LA ending its efforts to form a VP8 patent pool.

There is no indication of how much money changed hands.

A Patent Trolling Primer

The timing on my post yesterday (More Frivolous Patents) was pretty good. The patent trolling issues have been getting some coverage lately in more mainstream press (not just in legal or industry press) which has the potential to actually get noticed by someone (or someones) who can make a change.

Unfortunately, some people seem to think that patent reform is already in progress via the America Invents Act (S. 23 & H.R. 1249), which has so far only passed the House. The summary of the bill from OpenCongress.org:

Changes the patent application system from one that awards patents to the first person to invent something to a system that awards the first person to file a patent application on an invention. It would also allow for more public feedback on applications in order to prevent bad patents, establish new rules for challenging patents, guarantee that fees from patent applications can be retained by the patent office, and more.

Not being a lawyer, I have only the wisdom of the Internet on which to rely when it comes to interpreting the legalese of the bill. The bulk of what I have found is focused more on the first-to-file changes of the law. From CNN Money (Patent reform is finally on its way, June 24, 2011):

At the core of both bills is a transition of the U.S. patent law from a "first to invent" to a "first to file" system. That would give the patent to the first applicant, rather than the first inventor. It's the standard most of the rest of the world uses, since it prevents inventors from coming out of the woodwork and and laying claim to a patent.

The CNN article also says that the bill is designed to keep patent battles out of the courts and links to a 2008 article about a trampoline safety net inventor (Is your idea safe?).This bill, however, does not address the overwhelming surge of patent trolling in recent years. In addition, there are a lot of people who don't care for this bill, as a Google search will show.

Last week the radio program This American Life ran an episode titled "When Patents Attack!" detailing its experience with just one patent troll, Intellectual Ventures (who is against the America Invents Act). I have embedded the (too large to fit comfortably in my blog) audio player for the episode. It's perhaps a bit comedic, but it is worth a listen if patent trolling is new to you, especially if you aren't technically inclined.

To tear a piece out of the story out and provide some context about how broken the patent system is, enjoy this example of chaos:

Crawford's patent was for "an online backup system." Another patent from the same time was for "efficiently backing up files using multiple computer systems." Yet another was for "mirroring data in a remote data storage system."

And then there were three different patents with three different patent numbers but that all had the same title: "System and method for backing up computer files over a wide area computer network."

Martin says about 30 percent of U.S. patents are essentially on things that have already been invented. In 2000, for example, the patent office granted a patent on making toast — patent number 6080436, "Bread Refreshing Method."

SFGate takes it a bit further in its piece "Who Does Patent-Trading Firm Intellectual Ventures Work For, Anyway?"

What NPR does not get into, however, is the fact that Intellectual Ventures is backed by many of the biggest and most powerful companies in technology, including Amazon, Apple, Cisco, eBay, Google, Nokia, Sony, Yahoo, and — of course — Microsoft.

[...]

These companies are believed to have invested in Intellectual Ventures in part to make sure they have access to its huge patent trove. (Other investors include universities and investment firms.)

The Economist posted its own take on the chaos today, spurred by the piece from This American Life in the article Patents against prosperity. The author makes the case for patent trolling stifling the American economy:

Nevertheless, it remains that America is the world's leader in technical invention, and continues to attract many of the world's most inventive minds. That's why it is so important that America remain especially conducive to innovation. And that's why America's intellectual-property system is a travesty which threatens the wealth and welfare of the whole world.

Some of the commentors on the story make interesting arguments, but nobody denies that the current state of the patent system is broken.

Regardless of your source, they all tend agree that patent trolls are a real concern to businesses, with the risk of litigation ever-present. This has created the new business model of paying into patent companies as a form of protection money, whether by partnering up (such as the Nortel patent bidding) or paying one off.

Yesterday I talked about more companies on the web getting sued, tomorrow (if time allows) I plan to discuss the new MPEG-LA patent pool.

More Frivolous Patents

The patent trolling continues, which should really be no surprise. Consider that Nortel put its portfolio of 6,000 patents on the auction block, with Google starting bidding at $900 million dollars only to be beat out by a consortium of Apple, Microsoft, Research in Motion, EMC, Ericsson and Sony for $4.5 billion ($750 million each). Each of these companies, Google included, wants to protect itself from patent trolls along with finding new revenue streams from licensing options.

The fun doesn't end there...

Lodsys sues Rovio

Rovio, the makers of the wildly popular (and somewhat addictive) Angry Birds game, is being sued for patent infringement by Lodsys. Lodsys claims that it holds a patent that covers how Angry Birds allows players to purchase new levels in the game. While Apple has maintained that its licenses extend to third-party developers, Lodsys started laying its groundwork in a May 31 post, Apple's License Claim Disputed.

The Lodsys About the Inventor page claims that Dan Abelow is a prolific inventor and an expert on website usability, ease-of-use and assured user performance. It then uses a "click here" link to direct traffic to Dan Abelow's web site. His web site content consists of text as images with image maps for the in-content links. If you are web developer who isn't even at the "expert" level he claims, then you already know that his claim of being an expert on usability is clearly a flat-out lie.

That's the nature of the person behind this suit.

PacketVideo sues Spotify

Spotify has just made its debut in the United States and almost immediately after doing so is getting sued. PacketVideo is leaning on a 1995 (issued in 1997) U.S. patent related to streaming music, itself citing six other patents. PacketVideo was founded in 1998 and purchased by NTT Docomo Inc., Japan's largest mobile phone operator, in August 2010. The patent in question has stayed with PacketVideo. PacketVideo is also suing over a European patent originally owned by a Swiss company.

PacketVideo outlines its position in a press release on its site, PacketVideo Files Patent Infringement Lawsuits Against Spotify. Spotify's response is to be expected: PacketVideo is claiming that by distributing music over the Internet, Spotify (and by inference any other similar digital music service) has infringed one of the patents that has previously been acquired by PacketVideo. Spotify is strongly contesting PacketVideo's claim.

I am wondering if Turntable.fm is just too small to bother suing.

Geotag sues 397 Companies

Last year a client reached out to me over some concern in the jewelry industry about patent lawsuits from a company called Geotag. In particular, companies that used a "find a retailer" feature to allow users to provide a zip code to see nearby stores were being sued. Given that Geotag paid $119 million for the patent, it was clearly in it for the shakedowm. This sampling shows that Geotag was really after the big fish, probably in hopes of a payout: Cartier and its Find a Boutique function at Cartier.us; Rolex and its Dealer Finder at Rolex.com; Sterling Jewelers' store locator on both Kay.com and Jared.com; Van Cleef & Arpels Boutique Search on VanCleef-Arpels.com; the store locator on Zale Corp.-owned GordonsJewelers.com; the store and event finder on Tiffany.com and the store locator on Kroger Co.-owned FredMeyer.com.

Not being a lawyer, I could not counsel my client on any particular legal action, but I could reassure my client that since Google Maps and Microsoft Bing provide much of this capability, it would be a matter of time before those two would either get caught up or step in and duke it out with the patent troll. Strange bedfellows indeed: Microsoft and Google jointly sue GeoTag Inc. in order to invalidate a patent asserted against more than 300 entities and Microsoft And Google Jointly Hit GeoTag With A Lawsuit To Invalidate Its Patent.

Related

• Frivolous Patents on the Web
• Big bidding: Apple, Microsoft, RIM nab Nortel patents for $4.5 billion
• Angry Birds maker sued by US firm Lodsys for 'violating patents'
• That Didn't Take Long: Spotify Sued For Patent Infringement Just Weeks After Entering US Market
• Tech co. sues majors over Web site component
• Twitter Gets Sued For Letting Famous People Interact Online
• Fighting patent 'trolls' — there's no app for that

WebM, H.264 Debate Still Going

On February 2, Microsoft released a plug-in for Chrome on Windows 7 to allow users to play H.264 video directly in Chrome. In addition, Microsoft has said that it will support WebM (VP8) when a user has the codec installed. And so began the fragmentation of the HTML video model, back to relying on plug-ins to support what was otherwise intended to be supported natively within browsers.

Microsoft went on to ask three broad questions in a separate post (HTML5 and Web Video: Questions for the Industry from the Community), taking Google to task for what Microsoft considers inconsistent application of its own patent concerns and openness (emphasis Microsoft's):

1. Who bears the liability and risk for consumers, businesses, and developers until the legal system resolves the intellectual property issues;
2. When and how does Google make room for the Open Web Standards community to engage genuinely;
3. What is the plan for restoring consistency across devices, Web services, and the PC.

The same day Microsoft was announcing its plug-in approach to addressing the WebM and H.264 battle, the post On WebM again: freedom, quality, patents came out addressing what it felt were the five most common issues raised with WebM (which I have paraphrased):

1. Quality: the argument here is that it's a function of the encoder and the WebM can match H.264;
2. Patent Risk: comparing the 164 unexpired U.S. patents used in a single encoder, he finds that 126 of them are there for the encoder's H.264 support, the remaining (used by WebM) are in a library released by Google.
3. Not open enough: there is little argument here, given that it's currently in Google's hands to manage and develop.
4. H.264 is not so encumbered: but only for non-commercial use for freely-distributed web video.
5. Google provides no protection from infringing patents: nor does MPEG-LA.

Changing the Nature of the Battle

On February 11, the post MPEG LA puts Google's WebM video format VP8 under patent scrutiny outlines how MPEG-LA, the licensing entity for multimedia codecs such as H.264, has put out a call for patents related to VP8, the underlying technology in WebM. That deadline for submissions is March 18, or less than a month away as of this writing. From there, MPEG-LA will create a patent pool of contributing patent holders for any items that are deemed essential to the codec. This patent pool can then be used to negotiate licensing. In short, VP8/WebM could soon be more patent encumbered than it has been. This puts Google on the defensive as it will have to show that none of the patents in use are valid and/or infringed.

The same author of that last post posted on the 14th at The Guardian, Royalty-free MPEG video codec ups the ante for Google's WebM/VP8. In case that title isn't clear enough, a new royalty-free video standard may be in the pipes. MPEG, a standards body separate from MPEG-LA (the licensing body) has called for proposals toward a royalty-free MPEG video coding standard. One of the goals is to make this new standard comparable to the baseline used in H.264.

If this pans out, it puts another barrier in front of the WebM offering from Google, namely that for WebM to be adopted it will have to best any new royalty-free MPEG codec. The three items that can bring WebM (VP8) down:

1. If the MPEG-LA call for for patents and resulting patent pool for VP8 nets some patents, MPEG-LA will now form a patent pool to push for licensing agreements, which Google will have to fight at each step.
2. If MPEG can genuinely develop a royalty-free video coding standard, it can beat WebM either from the patent perspective or by forcing WebM to be technically superior.
3. Assuming WebM can get past the first two items, it's still back where it started — in a battle for adoption and endorsement against the already entrenched H.264 standard.

Real-World Needs

Considering Google is the company that delivers so much of the video viewed on the web via YouTube, it makes sense that Google presumes it can take the lead on video codec standards. Netflix, however, has its entire business model built around video, which is now moving inexorably to the web. Netflix commented back in December (HTML5 and Video Streaming) that it needs some things sorted out before it can even rely on HTML5 and video to deliver its content (the first 6 of which it has resolved through its own proprietary technology):

1. The acceptable A/V container formats (e.g. Fragmented MP4, WebM, etc.);
2. The acceptable audio and video codecs (e.g. H.264, VP8, AAC, etc.);
3. The streaming protocol (e.g. HTTP, RTP, etc.);
4. A way for the streaming protocol to adapt to available bandwidth;
5. A way of conveying information about available streams and other parameters to the streaming player module;
6. A way of supporting protected content (e.g. with DRM systems);
7. A way of exposing all this functionality into HTML5.

It's clear that the web video debate extends far beyond the academics of HTML5. As long as issues related to patents and licensing are unresolved, or perceived as unresolved, we will see more proprietary solutions gain more ground, further balkanizing the future of video on the web.

If you think this doesn't affect the end user, all that time and effort put into creating proprietary solutions ultimately costs you as the consumer in the form of increased fees for your content. Granted, it will be some time for browsers to catch up with the selected codec, and yet more time for users to catch up with the supporting browsers, but the more this debate continues, the longer before we can even start that long road of user adoption.

Possibly the best outcome we can hope for is that this battle results in a royalty-free MPEG standard, nullifying many of the arguments against both H.264 and WebM.

Related

• H.264 Getting Dropped from Chrome, January 11, 2011.
• Simple HTML5 video player with Flash fallback and custom controls, February 17, 2011.

Frivolous Patents on the Web

The history of the web is littered with unexpected patent claims that feel frivolous to nearly all. These are driven partly by a terrible system for reviewing and awarding patents and a litigious society that sustains a business model of lawsuits in the hopes that some will just pay.

The latest example involves a company called Webvention attempting to claim an $80,000 license fee from Novartis for using hyperlinked rollover images on a web site. To be clear, this isn't exactly a new idea. And the patent itself is painfully general. Because Webvention went for a large company, that company has the resources to fight it. Read the full article at ars technica: Rollover image on your website? That will be $80,000 (please)

A History of Frivolity

If you are new to the web, perhaps you didn't know about about other patent infringement lawsuits — ones that others have long since forgotten. For a little context I've included information about a few (this is not an exhaustive list) so you can see for yourself that the trend isn't dying, but the lawsuits aren't holding up. In short, you probably don't need to worry.

Unisys GIF Patent

Way back in the early days of the web, Unisys attempted to assert its claim to the LZW compression method used in the GIF iamge file format, at the time the most popular format on the early web and online providers. Ultimately this led to the development of the PNG format which, while a compelling format, doesn't have the footprint on the web that GIF has, even nearly a decade on.

• The GIF Controversy: A Software Developer's Perspective, published January 27, 1995, revised through June 20, 2004.
• GIF is NOW finally free - for real, with a final Unisys joke, published September 28, 2006.

Overture Search

Overture went after Google and other search engines under the claim the keyword bidding model used for paid search engine placement was a violation of Overture's search patent. This "business method patent," as they have come to be known have included others like the Amazon 1-click ordering process.

• Overture Sues Google, published April 7, 2002.

British Telecom and Hyperlinks

British Telecom went after ISPs by looking for license fees for hosting web pages that contained hyperlinks. Prodigy successfully fended off the lawsuit, probably in part by pointing out a hypertext system dating back to 1962.

• BT loses hyperlink patent case, published 23 August, 2002.

Eolas Targets Plug-ins

The University of California apparently came up with the concept of browser plug-ins, sold the license to that patent to Eolas, and Eolas turned around and sued Microsoft for patent infringement. Plug-in manufacturers quickly started offering workarounds to the way plug-ins are embedded. The developer community was in such an uproar that the W3C even formed an advisory group to address it. Once the USPTO issued a rejection of all claims, the W3C folded up the group and Microsoft was not required to make any changes.

• Plug-ins Getting Unplugged from IE, published October 7, 2003.
• HTML PAG Report, published March 22, 2004.

Images as Hyperlinks

Apparently convinced their approach is vastly different from what British Telecom tried to enforce, Singapore-based Vuestar has been invoicing companies for using images as hyperlinks, a violation of its patent. Given that the patent was filed in 2001, there is about a decade of prior art that makes this about as frivolous as they come.

• Patent troll sending out invoices for image hyperlinks, published May 28, 2008.

How This Affects Progress

When you look at even this abbreviated history, it's easy to understand why the video codec H.264, initially considered to be the next standard for web video, has undergone such an assault of late. Because the patents for H.264 are held by the MPEG-LA consortium, and because the browser makers are taking different approaches to implementing it based on license fears, it has been dumped from the HTML5 specification. HTML5 is now silent on codecs for video and audio elements. Take a few minutes to read up...

• [whatwg] Codecs for audio and video, published June 29, 2009.
• GIF, H.264, and Patents, published March 22, 2010.
• Why H.264 is disqualified from being a web standard, published September 4, 2010.