David Harper’s Different Things

Rules for Ethical Reformatting: A Developer Manifesto by Luca Passani.

A ghost is haunting the mobile web, the ghost of content reformatting. The mobile web has never been a simple platform to develop for, given the fragmentation of the underlying technology: different devices, different browsers and different networks have made mobile web development a challenge for programmers and content authors from the beginning.

Yet, in this hard environment, one thing has stayed sacred throughout the years: the HTTP Protocol.

Until today, developers of mobile websites could count on the fact that the HTTP headers from a mobile device would reach the intended webserver with their integrity respected.

Alas, this basic assumption on which thousands of developers have built their applications is put to the test by a recent trend among Mobile Network Operators (MNOs).

Tools that were originally devised as utilities to split up web pages and deliver a best-effort user-experience on a mobile device are today being marketed as solutions to bring the full web to mobile by reformatting vendors. Some MNOs are buying into this vision and are implementing reformatting proxies into their networks that will intercept and reformat all HTTP traffic to any HTTP server without safeguarding those sites which strive to deliver a mobile-optimised user-experience, or, at least, not safeguarding them sufficiently.

This situation is a threat to the neutrality of the web and one that can jeopardize the mobile web as a platform in the years to come.

To this end, mobile developers of various nationalities and background, assembled on the WMLProgramming list at Yahoo Groups create this manifesto to make their position known in the face of those who try to misrepresent the needs and the wishes of the mobile ecosystem for their own petty monetization needs.

Learn more and consider lending your support: Rules for Ethical Reformatting: A Developer Manifesto by Luca Passani.

–David Harper, Founder & CEO, Winksite

It’s getting even worse for mobile developers and content providers.

Now it’s Yahoo!/Novarra. Who else?

Novarra to Provide Mobile Web Tranformation Services to Yahoo!
http://novarra.com/press_rel/072407yahoo.html

Chicago, IL USA – July 24, 2007 – Novarra, the leading provider of next-generation mobile Internet platforms and services, announced today that Yahoo! has selected Novarra’s Vision™ server version 6.5 platform to provide web transformation for Yahoo!’s oneSearch service. When a user clicks on a link to a Web site in a Yahoo! oneSearch result, Novarra’s Vision server platform transforms the Web site for the mobile phone.

We decided to check up on what was happening to Winksite.

This is the result…

Yahoo!’s oneSearch/Novarra is providing mobile device search results that link to transcoded versions of our broadband web site rather then linking to our mobile version.

This is after requesting that our domain be placed on the oneSearch White List MONTHS AGO and submitting three of our sitemaps for indexing - broadband site, WML, & XHTML.

WTF.

I have a responsibility to 25K plus publishers who trust their mobile sites to our care to do everything I can to see to it that their content makes it through to their audience.

Sorting out an off-portal publishing system to deliver mobile device-specific 100% W3C mobileOK and .mobi standards-compliant templates was apparently not enough. Now it seems I have to take on everyone in the world who is bastardizing us with their freaking Novarra “web transformation” engines and poor customer service (i.e. whitelisting.)

Is there a reason why mobile developers and content providers have to jump through so many hoops and fight so many battles in order to reach their viewers in a manner equal to how the broadband Internet operates everyday?

Is anyone listening?

CLARIFICATION:
Yahoo!’s oneSearch service clearly distinguishes mobile and web results. They are returning both PC and desktop links to Winksite. Links in the PC section goes to the transcoder, while links in the Mobile Web section goes direct.

So why do I have a problem with this? Let me explain…

When someone visits Winksite or a mobile site published at Winksite we send that visitor to a version appropriate to their device. We asked to be White Listed because a mobile phone should never end up at a transcoded version of our broadband site. If Yahoo! respected that ALL the links in the results would pass to us directly where we could then do the job that people count on us to do.

Instead our routing is intercepted and adapted content NOT our content or the content of our publishers as it was intended to be distributed is sent. This cripples our site and renders our community features unusable.

I question Yahoo!’s (and anyone else’s) right to do that…

…and without permission to create a derivative work that violates our copyright AND hinders our ability to provide a service that took years of hard work to build.

UPDATE:
Nigel Choi has kindly invested his last weekend to provide the community with evidence of how the Novarra/Vodafone transcoding service disrupts the mobile experience:
http://wurfl.sourceforge.net/vodafonerant/vodawsj/nigel.html
————

For several weeks a firestorm has been taking place at Vodafone’s Developer Forum (Betavine, Mobile Internet Content Adaptation forum)

Simply put…

Vodafone UK started striping out essential device identification information that mobile phones send to content providers (i.e. the User-Agent String.)

The User-Agent String is used by developers and content providers to deliver mobile-optimized sites OR device-optimized content/services.

Vodafone’s actions thwart the efforts of companies in the mobile ecosystem who set out to provide a customized mobile presentation of their services, hurt these companies financially, and is counter to the advancements facilitated by groups such as the W3C and dotMobi.

For a distilled explanation of the Vodafone User-Agent string issue and how it affects the mobile Web please visit “Vodafone UK is abusing its position” by Luca Passani.

Mike Butcher over at Techcrunch UK has picked up on the story.

Luca Passani:

I am irritated with Vodafone. More that that. I am furious. I see an abuse and I am not sure what to do about it. But it’s an abuse. A Big one. Perpetrated by a large company in a dominant position against a myriad of small companies and against its own customers. An abuse that is damaging a whole industry in its infancy. I am talking about the industry of the mobile internet. I am talking about the possibilities for existing and new companies to have a new channel for selling content and services to consumers, and about a company which, from one day to the next, decides to pull the plug on the infrastructure that made this possible. The plug is pulled because this decision makes some tiny extra business sense for the big abusive company here and now, but it has no legitimacy whatsever, and the reason why the big abusive company can do it is merely technological: they manage the pipe the brings the data from the service provider to the consumer, and they decided to exploit this possibility to cut everyone else out.

Is this legal? I don’t know. Probably not. The problem is that, being this a relatively new field, there are no specific established regulations which spell out clearly what companies can or cannot do. By exploiting this uncertainty, the big abusive company is applying its dirty tricks and hoping to get away with it.

The abusive company I am talking about is Vodafone UK and the abuse is their decision to strip out essential device identification information that mobile phones send to content providers in order to let them serve customized content for each user’s device.

I want to bring the problem to public attention, make people aware of the issue and get everyone involved to do something about it. Read on…