How Britain's First Open Data Portal Began: Andrew Stott

Retired British civil servant Andrew Stott shares his experience of creating Britain’s first open data portal,
How Britain's First Open Data Portal Began: Andrew Stott

It was May 13, 2009, the first day at work for Britain’s first Director of Digital Engagement, Andrew Stott. Dubbed the “Twitter Tsar” by the local press, he was to help the country’s civil servants use social media to engage with citizens more effectively.

“It was my first day on the job – I thought I would be on Twitter all day,” the now retired civil servant told Challenge during a recent trip to Singapore.

Instead, he received a letter from then Prime Minister Gordon Brown’s office, informing him that Sir Tim Berners-Lee, the computer scientist credited with inventing the Internet, had been urging the government to create an “open data” platform so that data that can be freely used, re-used and redistributed. (See “Why open data?”)

Was this feasible? Could it be done in the next six months? The Prime Minister’s office wanted to know.

“In a way, they said: ‘There’s no particular hurry about it, but could we have a full project business plan by close of today?’” recounted Mr Stott with a burst of laughter.

So instead of firing away tweets, he spent the day drawing up a plan. This would form the foundation of, the British government’s central online repository of non-personal public datasets.

A month later, Sir Berners-Lee was appointed to oversee the creation of while Mr Stott was tasked to make the open data portal a reality. The Guardian newspaper called him the “perfect choice” as he had worked on IT projects and in communications for many years in the government.

How Britain's First Open Data Portal Began: Andrew Stott

Why open data?

Advocates say it increases transparency in governance by making data like public expenditure and salaries available. A more informed public would be encouraged to participate in governance and decision-making. Businesses and services can also tap public data in creative ways to deliver social and commercial value. In London, public data of every reported bicycle accident was used to map out the most dangerous areas for cycling in the city.

Changing mindsets

By January 2010, the public beta site of was launched with 3,000 data-sets. This was three times the number of datasets first released by the United States’, the world’s first national open data site. But it had been tough convincing public agencies – the data owners – to unlock their data. “The problems are not technical. It’s policies; it’s the mindsets of officials,” said Mr Stott. Hence, one of the first things he did was to appoint “open data champions” in ministries to change mindsets. Such a person must be of sufficient seniority to talk “as an equal to the person who could authorise release of data”, he said. The team also aimed for the low-hanging fruit – “data that was already published but wasn’t available as re-usable data” – to score quick wins.

To qualify as open data, datasets have to be in machine-readable formats, such as spreadsheets, and not in PDFs. Converting data to reusable formats was the first step in unlocking public data.

How Britain's First Open Data Portal Began: Andrew Stott

Show, don’t just tell

Another strategy was to show data owners how their data could be used to their advantage. “The policymakers were absolutely delighted to see how their data could be used to further the department’s objectives,” said Mr Stott of the closed-door hackathons organised with the British Home Office, which looks after policing, terrorism and immigration issues. During these sessions, software developers, who signed non-disclosure agreements, coded programmes using the agencies’ data.

It’s not your data, it’s the public’s data in the first place; they paid for it to be collected.

A successful outcome was the hyperlocal portal. Enter your postcode, town or street name and you will see details of the local police team, the crimes reported and their outcomes. Users can also track crime levels over time.

The portal is also “designed to get citizens to engage more with the police and to take part in setting local neighbourhood policing priorities” – an outcome that dovetailed with Mr Stott’s digital engagement mandate.

Create bespoke solutions

When faced with resistance, the team dug deeper and found two primary concerns:

First: the nail-biting uncertainty of what the data would be used for and the deeper fear that people would use the data to criticise the government. In response, Mr Stott would tell the data owners: “That’s not your concern. It’s not your data, it’s the public’s data in the first place; they paid for it to be collected.”

Second: the fear that the published data is inaccurate. “They are afraid that people will send them corrections and they wouldn't know what to do,” he explained.

To address these concerns, a unique Open Government Licence was introduced to give users the right to freely re-use published data, even for commercial purposes, but it also protects data owners from any errors or omissions in their data.

“One thing is apparent,” said Mr Stott. “Releasing data has generated more pressure and support internally to improve data quality much more than before.”

Now more public agencies are willing to put their data to the test through crowdsourcing. For instance, when the national database of bus stops was shared on Open Street Map (a Wikipedia of maps), the site’s volunteers validated the 300,000 bus stop locations and found 18,000 that were inaccurate. The national database was promptly corrected and updated.

How Britain's First Open Data Portal Began: Andrew Stott

Encourage the re-use of data

While it is critical to unlock datasets, it is just as important that governments actively encourage the use of the data, said Mr Stott.

This can be done by releasing data on matters people care about, such as healthcare costs, or by allowing users to fill in simple online forms to request and suggest new datasets. Open data must be genuinely open so users can use it freely, for any purpose, as long as it is lawful. Onerous terms and conditions will stifle the re-use and potential, he stressed.

These days, Mr Stott advises on open data and transparency issues as a member of Britain’s Public Sector Transparency Board, and travels widely to speak on open data.

He may no longer be the Twitter Tsar, but he hasn’t stopped tweeting from his @DirDigEng account, created when he took on the digital engagement job. On August 28, 2013, he tweeted to the world: “data/gov/UK has passed 10,000 data-sets this morning – well done everyone! #opendata”.

Saving millions thanks to open data

Using publicly available National Health Service (NHS) prescriptions data, data research company Mastodon C studied the prescribing patterns of doctors for a class of drugs called statins. Generic and branded statins are equally safe and effective so doctors are advised to prescribe the cheapest. However, Mastodon C’s research showed that doctors were prescribing the more expensive variety, which cost the NHS an extra £200 million (about S$398 million) a year. Open data helped the NHS to see clearly where it was overspending.

Singapore's open data initiative

Singapore’s open data initiative is driven by the Ministry of Finance, together with the Infocomm Development Authority of Singapore, Singapore Land Authority and the Department of Statistics. More than 8,700 publicly accessible datasets from 67 agencies are available on the open data platform, and over 65 thematic map layers on The Gov Data Steering Committee – comprising 22 key data source and user agencies – is pushing for greater data sharing to improve policy-making and to make public services more citizen-centric. It arbitrates on cross-agency data disputes and has developed a set of data-sharing principles to guide agencies’ open data efforts. So far, more than 110 apps and services have been created using public datasets. Following the success of ideas4Apps Challenge in 2012, a Public Service edition – ideas4SG challenge – was launched in June 2013. The result was more than 900 crowdsourced ideas on the use of government data. Winning entries include combining Singapore Customs’ immigration checkpoint data with the Land Transport Authority’s real-time traffic data to provide time estimates for users to decide which checkpoint to use or whether to proceed based on traffic conditions.

Shortlisted ideas at:

    Nov 28, 2013
    Bridgette See
  • link facebook
  • link twitter
  • link whatsapp
  • link email