What does 'open' mean in the open data revolution?
06 January 2015

In my last blog, I wrote about the risks to anonymity facing participants in confidential research projects if we don’t give sufficient thought to the risk posed from creative analysts accessing survey data.

As you can read in my next Editorial (57/1, January), the MRS Census and Geodemographics Group (CGG) conference held last November focused on the growing opportunities now available through open data initiatives. Whilst the conference focused on the UK, similar initiatives are happening in other countries, not just to provide access to data in the public sector, but to encourage commercial organisations to open up their data resources to wider audiences.

As I describe, one speaker provided feedback from the Open Data Institute summit, held the day before in London where Tim Berners-Lee had argued that open data would be as transformational as the internet has been. However, in the week I write this (19th December), we see three contrasting perspectives on open data.

First, there’s the revelation that Sony’s immense data stores have been comprehensively hacked, with a cascade of commercially sensitive data being ‘released’ to the world. It is believed that is an attempt, successfully, it seems to prevent the release of a new comedy film called ‘The Interview’ that involves an assassination of Kim Jong-un, the premier of North Korea. This underlines the need for boundaries in disclosure. 

Then, on Thursday two pieces in the Guardian touch on open data from different perspectives. Firstly, an article by Aditya Chakrabortty describing how a cash-strapped Barnet council in London has developed a strategy to outsource all possible services to the private sector. One result of this, argues Chakrabortty, is a loss of accountability as many aspects of these outsourced contracts are ‘shrouded in “commercial sensitivity”’. This was a key concern raised in the report recommending the future strategy for government open data strategy published in March:

‘We recommend that companies contracting with the Government to provide contracted or outsourced goods and services should be required to make all data open on the same terms as the sponsoring department. This stipulation should be included in a universal standard contract clause which should be introduced and enforced across Government from the beginning of the financial year 2015-16’. 

However, this is a recommendation looking to the future at a time when many local authorities, and government departments, are already locked into outsource agreements. And one stream of accountability comes from surveying the attitudes of users of these services, either on proposed changes, or to gauge satisfaction levels. Will such data be kept from the public gaze due to confidentiality agreements?

Coincidentally, I guess, the same issue of the Guardian contains ‘An open letter to digital change makers’ issued by Alex Willcock, CEO & Founder of VisualDNA, encouraging the release of data to create a new knowledge based economy facilitating the innovative development of new products and services in the public and commercial sectors that will enhance peoples’ lives. 

Key to this future, the author argues, is that, firstly, “permission based information about the authentic and in-depth ‘who’ of us becomes widely available”; secondly, ”when the growing silos of data that currently exist about an individual become joined”. The letter forecasts that: “We will eventually look back at today and be amazed at how it was possible to go through life being ‘not-understood”. Finally, the author argues that :” The only reason to hold back from granting this emancipation of our digital self is commerce’s desire to own something that never belonged to it in the first place”. You can find this manifesto here.

So what should be open, and what should remain confidential? How much do we want to be ’understood’, & by whom? This is crucial to the open data debate. What should we allow to be shared about us (the ‘who’ data), and can we truly appreciate how such data might be used when giving permission, especially when siloes are joined? What should be shared with us as citizens and consumers where we may have a vested interest in the performance of a service provider, and their strategy? Where will organisations decide the boundary will be between open and closed data, and how will such decisions be made? Lots of questions that need to be debated, and the research sector as we know it should be at the heart of it.