IJMR Landmark Paper: ‘Role of the ESRC Data Archive in the dissemination of data for secondary analysis’
27 July 2018

‘Role of the ESRC Data Archive in the dissemination of data for secondary analysis’, Denise Lievesley, then Director, ESRC Data Archive, University of Essex, JMRS Vol 35 No 3, July 1993

I’ve written in an earlier Editorial about the excellent work being undertaken by the Archive of Market and Social Research (AMSR) in the UK (IJMR 59/3, 2017, and see: http://www.amsr.org.uk/).

In June AMRS announced that Professor Denise Lievesley CBE, Principal of Green Templeton College, Oxford, had accepted their invitation to become the first President. As you will see from the AMSR Newsletter, Denise has enjoyed a very prestigious career as a social scientist of international renown, which began at OPCS (now ONS) and SCPR, the forerunner of today’s NatCen. I remember Denise as a fellow member of the Market Research Development Fund (MRDF), where we worked together on a number of projects and seminars in the 1980s examining various aspects of response level across research methods. When I later became chair of the re-constituted Research Development Fund in the mid 1990’s, Denise as a member was instrumental in helping develop the major two part project we conducted on non-response and  in gaining us access to the then OPCS omnibus survey to develop a sample of UK adults for the ‘Public Co-operation in Market Research’ project (the second part being the ‘Business Co-operation in Market Research’). An MRS Conference paper from 1996 describing the project can be found in the AMRS archive at https://amsr.contentdm.oclc.org/digital/collection/p21050coll4/id/76/rec/3 (‘RDF Working Party Public - Co-operation and Response’, Mouncey & Windle). Denise was also a keynote speaker at one of our IJMR Research Methods Forum one day conferences a few years ago.

So, it seemed appropriate to feature a paper by Denise from our journal archive as our latest IJMR Landmark Paper in recognition of her long association with the market research sector. However, I never expected to find one that so closely matched her new appointment, providing a very interesting, and timely, perspective on the value of establishing an archive in our sector. The ESRC Archive is now simply known as the Data Archive, covering the social sciences and humanities (http://www.data-archive.ac.uk/).

By 1993, the ESRC Archive based at the University of Essex was already 25 years old (founded ten years after the MRS introduced ‘Commentary’, the precursor of JMRS and IJMR), underlining the challenge facing the AMRS to capture material from the earliest days of market research. As Denise describes, the Archive did not own the data, but ‘holds and distributes them under licences held by site owners’. Its aim was to ‘preserve and disseminate machine-readable data’, with a catalogue of 3500 datasets. A key challenge described in the paper, and facing AMSR today, is storing data in such a way that it will remain accessible over time, protected against technological change. However, the description of 8mm video and DAT audio tapes, cartridge tape drives, floppy discs and CD ROM formats demonstrate the difficulties in keeping archives technologically neutral! The way this was achieved at that time is described. And, as the author describes, format is also an issue for users as well as depositors. This naturally leads to discussing the complexity of identifying users, and potential users, needs in terms of applications, outputs and formats. Links with other data libraries was also seen as key to success, complementing the Archive. The development of data documentation, and standards for documentation were described as a future focus, harnessing the opportunities from the then latest developments in machine-readable technologies.

In the paper, the author describes where the demand for the Archive came from, and how this affected how the Archive had developed. The social sciences were rapidly changing in that era, with a greater emphasis on quantified data. Today, the evolution in social research is driven by the big data agenda and integrating disparate sources to create a more holistic picture (see Editorial in 60/3, ‘Wither the (social) survey?’). An interesting comment is the aim to encourage re-usage of existing datasets.

Another important point discussed in the paper is the increasing recognition that fostering relationships between users and producers of data is of immense value in encouraging data sharing, and ways to encourage this are described. However, this is sometimes undermined by ‘sovereignty’ and the need to generate revenue via new public-sector agencies. Also, Denise describes the increasing financial pressure on the Archive, especially from market research companies, to bear the additional costs of archiving data, costs which had previously been borne by the data owner – leading to the need to establish a budget for data acquisition. However, research councils and the Higher Education Funding Councils were starting to recognise that creating an archive that promotes re-use of data facilitates a good return on the initial investment, but only if there is also sufficient investment in managing an archive and promoting the benefits.

However, persuading owners to provide data at low, or no cost, means also addressing any concerns they have about public archives. A key concern described is confidentiality, and the need to constantly reassess society’s views on this – well before the increasing constraints imposed by ever more proscriptive privacy legislation, but GDPR does include concessions that recognise the needs of the research sector.

Overall, the paper reminds us of the challenges faced in developing an effective archive, as AMSR know only too well, but it also demonstrates the value to future generations of both establishing an archive, and ensuring it is built on firm foundations. The AMSR site already contains a wealth of interesting and useful material, and I recommend a visit to see what is there.