The Data Documentation Initiative: making metadata reusable
19 November 2014

In a recent blog, I focussed on a presentation by Mike Page (Blueocean) at September’s ASC conference. Page discussed the reasons why market research data remains outside corporate data warehouses. One key point he mentioned was the lack of consistency in definitions used within questionnaires, for example in recording standard demographics.

I’ve now discovered the Data Documentation Initiative, which has very recently issued a call for action to develop metadata standards for questionnaire and data description – ‘Survey Metadata Reusability and Exchange: A Call for Action for Questionnaire Documentation’ (05-11-14): 

‘This collaborative statement calls upon the survey design, production and archiving communities to take leadership in facilitating survey metadata exchange through adoption of shared metadata standards for questionnaire and data description’.

In fact, this statement follows on from a meeting held at the RSS in London on June 28th, ‘Survey Metadata: Barriers and Opportunities’, that brought together survey owners, producers, computer assisted interviewing suppliers and archives to discuss the challenges of working in a more harmonised way to avoid the high cost of manually producing high quality data documentation.

I’m sure that some readers are already aware of the DDI, as their standards are already used internationally for survey data archives. However, I suspect that social researchers, and those working in the public sector, might be more aware of this than many commercial market researchers – especially those in client organisations.

The recent statement also includes three key recommendations for the stakeholder organisations in the research process invited to the initial meeting: 

Questionnaire design and CAI systems that embody a commitment to survey metadata re-use:

• Can accommodate description of variables in full context, including information such as full question text, universe and routing, showing information, and so on
• Offer import and export of metadata in the recommended DD13.2 Questionnaire profile in XML format
• Agree to keep exchange mechanisms up to date with future upgrades to the DD13.2 Questionnaire profile

My main concern is whether client-side researchers will be taking any interest in DDI, maybe discussing this with their external agencies, and internal IT department, to see if this helps address the issues raised by Mike Page, thereby making survey data eligible for an invite to the big data party.

For more details see the ‘Survey Metadata Reusability and Exchange: A Call to Action for Questionnaire Documentation’