Brian Burgoon is Professor of International and Comparative Political Economy at the Faculty of Social and Behavioural Sciences and Director of the Amsterdam Institute for Social Science Research (AISSR).
‘When I took on the role of director in 2014 I immediately started the discussion on Research Data Management (RDM). RDM is very important to our integrity policy. The standards and ideas about RDM vary widely within our research institute. This is because we have an extremely diverse set of research areas. AISSR consists of four departments: Anthropology; Political Science; Sociology; and Human Geography, Planning and International Development Studies. This shows you the full range of methodologies and epistemologies. This makes it hard to create a single RDM policy.’
Safeguarding integrity and validity
‘For our RDM policy I focus on ensuring the integrity and validity of research. What does this require? We have almost completed the AISSR data protocol. It still needs some slight modifications and approval from the AISSR Programme Council. The protocol currently has a provisional status.’
‘The RDM discussions have resulted in a number of standards. The most important thing is that our researchers must make their research data, as well as the codes for processing and analytical methods, available for replication as much as possible. They must be explicit regarding which data they are willing and able to show to fellow researchers and the public after their research has been published. When registering published research they must fill out a simple form which summarises their decisions and reasoning about replication. We will check whether this is being done before conducting the annual consultations.'
‘For our quantitative researchers who, for example, conduct experimental lab research or observational quantitative work with large existing datasets, replication is – generally – not a problem. They can make their data available for replication, in an anonymous format where necessary. This can be done via their website or in a journal, for instance.
But we also have qualitative researchers who collect empirical data which are difficult to store and/or who work on an entirely theoretical or hypothetical basis. This includes anthropologists who conduct research through participant observation and write down their experiences in a diary. Understandably, they are often sceptical about replication and may disagree with it on ethical grounds. This raises the question of how you, as a researcher, can continue to guarantee your respondents' confidentiality. They find that the requirement to share data for replication does not respect their type of research and interpretations. I encourage them to share whatever they have on paper, and if disclosure is not possible, then they should explain why not.’
Most social scientists embrace replication. While they endorse the RDM policy, it is unfortunately still only observed by a minority. This means there is still more work to be done. Soon, each published article must state where the data can be found, preferably via a persistent identifier. A culture shift is still needed among quantitative as well as qualitative researchers. Above all, we have to make RDM easy and inexpensive in order to get everyone on board.
As a member of the RDM steering group for the development of the central RDM system, I am the voice of the workplace and I continue to stress: as safe, easy and inexpensive as possible. One of my pet subjects is that a good link must be established between the CRIS (Pure) and the central RDM system. In carrying out the RDM mission, I see that it is difficult to develop a safe and affordable RDM system. That is why I believe we need a multifaceted data policy.
At the moment – and also soon, once the RDM system is in place – our researchers can choose where to save their data: in Amazon, DANS, on the FMG server, in the central RDM system or elsewhere. Of course, strict conditions apply to personally identifiable data. The central RDM system is interesting for us with respect to storage of confidential data, but this is not obligatory. Researchers may make their own decision, as long as it is safe and they can properly substantiate their choice. Even once the protocol is complete, we will continue to discuss RDM. We will share best and worst practices, and I expect that many questions will also arise in practice.’