2. FP9 ‘s ambitious aims for societal impact call for a step change in interdisciplinarity and citizen engagement#

The European Alliance for SSH welcomes the invitation of the Commission to contribute to the development of the next Framework Programme. In response EASSH has prepared two position papers, with a thematic and a cross-cutting focus respectively. This paper addresses the cross cutting issue. Its starting point is the mid-term review of Horizon 2020, in particular its finding that, although the Societal Challenges multidisciplinary approach offered benefits in comparison with previous funding programmes, there were two unintended effects; the calls appeared to promote projects with little or no innovation and, the evaluation process did not respond adequately to H2020’s ambitions for interdisciplinarity.

The on-going discussions in relation to FP9, including various high-level indications from the Commission, suggest that FP9 will continue to address societal challenges, will identify a number of key missions, will pay increased attention to social impacts and will include innovative approaches to promote citizen engagement and participation.

EASSH greatly welcomes these ideas and trends: the European SSH community, represented by EASSH, has been a strong champion of “real” interdisciplinarity and of concrete and innovative approaches to citizen engagement at various critical stages in the programme and project cycles.

Our concrete proposals are elaborated below, organized into three main groups of actions for FP9- especially its collaborative societal challenge elements. While we believe that each of our proposals will contribute to the above aims we are also convinced that by combining them within an overall strategic framework for interdisciplinarity and engagement FP9 can make an unprecedented contribution to the concerns and aspirations of citizens.

On the basis of our analysis and based on the review of papers submitted for the consultation, EASSH would like to make recommendations in three main areas: radical reform of instruments to support research; programme evaluation and key performance indicators; and dedicated evaluations of interdisciplinary proposals. The latter two sections also address the assessment of social impact. This paper also stresses that fundamental research in relation to the societal challenges has a central role, and should not be ignored but rather nurtured and encouraged.

1. Instruments to Support Multidisciplinary Research: Social Missions and Integrative Platforms#

EASSH encourages the expansion of the range of instruments to support multidisciplinary research in the 9th framework programme. First, we need to look beyond the short-termism of the current 3-year ‘project’ cycle. Many of the problems to be tackled through the missions will require contributions from across research fields and involve a wide variety of stakeholders. EASSH believes that such missions could be considered as being appropriate for longer-term investment in the form of ‘integrative platforms’, which bring together researchers and stakeholders in sustained collaborations. EASSH supports the introduction of ‘integrative research platforms’, which would be supported over 6-8 years, which will partners time to develop approaches to working across disciplines, to learn from early research outcomes and to respond to changing social dynamics in a context which encourages both research and innovation. These platforms will engage with bottom-up issues where there is emerging critical mass to enforce dynamic collaborations of different disciplines and different agents. This is where the real European Added Value for Research lies in sustained investment in key issues.

EASSH reiterates its position that research endeavours should be guided by the nature of the challenges being addressed and not by a limited number of predetermined ‘instruments’. EU-level collaborative research is not always best implemented by large-scale projects with many partners. EASSH believes that supporting more, and smaller, social missions can provide more targeted research insights for both local and European policy makers. Such missions should emerge from research stakeholders and users - such as policy makers, citizens and civil society organisations to provide momentum to address a relevant and pressing issue.

More focused social missions, on the one hand, and integrative platforms, on the other, are complementary approaches to address major research challenges. EASSH hopes that this is the meaning behind the HLG’s recommendations calling for adaptability in choice and the design of funding instruments. In fact, defining instruments and appropriate responses to the social missions should be primarily determined by those engaged in such missions, rather than decided in a rigid, top-down way.

2. Programme Assessment and Key Performance Indicators#

EASSH members have been involved in different projects to review the concept of impact assessments in different programmes. It has now been clearly established that impact cannot be expressed as a linear process. In-depth research has demonstrated that moving ‘from lab to market’ hardly ever occurred in such a simple progression and, more importantly, that it is not just a market ‘product’ that demonstrates achievement, but rather social uptake and scalability which also need to be taken into account.

We propose that assessment of research impact should be looked at on an aggregate level and not in relation to single projects. Assessment should take into account what the call design and the programme were originally intended to achieve. Impact of research, in fact, is not just about ‘fixing problems’ or ‘making things’. It is about generating new knowledge and evidence, and understanding how these are used for developing pathways and measures towards addressing an issue or a challenge, which may be relevant to European society now or in the future. Crucially, this process should also address the assessment of calls and of challenges, which will allow the unintended effects of research developments to be clearly identified.

In a similar view, we need to identify the right key performance indicators to measure whether a programme delivers what it has been designed for. We should not use or re-use indicators where the aims and purpose of a programme are new; we will need to identify appropriate new indicators. In Horizon 2020, the approach of the Societal challenges was innovative, but it continues to be evaluated on the basis of the same key performance indicators used in previous frameworks. This suggests that some of the aims remain consistent across FPs: namely, academic excellence and interplay between public and private research collaboration to produce new knowledge pathways and social and technical innovation. However, if FP9 is to introduce new approaches, like value for society, then we must recognise that we cannot rigidly apply the same sets of indicators. We must review and update these so that they can reflect the multiple dimensions of what the new programme is meant to achieve. For example, some of the current indicators remain far too remote in the attempt to assess whether research funding provided by the EU has a real value added for European citizens. A balanced evaluation must take into consideration academic publications, as they remain important for the dissemination and communication of science, but should be combined with tracking research communications beyond traditional journals into other media channels with a wider readership.

Patents and prototypes, as well as new products and process, are valuable indicators of the relevance of some research, but can tell us little about usage or market exploitation or social acceptance and benefits. Indicators need to address, among other dimensions, citizens’ involvement in the process of acceptance of products, or the reception of social innovation initiatives where entrepreneurs have been able to access freely the information generated within the project. Society does not just benefit from products, but more importantly from a fundamental understanding of the social dynamics that contributes to their success. Finally, we must encourage policy makers to play their part in recognising the influence of research by citing the emerging knowledge that has influenced a policy or an idea, directly or indirectly.

EASSH is working in collaboration with European civil society to facilitate more open access to knowledge on the part of those who can most benefit from funded research, with the expectation that they will then report back on the usefulness of the work they have been able to access and read. EASSH proposes a close collaboration between researchers and users of research for the purpose of generating a better social understanding of the role, importance and impact of research. Programme evaluations need in turn to engage with the research users to understand how projects have influenced a given sector or an area of specific interest.

3. Evaluation for multi and interdisciplinary projects#

In some preliminary analysis conducted by EASSH, we have come to the conclusion that it is questionable whether the current evaluation process for the selection of projects to be funded in H2020 is fit for purpose; whether the Commission has been able to create a pool of experts with the correct blend and depth of expertise; whether the conditions have been created to identify the best multi and interdisciplinary projects to deliver the overall aims of the Societal Challenges in the Horizon 2020 programme. EASSH provides below a set of technical recommendations for a more efficient evaluation of proposals.

For FP9 to provide a successful proposal evaluation process, EASSH proposes that the Advisory Groups that design the aims and purposes of the calls within the projected challenges should also contribute to the establishment of semi-permanent proposal evaluation panels. These must be populated with experts who have the full range of skills and expertise from all sectors of society to assess whether the call intentions are being fulfilled by proposed projects. We also call for greater stability in the membership of such panels, which will learn to work together over time and understand how to reward truly interdisciplinary projects in line with the aims of the calls. A certain degree of coherence and consistency emerges in this process too as those designing the calls can fully ensure that relevant expertise is brought to bear on the evaluation. Self-nominated experts and selection of reviewers on the basis of keywords are not efficient as we demonstrated in a previous paper on Evaluation in H2020 Societal Challenges.

Finally, proposal evaluation must reserve a space to assess - ex-ante - the potential for the impact of each project to be evaluated. This should include a declaration of objectives at the outset (i.e. relating to potential impact) on the part of the research consortium, highlighting how methodologically robust research is combined with a demonstration of relevance in society, via means of consistent engagement with the research subjects or beneficiaries. Alongside the scientific evaluation of projects, teams will also be expected to show evidence of their relevance in their direct or indirect influence over time of the social environment in which they are to be realized.

EASSH is working in collaboration with European civil society to facilitate more open access to knowledge on the part of those who can most benefit from funded research, with the expectation that they will then report back on the usefulness of the work they have been able to access and read. EASSH proposes a close collaboration between researchers and users of research for the purpose of generating a better social understanding of the role, importance and impact of research. Programme evaluations need in turn to capture from the research users how projects have influenced the sector or the area of specific interest. At the same time, the relevance of research must also be assessed according to the extent to which research has provided a fertile environment for ideas to develop and evolve, even through negative findings or the identification of unproductive pathways for future work.

Download the document