Innogen  ·  Publications  ·  Reports
ESRC evaluation report on Innogen and the ESRC Genomics Network (2002-2014)

ESRC

January 2016

The work of the ESRC Innogen Centre from 2002 to 2014 was evaluated by the ESRC in 2014/15.The evaluation was conducted for the whole ESRC Genomics Network (EGN) which consisted of four centres spanning five UK universities:
• The Centre for Economic and Social Aspects of Genomics (Cesagen), Cardiff University and Lancaster University
• The Centre for Genomics in Society (Egenis), University of Exeter
• The Centre for Social and Economic Research on Innovation in Genomics (Innogen), University of Edinburgh and the Open University
• The Genomics Policy and Research Forum, University of Edinburgh.

The parts of the evaluation report that concerned Innogen were sent to us in January 2016. We think they will be of interest to Innogen Centre and Innogen Institute members so the 50 page report has been synthesised into this summary. References to specific research relate to chosen projects and cases sent to the two reviewers on ESRC instructions. This makes the evaluation a bit uneven in coverage but still, it gives a sense of what we all did and was very complementary about our integrated and collegial style. Comments are very welcome and should be sent to David Wield (david.wield@ed.ac.uk or david.wield@open.ac.uk)

The final evaluation report:

The evaluation criteria for the Centres and the Network as a whole were based on: “design and implementation; academic achievement and impact; engagement activity; economic and social impact; capacity building, collaboration; management; legacy; and overall successes and weaknesses.”

Overall, the evaluation was very positive about Innogen:

“Innogen’s comprehensive, coherent and unified programme of policy-oriented research with an important international dimension and an emphasis on genuinely interdisciplinary working demonstrates the Centre’s success in these areas.”
“The Centre’s engagement activity, some of which involved working particular closely with the Genomics Forum, seemed to attract an appropriate range of audiences. The Centre also built on existing research capacity and enhanced the careers of its researchers, and a large number of early career researchers at Innogen have gone on to more senior positions.”

The report mentions that:
• “Innogen was designed appropriately, with activity constructed to achieve the Centre’s aims and objectives”.
• “The focus of the Centre evolved across the two funding phases, initially covering genomics as integrated into the life sciences within the economy and society, then in the second phase focusing on long term overall outcomes of significance, producing both theoretically and empirically grounded research to advance knowledge and inform decision making on life science-based technologies. Emerging technologies during the life of the Centre meant that priorities and focus shifted, most evident in the Centre’s work on the evolution of the bio-economy.”
• “The Centre’s academic staff were brought together from a range of disciplinary backgrounds, and it was the expertise of the Centre’s core members around which the Centre was designed.”

Academic evaluation

The academic evaluator reported that “the toolkits and the book produced by the Innogen Centre set out useful approaches to aid interdisciplinary working, and contain practical guidance on how to work in this way.” and “the Innogen Centre produced a comprehensive, coherent and unified programme of policy-oriented research with an important international dimension and an emphasis on genuinely interdisciplinary working.”
She also stated that “in many ways the Centre was highly successful in achieving this laudable but demanding goal [of carrying out research with the key actors—innovators, policy makers, public and stakeholder groups—not just studying them], particularly in terms of its collaboration in Africa with the International Aids Vaccine Initiative. Innogen collaborated on projects with institutions in around thirty countries and ran over twenty international workshops”.

This section of the report also recognised that:
• “On the UK front, Innogen has been strongly involved with governmental bodies and public engagement activities.”
• “Innogen consciously and explicitly sought to bridge the natural and social sciences through its study of life science innovation.”
• “The independent assessment conducted in 2012, assessing Innogen’s major academic publications on Web of Science, concluded that it had substantially succeeded. Innogen deliberately set out to create a set of research approaches and ‘toolkits’ in order to aid interdisciplinary working and to share skills with the wider research community.”
• “In recognising the need to go beyond merely multidisciplinary working and the difficulties of creating genuinely interdisciplinary models, Innogen deserves considerable credit.”
• “Likewise, Innogen explicitly studied the pressures that have sometimes produced overinflated expectations of the new biotechnologies. Its projects were generally prescient and accurate about new scientific developments.”
• “The quantity of Innogen’s outputs was certainly very impressive and substantial”, although the academic evaluator (Prof Donna Dickenson) felt that some of the outputs selected by Innogen for the evaluation may not have reflected the true international dimension and quality of the research carried out by Innogen.

Impact evaluation

The impact evaluation was consistently positive, mentioning that “an impressive number of impacts outside of academia were reported by the Centre”.

The three impact case studies selected were:

Innogen Case Study 1: Stem Cell regulation in Argentina
• “Innogen’s work on stem cell regulation in Argentina made a key contribution to the development of knowledge about the norms and values driving the formation of new regulatory structures, and helped to overcome a regulatory impasse in stem cell research in that country.”
• “According to the Chair of the Argentinian Advisory Commission on Regenerative Medicine and Cellular Therapies, Fabiana Arzuaga, Innogen’s research ‘served to inform and shape crucial steps in Argentina towards the construction of a legal framework and regulatory policies and action for stem cells research and therapies, which led to a proposal to modify the Civil Code of Argentina’.”

Innogen Case Study 2: Biofuels ethical framework
• “Innogen’s contribution to the UK Nuffield Council on Bioethics’s inquiry into the ethical issues raised by biofuels resulted in a timely report that set out an ethical framework to guide policy-making for biofuels and attracted the interest of senior policy-makers in the UK and the EU.”
• “The report was widely shared with policy-makers in the EU and the UK, and with the media and wider public; and was recognised as a timely and pertinent contribution by senior UK politicians. The ethical framework in the report also informed subsequent work by the UK Technology Strategy Board on the commercialisation of research on synthetic biology, and were reflected in the ‘Bio-energy principles’ set out in the UK Government’s Bio-energy Strategy in 2012.”
• “The Secretary of State for Environment, Food and Rural Affairs, Caroline Spelman, noted that ‘I was impressed by the thorough and well researched recommendations, which came at a good time to inform a number of policy reviews, including the development of a new Government strategy on sustainable bio-energy.’ “

Innogen Case Study 3: Targeted R&D Policy (TARGET): Regional innovation and economic growth
• “Innogen researchers developing guidance for targeted R&D policies to promote the biotech sector in Europe helped to overcome a major barrier to effective biotechnology policy-making in Lithuania by creating opportunities for constructive discussion between two key Ministries and recommending the creation of an independent steering committee.”
• “New ideas developed with Lithuanian colleagues were tested with policy makers and senior people from the biotech industries during a series of interdisciplinary cross-sector meetings. These actors became increasingly interested in applying these ideas in practice, and one early result was a new locus for on-going discussion between the two Ministries and the Agency for Science, Innovation and Technology. The project also produced a set of guidelines that can be used to help countries/regions develop and implement targeted innovation policies, and the co-investigators have since contributed to the development of related OECD work on smart specialisation.”

Other impacts mentioned include the following:
• “Innogen’s research is reported to have led to a policy brief for the International Risk Governance Council, which was used by the US Presidential Commission for the Study of Bioethical Issues in preparing its recommendations for President Obama on the ethics of synthetic biology and emerging technologies.”
• “The Centre was reported to have had direct impact on the health sectors and on innovation policy in Africa, leading to requests for further policy-oriented research, the inclusion of research results into policy and practitioner publications, and direct implementation into policy.”
• “Innogen’s research on regulation and governance of the new life sciences was reported to have led to a series of systematic frameworks and toolkits, two of which have been highly used with government and industry collaborations.”

“User survey respondents noted the policy impacts to be the most significant of the Centre’s engagement achievements, and also that these were enabled by interdisciplinary expertise and existing contacts”.

Capacity-building

The report acknowledged that “Innogen also aimed to build on existing research capacity and enhance the careers of researchers.”

• “Staff at the Centre noted the large number of early career researchers at Innogen who had gone on to more senior positions, and the Centre’s funding model (10 postdoctoral researchers funded at 50 per cent instead of five researchers at 100 per cent) facilitated the development of a larger cohort of researchers.”
• There was a general agreement among the staff survey respondents that “there were appropriate skills and knowledge development opportunities for the PhD researchers at Innogen, [..] that there were appropriate skills and knowledge development opportunities for the post-doctoral researchers at the Centre, and [...] that the Centre positively impacted on the career destinations of researchers and PhD students.”
• The report further noted that “the Centre had built a cohort of researchers working on life sciences and development issues. Staff survey respondents were also positive about the environment at the Centre, and about the encouragement and mentoring they were given.”
• “The Centre’s model for employing and supporting Postdoctoral researchers [...] helped build capacity. Each of the Postdoctoral researchers supported through this method have since gone on to become Principal Investigators of other projects.”
• “The Centre offered monthly training activities and workshops (both prior to and since the introduction of the Scottish DTC), and encouraged its PhD students to take part in the Genomics Postgraduate Forum. Innogen researchers made various contributions to the training delivered through the ESRC Scottish Doctoral Training Centre, and also taught on Postgraduate Programmes at both institutions. Innogen researchers, drawing upon their experience and skills in interdisciplinary research, ran a series of ‘interdisciplinary masterclasses’ which were offered to the wider academic community”.
• “The Centre ran a series of professional development courses for managers in industry and government policy makers, some of which worked in national health systems in East Africa.”

On the Centre’s management

Accounts collected on the Centre’s management were highly positive:
• “In particular, the ESRC’s Committee Liaison Member for the EGN, who was in post during the Centre’s second phase, spoke highly of the Centre Director’s performance.”
• “There was a general agreement from staff survey respondents [..] that the Centre was successfully managed.”
• “There were a number of noteworthy achievements that are testament to the successful management of the Centre. The successful integration of the Centre’s two sites, aided by regular joint meetings and collaborative activity, both directors holding posts in both universities, and a group of permanent staff from each university given visiting appointments at the other university, is something that not all research Centres manage to achieve.”
• “The way in which the Centre’s management reacted to the Mid-Term Review recommendations (which included steering the Centre towards ensuring that there were clear strategic research priorities in the second phase, and that the Centre’s resources were not spread too thinly) by realigning its focus was further evidence of successful management.”

Conclusions

“Three of the Centre’s objectives concerned scientific achievement, and Innogen’s comprehensive, coherent and unified programme of policy-oriented research with an important international dimension and an emphasis on genuinely interdisciplinary working demonstrates the Centre’s success in these areas.”

“Although priorities shifted during the life of the Centre due to new emerging technologies, the Centre reacted well to this, evolving where necessary”.

“Two of the Centre’s objectives concerned its engagement activity. This was considerable and included seminars; workshops; policy briefings; policy, advisory and stakeholder group membership; advice and consultancy, and media reports. The Centre’s objectives for engagement included integrating their activity with the rest of the EGN.”

“Innogen was well-managed, evident through achievements such as successful split site working and the impressive volume of additional funding that was secured by the Centre. Those at the Centre were all confident that Innogen [...] had either met or exceeded all of its objectives.”

“The [Innogen] Centre has now rebranded itself [as the Innogen Institute] and secured funding from a number of other sources to ensure that it continues”.