Standards of Evidence
The Nesta Standards of Evidence is used to help us determine how confident we can be that a particular intervention is having a positive impact
Standards of Evidence as an innovation method
Our Standards of Evidence is used to evaluate the evidence created by both our practical innovation programmes and our investments. It lays out five levels of evidence, ranging from early-stage evidence (Level 1) to robust, replicable evidence (Level 5). It helps us to seek the type of evidence that is proportionate to the stage of development of an innovation.
We work with organisations to move up these levels of evidence over time, increasing our confidence that what we are doing is having a credible and measurable impact. We also use the framework as part of assessing the impact risk of investments.
As an innovation method, it also helps structure an evaluation strategy and helps determine future funding decisions that make innovation possible.
Nesta's work on Standards of Evidence
Put simply, innovators, commissioners, service users and investors all need evidence to know whether the products or services they develop, buy or invest in make a positive difference.
At Nesta, good intentions are not enough. Is your innovation doing good - or even harm? Only robust and proportionate evidence will help us answer this question.
In 2012, we developed a framework to analyse the quality of evidence about our investments. This framework created the building blocks of the Nesta Standards of Evidence.
As well as being used to assess our social investment funds since then, the Standards of Evidence have been used in other parts of the organisation. For example, the Centre for Social Action Innovation Fund, a partnership between Nesta and the Cabinet Office, has funded more than 30 evaluations to determine impact and inform further funding decisions. The innovations assessed across different levels can be viewed here.
In 2016, we developed the Using Research Evidence: A Practice Guide to help people learn about evidence-informed decision-making, and why research is an essential element of it.
The Standards of Evidence also continue to be championed by Nesta’s Alliance for Useful Evidence. Founded in 2011, this network champions the smarter use of evidence in social policy and practice.
Outside of Nesta, other innovation bodies, charities, businesses and public bodies have adapted the standards for their own use - these include the Early Intervention Foundation, HACT, Pearson, Office for Fair Access, UK Active, and the Swedish innovation body Vinnova.
‘Our Organisation Makes People Happy!’ is a social enterprise that trains care staff to deliver fun, interactive, group exercise classes within care homes and in the community. Its specialist exercises are based on expert methodology, and use sensory props, reminiscence imagery, music and simple routines to deliver physical and mental health benefits. It aims to improve quality of life of older people by improving their physical mobility, social interaction and mental stimulation. Nesta Impact Investments is a shareholder in Oomph!
Oomph! captures ongoing monitoring information on care homes’ satisfaction as well as observational analysis on attendees’ well–being. It uses this information to report back to clients on the impact of the classes, showing them the return on their investment. It also helps to identify instructors that require extra support and improve its core product. Oomph! have worked closely with Nesta and are committed to working through their standards of evidence.
In 2014 Oomph! began tracking direct beneficiary quality of life outcomes, using a widely used and well–regarded scale called the EQ5D - it measures self–reported health and quality of life. In autumn 2014 Oomph! gave questionnaires to a sample of 50 of its class attendees and then repeated the questionnaire three months later. Results from this survey showed a 10 per cent improvement in quality of life scores.
Oomph! is now repeating a similar survey with a more rigorously controlled sample and are working with the Health Economics department from UCL to analyse the results. Off the back of this study, Oomph! plans to carry out a Level 3 evaluation as set out in the Nesta Standards of Evidence.
Centre for Social Action Innovation Fund
Extra case studies that have used the Nesta Standards of Evidence can be viewed here. This blog ‘Putting out money where the evidence is’ presents a case study about Volition, a social business that runs volunteering programmes in Cathedrals to get people back into work.