Tag Archives: Evaluations Online

Shining a spotlight on Evaluations Online: Scotland’s essential economic development resource

Posted on by

Image: Marcus Winkler (via Unsplash CC)

The UK is currently at the beginning of what is expected to be the deepest recession in living memory. From a policy point of view, governments around the world are facing the daunting task of navigating a route through uncharted territory. As the recently launched cross-institutional Economics Observatory noted last month, “sound and non-partisan advice is needed to inform decision-makers across all parts of society, about the choices they face in dealing with the crisis and the recovery”.

Key role of economic development and sustainability in the Covid-19 recovery

As statistical analysis suggests that Scotland’s GDP fell 18.9% during the month of April, and that in May output remains 22.1% below the level in February, the need for a recovery approach that is based on empowering regions, cities and local communities is clear.

The independent Advisory Group established by the Scottish Government to advise on Scotland’s economic recovery in the wake of the COVID-19 pandemic, published its report at the end of June. This identified interventions to support Scotland’s economic recovery within the context of the strategic goal of shifting to a greener, fairer and more inclusive economy with wellbeing at its heart.

New economic development initiatives and programmes in response to the pandemic have already been launched in Scotland. Some are focusing on helping specific sectors such as tourism and the creative industries. There is also a recognition that it is important during the recovery to build on current strengths, such as inward investment and low-carbon technologies.

What works in economic development

Here at the Knowledge Exchange, we’re committed to supporting the use of evidence to inform policy development and practice. So in the light of the current importance of economic development, we thought we’d highlight a useful resource which makes available the results of evaluation work and research in order to enhance decision-making and investment in the future.

Evaluations Online is a public portal providing access to a collection of evaluation and economic development research reports commissioned by Scottish Enterprise, Scotland’s main economic development agency.

Ensuring that public investment generates economic and social benefits, and long-term inclusive growth for Scotland is core to Scottish Enterprise’s remit. Making evaluation and research reports publicly available supports this aim, as well as ensuring transparency.

Established for over a decade, the site now contains over 750 research and evaluation reports dealing with different aspects of economic development activity, such as business support, investment, sector growth and improving skills. All of the reports are publicly accessible and free to access.

Learning lessons from previous programmes

Developing the economic response to Covid-19 is happening at a much faster pace than usual policy-making cycles. It is important, though, that spending and investment is focused on areas that will have most impact, and will also contribute to the overall goals of supporting jobs, protecting and progressing education and skills, and tackling inequality. Considering lessons from previous interventions when commissioning new projects or allocating funding, is one way to address effectiveness.

It’s worth repeating that repositories of evidence can help bring about better policy in a number of ways:

  • improving accountability by making it easier for people to scrutinise the activities and spending of public sector organisations;
  • improving the visibility and therefore the impact of evidence;
  • helping identify gaps in evidence by making it easier to compare research findings; and
  • increasing our understanding of what works (‘good practice’), not only in the activities covered, but also in evaluation and research methods.

Evaluations Online offers resources in key areas such as entrepreneurship, regeneration, social enterprise, economic inclusion, skills development, financing, inward investment and commercialisation, as well as by sector. In recent years, questions about inclusive growth and generating social value have also become more important policy issues.

Some of the most popular recent reports added to the site have focused on:

It’s clear that there are huge sectoral and regional challenges within the economy which will need faced immediately and in the longer term, as a result of Covid-19. Business practices have changed, as have all our lives. But we believe that the use of evidence and research will be fundamental in successful recovery and the transition towards a greener, net-zero and wellbeing economy.

The Knowledge Exchange work with Scottish Enterprise to manage the Evaluations Online portal.

Evaluations Online is a publicly accessible collection of evaluation and research reports from Scottish Enterprise. The reports cover all aspects of Scottish Enterprise’s economic development activities and are available for download at no cost.

A world of evidence … but can we trust that it is any good?

Posted on by

What is good evidence? And how can policymakers and decisionmakers decide what is working and what isn’t, when it comes to deciding where public money is spent and how?

These are the kinds of questions that models and tools such as randomised controlled trials and cost-benefit analysis attempt to answer. The government has also supported the development over the last five years of the What Works Network, which now consists of 10 independent What Works Centres. When talking about impact there’s also been a move to capturing and recognising the value of qualitative data.

As one of our key aims is to support and facilitate the sharing and use of evidence in the public sector, we were interested to read a new publication ‘Mapping the standards of evidence used in UK social policy’.

Standards of evidence

Produced by the Alliance for Useful Evidence, the research has found 18 different Standards of Evidence currently in use across UK social policy.

The report notes that over the last decade there has been increasing interest in grading effectiveness or impact against a level or scale. Typically, the higher up the scale, the more evidence is available. Theoretically this means that decision-makers can have higher confidence in deciding whether a policy or intervention is working.

While all the evidence frameworks generally aim to improve the use of evidence, the different goals of the organisations responsible can shape the frameworks in different ways. They can be used to inform funding decisions, to make recommendations to the wider sector about what works and what doesn’t, or as a resource to help providers to evaluate. And unfortunately this means that the same intervention can be assessed differently depending on which framework is used.

The Alliance for Useful Evidence concludes that while a focus on evidence use is positive, the diversity of evidence standards risks creating confusion. Suggested options for improving the situation include introducing an independent accreditation system, or having a one-stop shop which would make it easier to compare ratings of interventions.

Dissemination and wider engagement

The question of standardising evidence frameworks is just one part of a wider effort to increase transparency. As well as collecting evidence, it’s important that when public money has been invested in carrying out evaluations and impact assessments, that this evidence remain accessible over the longer term and that lessons are learned. It can often seem that government departments have very short organisational memories – especially if they’ve suffered a high churn of staff.

Two projects which we support in Scotland are focused on increasing the dissemination and awareness of evaluation and research evidence. Research Online is Scotland’s labour market information hub. Produced by ourselves and Skills Development Scotland, the portal brings together a range of statistics and research and acts as the centre of a community of practice for labour market researchers, practitioners and policy-makers.

Meanwhile Evaluations Online is a publicly accessible collection of evaluation and research reports from Scottish Enterprise. The reports cover all aspects of Scottish Enterprise’s economic development activities – some of the latest added to the site cover megatrends affecting Scottish tourism, innovation systems and the gender gap, and the commercial flower-growing sector in Scotland.

When working within the policy world it can be easy to suffer from fatigue as ideas appear to be continually recycled, rejected and then revisited as policy fashions change and political parties or factions go in and out of power. The spotlight, often driven by the media, will shine on one hot policy issue – for example, moped crime, cannabis legislation or health spending – and then move on.

Online libraries of evaluations and research reports are one tool which can help support a longer-term culture of learning and improvement within the public sector.

Evidence Week 2018

Inspired by similar objectives, Evidence Week runs from 25th to 28th June 2018 and aims to explore the work of parliamentarians in seeking and scrutinising evidence. It will bring together MPs, peers, parliamentary services and the public to talk about why evidence matters, and how to use and improve research evidence.

This may be the start of wider knowledge sharing about standards of evidence, to help those using them to improve their practice.

The Knowledge Exchange is a member of the Alliance for Useful Evidence. Our databases are used by government and the public sector, as well as private-sector consultancies, to keep abreast of policy news and research in social and public policy.

Focus on: Evaluations Online

Posted on by

 

Evaluations Online is a public portal providing access to a collection of evaluation and economic development research reports commissioned by Scottish Enterprise, Scotland’s main economic development agency.

Ensuring that public investment generates economic and social benefits, and long-term inclusive growth for Scotland is core to Scottish Enterprise’s remit. Making evaluation and research reports publicly available, supports this aim as well as ensuring transparency.

Some of the most popular recent reports added to the site have focused on:

Working in partnership

Since 2007, Idox has been working with Scottish Enterprise to deliver Evaluations Online using a publishing platform designed specifically to deal with research material. Users can easily navigate to and assess the relevance of material thanks to specially-written abstracts and structured search functions based on a bespoke classification and record structure.

The site now contains over 600 evaluation and research reports commissioned by Scottish Enterprise, dealing with different aspects of economic development activity such as business support, investment, sector growth and improving skills. All of the reports are publicly accessible and free to access.

Since the site launched we have continued to refresh and improve the site, ensuring it better meets the needs of key user groups, including economic development policy-makers and practitioners across Scotland. In the last quarter of 2016, the reports hosted on the site were accessed over 30,000 times.

The importance of evaluation

We’ve highlighted the importance of evidence and evaluation and assessment of information quality on the blog several times before. It’s worth repeating that repositories of evidence can help bring about better policy in a number of ways:

  • improving accountability by making it easier for people to scrutinise the activities and spending of public sector organisations – this helps organisations meet Freedom of Information responsibilities;
  • improving the visibility and therefore the impact of evidence;
  • helping identify gaps in evidence by making it easier to compare research findings; and
  • increasing our understanding of what works (‘good practice’), not only in the activities covered, but also in evaluation and research methods.

We’re proud to support Scottish Enterprise in the dissemination of their evaluation and research output, through a portal which they believe increases the return on these activities.

You can find out more about the projects The Knowledge Exchange team has been involved in, and the consultancy services we offer, here.