Tag Archives: what works network

Changing government, changing society: what now for public innovation?

Posted on by

Image: CC Ross Findon via Unsplash

States of Change is an independent, global non-profit organisation which focuses on supporting and promoting learning around public innovation. During June, they hosted an online festival, bringing together practitioners, academics and policymakers from around the world in conversation. The recordings of all the sessions are still available online and in this blog we reflect on some of the discussions.

In the UK, we’ve seen over the last decade the growth and mainstreaming of government-supported initiatives, such as the network of What Works centres, and innovation centres such as Y Lab in Wales.

We’ve also seen the high profile introduction and use of agile methods and service design approaches, such as in the Government Digital Service and Gov.UK.

The question remains, however, how to build capacity for innovation at all levels of government and the public sector. To be successful this requires public sector managers and employees to be supported to develop the skills, mindset and culture that enables innovation.

We’ve written before on this blog about the transformational approach to public sector innovation in countries such as Singapore and Estonia. And during the States of Change Learning Festival there were some great discussions on international experiences. Of particular note, given the current UK Government’s stated desire to reform the civil service, was a webinar on How Singapore Learns, which explored how the Singapore government had, over a number of years, developed capabilities for forward thinking, openness to new ideas and agility.

From ideas to practice

Charles Landry, famous for developing the concept of the creative city, participated in a discussion during the festival on what being creative at an organisational or system-wide level actually means. He noted that what the current pandemic has done to generate urgency and a sense of focus within public sector systems, which may have included toying with small tweaks to bureaucracies, or taking an innovation approach built on numerous small-scale pilots or projects.

There were also sessions looking at tools and methodologies for public innovation, such as behavioural science and public innovation labs. And a discussion between Aarathi Krishnan and Panthea Lee highlighted the need for those working in government to recognise their own privilege. Questions such as “whose voices are we hearing and who is out in the cold?” and “who’s in the room when we ‘imagine’ new futures and how are we making the future just and equitable?” may seem challenging but they are necessary in order to build a different and better future.

How governments think

Another thought-provoking session was a conversation between Geoff Mulgan (Professor of Collective Intelligence, Public Policy and Social Innovation) and Aaron Maniam (Deputy Secretary in the Singapore Government) on how governments “think”. Mulgan highlighted two challenges for government that have been brought to the fore during the pandemic. Firstly, there is a need for new skills and transparent communication about the use of competing and often contradictory knowledge in decision making. Secondly, there’s a need for a new ecosystem of data governance, which would offer protection and public trust about the use of personal data by government. Interestingly, the session also highlighted the differing use of metaphors and language to describe the role of government. How ‘government’ is talked about shapes our view on what it could and should do. For example, is government a brain, or a war machine, or a facilitator, or a steward for future generations?

Evolving public innovation

The COVID crisis has forced rapid advances in how governments observe, make sense of patterns, use models and plan big interventions. They’ve had to make use of data of all kinds, draw on complex scientific advice, and mobilise local and national systems – all while maintaining public trust and compliance.

As we move into the recovery planning phase, there is now a need for medium-term and longer-term visions on how every policy area could be different in the future, from health to social care to education to transport to urban planning. This use of the imagination has traditionally been seen as the specialist realm of futurists and horizon scanners. It also requires strong storytelling, to create confidence and acceptance of change. Whether governments around the world are able to leverage new visions, will determine whether the rapid changes we have seen in response to COVID-19 will lead to genuine positive transformations or ultimately just a return to the status quo.

Enjoy this article? Read more from the Knowledge Exchange blog on public sector innovation …

And follow us on Twitter to see which topics are interesting our research team.

What works now: how can we use evidence more effectively in policymaking?

Posted on by

Evidence use in policymaking is nothing new. It has been talked about by policymakers, academics and professionals for the best part of ten years, and has been highlighted a lot, among other places, on this blog. Over the years various government initiatives have been set up to try to establish how best to use evidence and identify “what works” in relation to specific policy interventions, and “evidence-based” policymaking has become the catchphrase of policymakers across most sectors.

One of the newest books to be added to the Idox Information Service library reflects on the rise of “what works” as an approach to policy development. The book builds on discussions from the first edition of the book, and provides a sector-by-sector breakdown of how evidence is – and could be – used in policymaking across areas like health, the environment, education and criminal justice. It also offers some insight into appraising evidence and how to assess quality, as well as how evidence is used internationally, providing examples from the USA, Australia, New Zealand and Scandinavia.

As one of our key aims is to support and facilitate the sharing and use of evidence in the public sector, this book has been a welcome addition to our collection.

Making use of research across policy

In 2013, the UK government launched the What Works Network, which is now made up of 10 independent centres committed to “supporting the creation, supply and use of evidence” in specific policy areas including crime and policing, education and economic growth. The centres aim to improve the way government and other organisations create, share and use (or ‘generate, translate and adopt’) high-quality evidence for decision-making. According to the UK government, the initiative is the first time a government has taken a national approach to prioritising the use of evidence in decision making.

What Works Now? highlights research from Weiss (1979) which suggests that there are “7 types of research use”:

  • Knowledge Driven – research will be developed, applied and used once it has been produced
  • Problem Solving – research will be applied directly to a particular policy problem in order to solve it
  • Interactive – research forms part of a wider web of knowledge, policy and other research which all interact with each other
  • Political – research could (and probably will) be used to retrospectively provide support for a policy decision which has already been made
  • Tactical – research can be used as a tool to delay or deflect from decision making or action around a particular issue (i.e. “more research is needed in this area”)
  • Enlightenment – research informs policy through encouraging people to think and discuss particular ideas or concepts in a different way
  • Embedded research – research production is embedded in a wider contextual system which includes political priorities, the law and the media

Building a research base to support “what works”

Creating and disseminating research effectively have been cited as being key to creating a “what works” evidence base. A number of research institutes and think tanks contribute alongside real-life experiences of practitioners and other stakeholders to try and establish the conditions which support effective interventions and lead to positive policy outcomes.

One of the big discussions currently is around the creation of academic research to support what works programmes. Exploring what sort of research is useful to practitioners and policymakers and aligning this with the research agenda of academics and universities can help to create an effective supply chain of evidence to inform policymaking. However, often academics often do not engage with the policy process, or politicians politicise evidence, picking and choosing which findings to take notice of, which can distort the perception of what evidence is available in a particular area.

Encouraging fuller participation and a more robust appraisal of research from across the board is something which many institutions are trying to work towards. Research impact and knowledge exchange is now integrated into research funding and a growing number of people are working to feed research more effectively into the policy arena.

Evaluating research and evidence and judging which to take forward to inform policy decision making is also important. Along with discussions around assessing and labelling evidence the book considers how some of the main organisations in the UK concerned with promoting evidence-informed policy have gone about appraising evidence, weighing it up, assessing quality and “fitness for purpose” and taking account of non-research based forms of knowledge and evidence, such as the personal experience of practitioners.

Applying “what works” in practice

Applying “what works” in practice can be a challenge, especially in a setting that is perhaps very different from the conditions of a study that has been shown to produce successful outcomes from a particular intervention.

In the book, 10 guiding principles to support the use of evidence in practice are set out:

  • Translated – To be used research must be adapted and reconstructed to fit with local contexts. Simply providing findings is not enough
  • Ownership – Ownership of the research and allowing people to feel a sense of ownership over the development of research
  • Enthusiasts – Individual “champions” can be useful in ensuring that research actually gets used
  • Local context – Local context must be taken into account, particularly in relation to specific barriers and enablers which might help or hinder change
  • Credibility – Credibility of researchers and the people who support the research is key to ensuring that the research is taken seriously
  • Leadership – Strong leadership provides motivation, authority and integrity in the implementation of evidence
  • Support to implement change – Ongoing support to implement change is important, this could include financial, technical, organisational or emotional support
  • Develop Integration – Activities need to be able to be integrated with existing organisational systems and practices, changes do not happen within a bubble
  • Engage key stakeholders – To ensure effective uptake and buy-in key stakeholders should be involved as fully as possible form the earliest possible stage
  • Capture learning/ Effective evaluation – Don’t forget the importance of evaluation, identify what worked and what didn’t to help share learning and support future projects

Final thoughts

In theory, using evidence to inform policy sounds straightforward. The reality can be quite different. What Works Now? highlights that the “what works” agenda remains dominant across the policy landscape, even if the application or approach to it differs from policy area to policy area.

What counts as evidence is still disputed; getting evidence “out there” and encouraging academics to be involved in the policy process is still hard to achieve (although there is good work being done in this area to try and combat this); and context is still key to making evidence work in a particular environment.

Understanding evidence, and how to use it effectively has been a core aim of policymakers in the UK, and across the world for the many years. This book, and the supporting research outlined in it highlights that while evidence is still at the fore of policymaking, actually identifying what works and putting it into practice is a bit more of a challenge.

Members of the Idox Information Service can log into our website to request a loan of “What works Now?”

If you enjoyed this post, you may also be interested in:

A world of evidence … but can we trust that it is any good?

Follow us on Twitter to find out what topics are interesting our research team.

A world of evidence … but can we trust that it is any good?

Posted on by

What is good evidence? And how can policymakers and decisionmakers decide what is working and what isn’t, when it comes to deciding where public money is spent and how?

These are the kinds of questions that models and tools such as randomised controlled trials and cost-benefit analysis attempt to answer. The government has also supported the development over the last five years of the What Works Network, which now consists of 10 independent What Works Centres. When talking about impact there’s also been a move to capturing and recognising the value of qualitative data.

As one of our key aims is to support and facilitate the sharing and use of evidence in the public sector, we were interested to read a new publication ‘Mapping the standards of evidence used in UK social policy’.

Standards of evidence

Produced by the Alliance for Useful Evidence, the research has found 18 different Standards of Evidence currently in use across UK social policy.

The report notes that over the last decade there has been increasing interest in grading effectiveness or impact against a level or scale. Typically, the higher up the scale, the more evidence is available. Theoretically this means that decision-makers can have higher confidence in deciding whether a policy or intervention is working.

While all the evidence frameworks generally aim to improve the use of evidence, the different goals of the organisations responsible can shape the frameworks in different ways. They can be used to inform funding decisions, to make recommendations to the wider sector about what works and what doesn’t, or as a resource to help providers to evaluate. And unfortunately this means that the same intervention can be assessed differently depending on which framework is used.

The Alliance for Useful Evidence concludes that while a focus on evidence use is positive, the diversity of evidence standards risks creating confusion. Suggested options for improving the situation include introducing an independent accreditation system, or having a one-stop shop which would make it easier to compare ratings of interventions.

Dissemination and wider engagement

The question of standardising evidence frameworks is just one part of a wider effort to increase transparency. As well as collecting evidence, it’s important that when public money has been invested in carrying out evaluations and impact assessments, that this evidence remain accessible over the longer term and that lessons are learned. It can often seem that government departments have very short organisational memories – especially if they’ve suffered a high churn of staff.

Two projects which we support in Scotland are focused on increasing the dissemination and awareness of evaluation and research evidence. Research Online is Scotland’s labour market information hub. Produced by ourselves and Skills Development Scotland, the portal brings together a range of statistics and research and acts as the centre of a community of practice for labour market researchers, practitioners and policy-makers.

Meanwhile Evaluations Online is a publicly accessible collection of evaluation and research reports from Scottish Enterprise. The reports cover all aspects of Scottish Enterprise’s economic development activities – some of the latest added to the site cover megatrends affecting Scottish tourism, innovation systems and the gender gap, and the commercial flower-growing sector in Scotland.

When working within the policy world it can be easy to suffer from fatigue as ideas appear to be continually recycled, rejected and then revisited as policy fashions change and political parties or factions go in and out of power. The spotlight, often driven by the media, will shine on one hot policy issue – for example, moped crime, cannabis legislation or health spending – and then move on.

Online libraries of evaluations and research reports are one tool which can help support a longer-term culture of learning and improvement within the public sector.

Evidence Week 2018

Inspired by similar objectives, Evidence Week runs from 25th to 28th June 2018 and aims to explore the work of parliamentarians in seeking and scrutinising evidence. It will bring together MPs, peers, parliamentary services and the public to talk about why evidence matters, and how to use and improve research evidence.

This may be the start of wider knowledge sharing about standards of evidence, to help those using them to improve their practice.

The Knowledge Exchange is a member of the Alliance for Useful Evidence. Our databases are used by government and the public sector, as well as private-sector consultancies, to keep abreast of policy news and research in social and public policy.

Why UK-sourced evidence matters … and why it is so often ignored

Posted on by

By Morwen Johnson

If you follow our blog, you’ll know that we care passionately about promoting the uptake of evidence and research by policymakers and practitioners. It’s easy to be complacent and assume that when public money is at stake, decisions are made on the basis of evaluations and reviews. Unfortunately, this is still not always the case.

The current evidence-based policy agenda in the UK encompasses initiatives such as the What Works network, the Local Government Knowledge Navigators and independent organisations such as the Alliance for Useful Evidence. They are working on fostering demand for evidence, as well as linking up academics with those in the public sector to ensure that the research community is responsive to the needs of those making decisions and designing/delivering services.

A recent article in Health Information and Libraries Journal highlights another challenge in evidence-based policy however. A mapping exercise has found that literature reviews often ignore specialist databases, in favour of the large, well-known databases produced by major commercial publishers. Within the health and social care field (the focus of the article), literature reviews tend to use databases such as Medline, Embase and Cinahl – and overlook independent UK-produced databases, even when they are more relevant to the research question.

Why does it matter?

Research has shown that how (and why) databases are chosen for literature searching can “dramatically influence the research upon which reviews, and, in particular, systematic review, rely upon to create their evidence base”.

To generate useful evidence for the UK context (relating to UK policy issues or populations), researchers need to understand the most appropriate database to search – but unfortunately our own experience of looking at the detail of methodologies in evidence reviews, suggests that in many cases the only databases searched are those produced by American or international publishers.

Grey literature is a valuable source in evidence reviews – and again this is often overlooked in the major databases which tend to focus only on peer-reviewed journal content. A recent Australian report ‘Where is the evidence?‘ argued that grey literature is a key part of the evidence base and is valuable for public policy, because it addresses the perspectives of different stakeholder groups, tracks changes in policy and implementation, and supports knowledge exchange between sectors (academic, government and third sector).

Another benefit of UK-produced databases is that they will make use of UK terminology in abstracts and keywords.

Social Policy and Practice – a unique resource

At this point I should declare a vested interest – The Knowledge Exchange is a member of a UK consortium which produces the Social Policy and Practice (SPP) database. The SPP database was created in 2005 after five UK organisations, each with a library focused on sharing knowledge in community health and social care, agreed to merge their individual content in order to make it available to the widest possible audience.

The current members of the SPP consortium – the National Children’s Bureau, the Idox Knowledge Exchange, the Centre for Policy on Ageing and the Social Care Institute for Excellence – have just been joined by the National Society for the Prevention of Cruelty to Children. Inclusion of the NSPCC’s bibliographic data greatly enhances the coverage of child protection research in the database. SPP has been identified by NICE, the National Institute for Health and Care Excellence, as a key resource for those involved in research into health and social care.

We want the UK research community to understand what SPP offers, and to use it when undertaking literature reviews or evidence searches. This process of awareness raising should start with students – librarians in universities and the UK doctoral training centres have a key role in this as it ties in with the development of information literacy and critical appraisal skills. Ignoring specialist sources such as SPP risks introducing bias – at a time when initiatives are attempting to embed research and analytics in local government and the wider public sector.

Information on the coverage of Social Policy and Practice is available here and the distributor Ovid is offering a free 30-day trial.