Tag Archives: Alliance for Useful Evidence

A world of evidence … but can we trust that it is any good?

Posted on by

What is good evidence? And how can policymakers and decisionmakers decide what is working and what isn’t, when it comes to deciding where public money is spent and how?

These are the kinds of questions that models and tools such as randomised controlled trials and cost-benefit analysis attempt to answer. The government has also supported the development over the last five years of the What Works Network, which now consists of 10 independent What Works Centres. When talking about impact there’s also been a move to capturing and recognising the value of qualitative data.

As one of our key aims is to support and facilitate the sharing and use of evidence in the public sector, we were interested to read a new publication ‘Mapping the standards of evidence used in UK social policy’.

Standards of evidence

Produced by the Alliance for Useful Evidence, the research has found 18 different Standards of Evidence currently in use across UK social policy.

The report notes that over the last decade there has been increasing interest in grading effectiveness or impact against a level or scale. Typically, the higher up the scale, the more evidence is available. Theoretically this means that decision-makers can have higher confidence in deciding whether a policy or intervention is working.

While all the evidence frameworks generally aim to improve the use of evidence, the different goals of the organisations responsible can shape the frameworks in different ways. They can be used to inform funding decisions, to make recommendations to the wider sector about what works and what doesn’t, or as a resource to help providers to evaluate. And unfortunately this means that the same intervention can be assessed differently depending on which framework is used.

The Alliance for Useful Evidence concludes that while a focus on evidence use is positive, the diversity of evidence standards risks creating confusion. Suggested options for improving the situation include introducing an independent accreditation system, or having a one-stop shop which would make it easier to compare ratings of interventions.

Dissemination and wider engagement

The question of standardising evidence frameworks is just one part of a wider effort to increase transparency. As well as collecting evidence, it’s important that when public money has been invested in carrying out evaluations and impact assessments, that this evidence remain accessible over the longer term and that lessons are learned. It can often seem that government departments have very short organisational memories – especially if they’ve suffered a high churn of staff.

Two projects which we support in Scotland are focused on increasing the dissemination and awareness of evaluation and research evidence. Research Online is Scotland’s labour market information hub. Produced by ourselves and Skills Development Scotland, the portal brings together a range of statistics and research and acts as the centre of a community of practice for labour market researchers, practitioners and policy-makers.

Meanwhile Evaluations Online is a publicly accessible collection of evaluation and research reports from Scottish Enterprise. The reports cover all aspects of Scottish Enterprise’s economic development activities – some of the latest added to the site cover megatrends affecting Scottish tourism, innovation systems and the gender gap, and the commercial flower-growing sector in Scotland.

When working within the policy world it can be easy to suffer from fatigue as ideas appear to be continually recycled, rejected and then revisited as policy fashions change and political parties or factions go in and out of power. The spotlight, often driven by the media, will shine on one hot policy issue – for example, moped crime, cannabis legislation or health spending – and then move on.

Online libraries of evaluations and research reports are one tool which can help support a longer-term culture of learning and improvement within the public sector.

Evidence Week 2018

Inspired by similar objectives, Evidence Week runs from 25th to 28th June 2018 and aims to explore the work of parliamentarians in seeking and scrutinising evidence. It will bring together MPs, peers, parliamentary services and the public to talk about why evidence matters, and how to use and improve research evidence.

This may be the start of wider knowledge sharing about standards of evidence, to help those using them to improve their practice.

The Knowledge Exchange is a member of the Alliance for Useful Evidence. Our databases are used by government and the public sector, as well as private-sector consultancies, to keep abreast of policy news and research in social and public policy.

Why a holistic approach to public health and social care needs a wider evidence base … and how Social Policy and Practice can help

Posted on by

SPP screenshot2016 has been described as “make or break time for the NHS”, and with pressures on finances increasing, social care and public health are in the spotlight. Around £1 in every £5 of NHS spending is estimated to be the result of ill health attributable to the big five risk factors of smoking, alcohol, poor diet, obesity and inactivity. Investing in prevention, and understanding the complex wider community and social factors that lead to poor health, is therefore important. In cash-strapped local authorities however, investment in preventative projects can be sidelined in the face of tackling acute issues.

Prevention and behaviour change are linked

Recent health policy has included an expectation that individuals should take greater responsibility for their own health. But where we are talking about behaviour change, there is no quick fix. Glib use of the term ‘nudge’ to promote change can suggest that laziness is the only issue. However, research such as that by the King’s Fund has highlighted that motivation and confidence are essential if people are to successfully modify their health behaviours.

Practitioners within the field of both public health and social care need help understanding what works – but as two great recent blogs from the Alliance for Useful Evidence noted, change can be achieved in multiple ways and evidence shouldn’t be used to prove a service works but as part of a journey of improvement and learning.

We talk about the “caring professions”, but it seems that it can be difficult to maintain a focus on the ‘person not the patient’ when budgets are being cut. Well-reported issues such as the rise in the use of 15-minute home care appointments are just one symptom of this. More generally, making time to consider alternative approaches or learn from good practice elsewhere can be hard. That is where having access to a trusted database can help.

Trusted source of research and ideas

The Alliance for Useful Evidence, most recently in its practice guide to using research evidence, has highlighted the importance of using trusted sources rather than “haphazard online searches”. One of these resources is Social Policy and Practice, a database which we have contributed to for twelve years.

“SPP is useful for any professional working in the field of social care or social work who can’t get easy access to a university library.” Alliance for Useful Evidence, 2016

The partners who contribute to the database – Centre for Policy on Ageing, Idox Information Service, National Children’s Bureau, the NSPCC and the Social Care Institute for Excellence – are all committed to sharing their focused collections with the wider world of researchers and to influence policy and practice.

Social Policy and Practice is the UK’s only national social science database embracing social care, social policy, social services, and public policy. It boasts over 400,000 references to papers, books and reports and about 30% of the total content is grey literature.

Social Policy and Practice has been identified by the National Institute for Health and Care Excellence (NICE) as a key resource for those involved in research into health and social care. And importantly, it supports the ability to take a holistic approach to improving outcomes, by covering social issues such as poor housing, regeneration, active ageing, resilience and capacity building.

Find out more about the development of the Social Policy and Practice database in this article from CILIP Update. Update is the leading publication for the library, information and knowledge management community and they’ve given us permission to share this article.

If you are interested in using the Social Policy and Practice (SPP) database for evidence and research in health and social care, please visit www.spandp.net for more information and to request a free trial.

Read some of our other blogs on evidence use in public policy:

Who’s who in the UK’s evidence landscape

Posted on by

a4ue ecosytem pic

Last week the Alliance for Useful Evidence and The Social Innovation Partnership published an evidence ecosystem map, designed to give a picture of the diversity of organisations involved in supporting evidence use in the government and public sector.

We were proud that two key Idox products were recognised for making research relevant and accessible to practitioners – not just researchers. The Idox Information Service holds over 200,000 summarised research resources and covers over thirty areas of public policy, including planning, economic development and housing. Idox also contributes data to the Social Policy and Practice database, which focuses on health and social care evidence.

Our research team work every day on creating relevant content for our public policy databases. And for the last forty years our mission has been to improve access to research and evidence for local authorities, government agencies and consultancies.

You might be surprised to hear though that UK-produced databases are now a rarity, despite the desire for evidence-based policy being stronger than ever.

Evolving information needs

Over the last four decades, information services have rapidly evolved, responding to both technology developments and to changes in user expectations. The UK used to be a strong competitor in the provision of databases however there are now very few remaining British social science databases.

One of the problems is that many people are unaware that UK databases exist or why they are important. We wrote earlier this year about why using UK-sourced evidence is part of a good literature searching technique and also means that UK policymakers and practitioners draw on relevant research, case studies and commentary.

UK-produced databases

In the last ten years many UK-produced databases have ceased – funding has stopped, publishers have closed or databases have been taken over by international publishers (which reduces the balance of UK content). Some key databases which every social policy researcher should know about and use are:

  • The Idox Information Service

The Idox Information Service (formerly The Planning Exchange) has been providing information services on public policy and practice to central government, public agencies, councils and universities since its inception in the late 1970’s. Its central aim is to support evidence-based policy, by providing UK-relevant resources and research support. Today it holds over 200,000 resources, increasing by up to 1,000 abstracts every month, across 30 public policy areas. These include planning, regeneration, housing, social policy and economic development. Every item is abstracted specially, rather than re-using publisher abstracts.

  • Ageinfo

Ageinfo is the only UK database covering all aspects of ageing and older age, including research and practice in the social and health issues of older age. It is a bibliographic database of over 55,000 books, articles and reports from the specialist collection held by the Centre for Policy on Ageing, who also undertake commissioned research. Created mainly by volunteers now, the database covers policy, support and services on ageing – including health and social services; residential and community care; living arrangements; financial inclusion; independent living; citizenship; rights and risks.

  • NSPCC Inform

A free resource for those working in the childcare and protection sector, the NSPCC library catalogue, known as NSPCC Inform, is dedicated to child protection, child abuse and child neglect. It includes case reviews,  training resources and practice toolkits, international journals and grey literature.

  • Social Care Online

Previously known as Caredata, Social Care Online is a database produced by the Social Care Institute for Excellence, with over 150,000 abstracts covering all aspects of social care, social welfare and social policy. It is currently free to access. It covers information on people with social care needs; those receiving care services; key issues such as integrated services, safeguarding or legislation; and the social care workforce.

  • ChildData

From the National Children’s Bureau charity, ChildData is a bibliographic database covering all aspects of research and practice in young people’s social care. It is now only available through Social Policy and Practice. Content includes reports, research and resources on early childhood; education and learning; health and wellbeing; involving young people; play; sector improvement; SEN and disability; and vulnerable children.

  • Social Policy and Practice

A one-stop-shop for research, analysis and discussion of health and social care, the Social Policy and Practice database holds over 350,000 abstracts on social policy, and 30% of content is grey literature. The database made up from selected content from the major UK database providers: Idox Information Service, Social Care Institute for Excellence, National Children’s Bureau, the Centre for Policy on Ageing and the NSPCC. It is sold and distributed by Ovid Technologies, primarily to universities and the NHS.

  • Health and medical databases

There are some other specialist UK libraries and database producers in the field of health policy. The Kings Fund produces an online database in the area of health management and policy (not clinical information). The Royal College of Nursing and the Royal Society of Medicine have their library catalogues online.

Why use UK databases

A scoping review in 2005 suggested that people searching for social science evidence tend to neglect the question of geographical and coverage bias within research sources. By using these UK services described above, users know they have taken the quickest path to reviewing relevant evidence, confident that they are up to date,  and focused on best practice within the UK.

The rise of the internet makes it increasingly difficult to assess the quality of evidence and all these databases are produced by teams who specialise in the subejct area.

Sourcing and selection of resources is based on the knowledge, experience and expertise of real people and organisations operating within the policy fields. Keywords  and indexing is also UK-focused, which makes searching easier.

Finally, you can get a fuller picture of a subject area, by looking at valuable grey literature rather than relying on peer reviewed journals. Grey literature is produced directly by organisations, including government departments and agencies, academic research centres, NGOs and think tanks, and commercial consultants, and has been found to be especialy useful for the complex information needs of policy makers.

Disappointing lack of awareness

There are many students and academics who remain unaware of UK databases, and it is disappointing how many commisisoned literature reviews will rely on one or two commercial (American-produced) databases.

To change this, and ensure that the next generation of policymakers and practitioners know the valuable resources that are available to them, we would love to see academic librarians advocate for specialist databases, rather than relying on what the major publishers will bundle in discovery systems.

And we hope the new evidence ecosystem map will raise awareness of the wide variety of organisations and groups who produce and use evidence in the UK.

We are currently offering a free trial of our database to librarians or academics who run courses in social policy, public policy or planning and the built enviornment. Contact us for more information.

Follow us on Twitter to see what developments in public and social policy are interesting our research team.

Supporting professional practice within planning … how the Knowledge Exchange helps

Posted on by

wordle3By Morwen Johnson

The Idox Information Service has been offering an information service for planners and other built environment professionals for forty years. Our new individual subscription offer allows planners to benefit from our expertise and acess a wide range of resources for use in their work and continuing professional development.

Use of evidence within planning

“Research and theory can help to lift the perspective of practitioners beyond the day-to-day demands of the job, to provoke reflection and discussion about the wider social purposes and values of planning. It can also help us better to defend planning from those who would seek to erode it further.”

These words from Dr Mike Harris, Deputy Head of Policy and Research at the Royal Town Planning Institute, highlight the importance of ensuring that the planning profession is able to access and use evidence and research. Research is inherent to what it means to be a professional – to take informed, evidence-based decisions.

Barriers to the use of research

Like other professions, planners face barriers to keeping up with the latest evidence – most obviously lack of time and the accessibility of much research (both in terms of knowing it is out there and being able to understand how academic reserach relates to practice).

With resources in local planning authorities in particular likely to remain very tight, but with practitioners under more pressure to deliver than ever, the question is how can the profession retain and enhance the evidence base it needs to do its job effectively? And how can practitioners engage with the research and analysis on key developments in policy that affect planning, such as devolution, economic growth, sustainability and housing?

Cross-cutting issues

It was clear at this year’s RTPI Convention that the planning profession has a significant contribution to make to current policy priorities such as improving wellbeing, sustainability and economic growth. Understanding these wider debates and engaging with them effectively, in a way that positions planners as a solution to a problem rather than a barrier, is something that is crucial for the future of the profession.

Reflecting recently on this year’s Association of European Schools of Planning (AESOP) research conference, the RTPI’s Mike Harris suggested that “we urgently need the academic community to help us demonstrate how planning can produce better, more efficient markets for local development, in ways which promote growth and meet social and environmental objectives”.

New, exclusive offer for planners

In response to demand from planners to access research, evidence, opinion and commentary on both planning and wider cross-cutting issues, we have introduced a new individual membership offer to our Information Service, exclusively for RTPI Members.

Individual membership provides RTPI members with access to:

  • Searchable online database of planning research resources with over 200,000 records and with 10,000 new items added every year;
  • Free access to member-only briefings on key planning topics;
  • 50 current awareness bulletins per year;
  • Twice-monthly topic updates – choose from over 32 topics including planning, regeneration, economic development, management, architecture, transport and community development;
  • Free user support to help planners find and access the research they need;
  • Book loans and copyright material are available on request (subject to additional charges).

The annual subscription of £179 (plus VAT) represents a 30% saving on the cost of the normal rate for individuals.

We’re happy to be working with the RTPI to enhance professional knowledge and intelligence at a crucial time for planning, and we look forward to welcoming more RTPI members as subscribers to our service.

To find out more about what we cover, read our planning subject guide.

You can find out more about the offer for Institute members on the RTPI website.

Why UK-sourced evidence matters … and why it is so often ignored

Posted on by

By Morwen Johnson

If you follow our blog, you’ll know that we care passionately about promoting the uptake of evidence and research by policymakers and practitioners. It’s easy to be complacent and assume that when public money is at stake, decisions are made on the basis of evaluations and reviews. Unfortunately, this is still not always the case.

The current evidence-based policy agenda in the UK encompasses initiatives such as the What Works network, the Local Government Knowledge Navigators and independent organisations such as the Alliance for Useful Evidence. They are working on fostering demand for evidence, as well as linking up academics with those in the public sector to ensure that the research community is responsive to the needs of those making decisions and designing/delivering services.

A recent article in Health Information and Libraries Journal highlights another challenge in evidence-based policy however. A mapping exercise has found that literature reviews often ignore specialist databases, in favour of the large, well-known databases produced by major commercial publishers. Within the health and social care field (the focus of the article), literature reviews tend to use databases such as Medline, Embase and Cinahl – and overlook independent UK-produced databases, even when they are more relevant to the research question.

Why does it matter?

Research has shown that how (and why) databases are chosen for literature searching can “dramatically influence the research upon which reviews, and, in particular, systematic review, rely upon to create their evidence base”.

To generate useful evidence for the UK context (relating to UK policy issues or populations), researchers need to understand the most appropriate database to search – but unfortunately our own experience of looking at the detail of methodologies in evidence reviews, suggests that in many cases the only databases searched are those produced by American or international publishers.

Grey literature is a valuable source in evidence reviews – and again this is often overlooked in the major databases which tend to focus only on peer-reviewed journal content. A recent Australian report ‘Where is the evidence?‘ argued that grey literature is a key part of the evidence base and is valuable for public policy, because it addresses the perspectives of different stakeholder groups, tracks changes in policy and implementation, and supports knowledge exchange between sectors (academic, government and third sector).

Another benefit of UK-produced databases is that they will make use of UK terminology in abstracts and keywords.

Social Policy and Practice – a unique resource

At this point I should declare a vested interest – The Knowledge Exchange is a member of a UK consortium which produces the Social Policy and Practice (SPP) database. The SPP database was created in 2005 after five UK organisations, each with a library focused on sharing knowledge in community health and social care, agreed to merge their individual content in order to make it available to the widest possible audience.

The current members of the SPP consortium – the National Children’s Bureau, the Idox Knowledge Exchange, the Centre for Policy on Ageing and the Social Care Institute for Excellence – have just been joined by the National Society for the Prevention of Cruelty to Children. Inclusion of the NSPCC’s bibliographic data greatly enhances the coverage of child protection research in the database. SPP has been identified by NICE, the National Institute for Health and Care Excellence, as a key resource for those involved in research into health and social care.

We want the UK research community to understand what SPP offers, and to use it when undertaking literature reviews or evidence searches. This process of awareness raising should start with students – librarians in universities and the UK doctoral training centres have a key role in this as it ties in with the development of information literacy and critical appraisal skills. Ignoring specialist sources such as SPP risks introducing bias – at a time when initiatives are attempting to embed research and analytics in local government and the wider public sector.

Information on the coverage of Social Policy and Practice is available here and the distributor Ovid is offering a free 30-day trial.

Knowledge insider…. a Q & A with Jonathan Breckon

Posted on by

jonathan_breckon_150x150In the latest of our series of Q&As with leading advocates of the use of evidence in policymaking and practice, we talk to Jonathan Breckon. Jonathan is Head of the Alliance for Useful Evidence – a partnership which champions the need for useful evidence, providing a focal point for improving and extending the use of social research and evidence in the UK.

Jonathan, what led you to a role about promoting and improving knowledge development? 

There are two ways in which I am interested in knowledge development; professionally I have always worked around universities, loved doing and finding out new research and working within research in UK. I have always been conscious however, of the gap between research and front line services, even when research is relevant to the service, and felt this was a great loss and disadvantage to public services.

My personal interest, as a user of public services, with my kids going through services such as schools, health and sports, I have been desperately aware that things are business as usual rather than continuously striving for innovation and change. The debate is now all about money and reductions when it should be about improvement and future proofing.

We don’t always know what it takes to bridge the gap between what we need, and what services can provide; research can actually help that. The What Works approach is really important but very hard, as it’s difficult to stop doing things we have already invested in. An evidence-based research approach can challenge and support this evaluation and we have a moral duty to do it and not continue to invest in services which don’t work.

What do you think the main benefits of developing your knowledge are?

The challenge of seeing if things work or not, why they work, where they work and who they work for – developing your knowledge is the critical aspect of improving how you do things.

It’s also important for a whole host of other benefits. I particularly like Carol Weiss’ work, which is instrumental – this ‘enlightment’ operational research should not be dismissed. This approach can support the ideas of learning continuously through research; it implies a continuous review of theory, methods, practice and we should always be striving to improve our methods and outcomes.

When people are talking to you about evidence, research or knowledge, what do they most frequently raise as issues?

The most common one is that investing in evidence is just rhetoric; politicians, charities, parties etc will never really be informed by the research agenda, and I agree. We aren’t in a super-rational culture, it’s about our wider culture, values and beliefs as well. But it’s a fundamental misunderstanding that research trumps anything. It is part of the mix, part of the overall democratic and rational approach to doing anything.

The Behavioural Insights Team has a massive role to play in understanding the biases in how we make decisions, whether in prisons, police, policy etc. We don’t work rationally all the time and evidence can help us understand the messiness of policy making. We just need it to be a bigger part of the mix.

Everybody has this view that they use evidence but we don’t really understand how effectively they use it.

What are the hard to spot mistakes when it comes to developing your knowledge, which you really need to avoid?

The main one is that not all evidence is equal; that you have to make judgements about it. This is hard for those writing the research – it’s not about the quality of the research, and it’s about the point of view from demand. They need some things and not others.

The big challenge, if you are looking at impact, is you need different approaches, experiments, systematic reviews – one study is not enough. Such as when you see studies reported in newspapers – until replicated we don’t really know if it is robust. You need to avoid literature reviews where you cherry pick, go to something which is transparent and is systematic. This is true of both policy makers and researchers’ point of view; we underestimate the challenges facing both sides.

Need more about impact. We are very good at qualitative – world class – but we are behind in quantitative methods. It is being addressed but it will need to filter through.

How do you think people will be doing evidence, research and knowledge development in 5 years’ time?

What Works Centres will, I hope, be a key part of the evidence ecosystem, in the way that NICE have done, helping providers and policy makers make decisions. Although it doesn’t do research itself it sucks in research and uses it well.

There will always be critics of them, even of just the name, but they will change the system. Some have been around for a while and are well established, but others are new and are just about to be. As well as synthesising research they will commission new work. For instance in wellbeing, we know a lot about the correlation with health and wellbeing but don’t know a lot about what will work in improving it.

Technology makes it very difficult to guess about the future, who would have predicted the work in social media research? Big data is emerging now and in 5 years’ time might be a standard tool. The fundamental principles like statistics will be there but we will have to adapt to the possibilities offered by technology

If you had a list of ‘best-kept secrets’ about research, evidence and knowledge you would recommend, what would you include and why?

Just because you have done a social science masters and PhD, does not make you an expert in evidence, partly because people over-specialise. People need open their minds to different methods and how people do it in other places. The Department for International Development have an amazing range of techniques, nothing like anything you have seen, with a database of all the research they have funded or delivered.

Emerging opportunities such as social media research – still early days and fundamentally new, and could have a huge impact. Most people’s default though is to go to an expert and be frightened off journals and academics; I don’t think you always have to commission something new, it’s about variety, breadth and developing your understanding in as many ways as possible.

 

You can also read Q&As with Tim Allen, Local Government Knowledge Navigator; Clive Grace, Local Government Knowledge Navigator; Sarah Jennings of the Knowledge Hub; and Kim Ryley, recent Past President of the Society of Local Authority Chief Executives.