Assessing information quality: sorting the wheat from the CRAAP

The rapid expansion of the internet has enabled users to access unprecedented amounts of information.  However, not all of this information is valid, useful or accurate. In the world of ‘post-truth’ and ‘fake news’, the ability to critically assess information and its source is an essential skill.  Let’s consider what this means for public policy.

Growth of evidence-based policy

The need to assess research evidence is no longer limited to academics and scientists. The shift towards evidence-based decision-making means that policy makers and decision makers at every level now need to incorporate evidence into policy and practice.

The growth of the randomised controlled trials movement in public policy also reinforces the need for decision makers to be familiar with a range of research approaches.

In the UK, the What Works Network and the Alliance for Useful Evidence both work towards encouraging and improving the use of evidence to improve public services. Similarly, the Evidence Matters campaign seeks to promote the importance of evidence in policymaking, and tackle the misuse of research findings.

Publication doesn’t guarantee quality

While most of us are aware of the risks of encountering ‘fake news’ online, relying uncritically upon Google as an information source can leave one falling foul of ‘predatory’ open access journals, which masquerade as legitimate, peer-reviewed publications.

In recent years, there has been a boom in articles being published open access. There are now a vast number of good quality, open access publications in just about every subject imaginable.  Overall, this has been a positive development – who can argue with making more research free and easily accessible?

Open access not only has an ethical dimension – in many situations, it is also an obligation.  The UK government has already committed to ensuring that all publicly funded research is made available via open access.

However, the proliferation of open access material has led to a new problem – that of predatory open access journals. These journals operate using a business model that involves charging publication fees to authors, without providing the editorial and publishing services associated with legitimate journals. They may even include fake editors or members of the editorial board. Librarian and researcher, Jeffry Beall, has compiled a rather impressive list of ‘potential, possible, or probable predatory scholarly open-access publishers’.

The quality of articles published in predatory journals is therefore questionable. Recent (rather entertaining) examples of how unreliable such journals can be include the neuroscientist who managed to trick a number of scientific journals into publishing a nonsensical piece of research complete with a number of Star Wars references, including the authors Dr Lucas McGeorge and Dr Annette Kin, and the article reporting the case of a man who develops ‘uromycitisis poisoning’, inspired by a 1991 episode of Seinfeld (the actual article is still online).

Is this information CRAAP?

So how do you assess the quality of a piece of information?

One way to do this is to ask yourself – is this information CRAAP? The CRAAP test was developed by Meriam Library at California State University to help students think critically about the sources of information they had identified.  Some of the key questions to consider when evaluating information sources are:

  • Currency
    • When was the information published?
    • Has it been revised or updated?
  • Relevance
    • Who is the intended audience?
    • Is the information at an appropriate level (i.e. not too advanced/basic?)
    • How well does the information relate to your topic?
  • Authority
    • Who is the author/publisher/source/sponsor?
    • Is the author qualified to write on this subject?
  • Accuracy
    • Is the information supported by evidence?
    • Has the information between reviewed or refereed?
    • Are there any spelling/grammatical errors?
  • Purpose
    • Is the information fact, opinion or propaganda?
    • Is it objective and impartial?
    • Are there any religious/political/cultural/personal biases?
    • Is it trying to sell a product?

How we can help

While the increasing availability of information via the internet and the growth of open access content has undoubtedly been a positive development, it goes hand in hand with the need for users to critically assess information sources.

Here in the Idox Information Service, we take pride in our database of high quality resources. Our Researchers select only the best quality resources from hundreds of verified sources to populate our database. Our research database has been recognised by the Alliance for Useful Evidence as a key tool within the UK.

Each week, we support policy and decision makers by providing the latest research and evidence on a range of public policy issues – both through our current awareness services, and through bespoke literature reviews. In doing so, we hope to contribute in our own small way to the wider drive to improve the use of evidence in public policy decision making.

As we have seen, the use of inaccurate or misleading information can have significant real-world consequences.  The need for authoritative, accurate and relevant research has never been greater.


Find out more about the Idox Information Service and our subscription services here.

Follow us on Twitter to see what developments are interesting our research team.

From data to intelligence and improvement – what cutting edge councils are doing in the UK

Group of workers having a meeting

By Steven McGinty

Data has the potential to revolutionise the delivery of local services. Just like the private sector – where organisations such as Amazon and Facebook have leveraged user data – local councils have the opportunity to reap significant benefits from analysing their vast silos of data. Improving efficiencies, increasing levels of transparency, and providing services which better meet people’s needs, are just some of the potential benefits.

Although many councils are still at the early stages of utilising their data, some are innovating and introducing successful data initiatives.

Wise Councils

In November 2016, the charity NESTA published a report highlighting the most ‘pioneering’ uses of data in local government. The report emphasised that most local services would benefit from data analysis and that a ‘problem-oriented’ approach is required to generate insights that have an impact on services. The case studies included:

Kent County Council

Kent County Council (KCC), alongside Kent’s seven Clinical Commissioning Groups (CCGs), have created the Kent Integrated Dataset (KID) – one of the largest health and care databases in the UK, covering the records of 1.5 million people. The core requirement of the dataset was to link data from multiple sources to a particular individual, i.e. that information held about a person in hospital, should also be linked to records held by other public bodies such as GPs or the police.

This integrated dataset has enabled the council to run sophisticated data analysis, helping them to evaluate the effectiveness of services and to inform decisions on where to locate services. For example, Kent’s Public Health team investigated the impact of home safety visits by Kent Fire and Rescue Service (KFRS) on attendances at accident and emergency services (A&E). The data suggested that home safety visits did not have a significant impact on an individual’s attendance at A&E.

Leeds City Council

Leeds City Council have focused their efforts on supporting open innovation – the concept that good ideas may come from outside an organisation. This involved the initiatives:

  • Data Mill North (DMN) – this collaborative project between the city council and private sector is the city’s open data portal (growing from 50 datasets in 2014 to over 300 data sets, in over 40 different organisations). To encourage a culture change, Leeds City Council introduced an ‘open by default’ policy in November 2015, requiring all employees to make data available to the public. A number of products have been developed from data published on DMN, including StreetWise.life, which provides local information online, such as hospital locations, road accidents, and incidents of crime.
  • Innovation Labs – the city has introduced a series of events that bring together local developers and ‘civic enthusiasts’ to tackle public policy problems. Leeds City Council has also provided funding, allowing some ideas to be developed into prototypes. For example, the waste innovation lab created the app, Leeds Bins, which informs residents which days their bins should be put out for collection.

Newcastle City Council

Newcastle City Council have taken a data-led approach to the redesign of their children’s services. The Family Insights Programme (FIP) used data analysis to better understand the demand and expenditure patterns in the children’s social care system. Its aim was to use this insight to support the redesign of services and to reduce the city’s high re-referrals and the number of children becoming looked-after.

The FIP uses data in three different ways:

  • Grouping families by need – The council have undertaken cluster analysis to identify common grouping of concerning behaviours, such as a child’s challenging behaviour or risk of physical abuse. When a child is referred to long term social work, senior social workers analyse the concerning behaviours of the case, and then make a referral to a specialist social work unit. Since introducing this data-led approach, social work units have been organised based on needs and concerning behaviours. This has resulted in social workers becoming specialists in supporting particular needs and behaviours, providing greater expertise in the management of cases.
  • Embedding data analysts – Each social work unit has an embedded data analyst, who works alongside social workers. Their role is to test what works, as well as providing insights into common patterns for families.
  • Enabling intelligent case management – Social workers have access to ChildSat, a tool which social workers use to help manage their cases. It also has the capability to monitor the performance of individual social work units.

Investing in data

Tom Symons, principal researcher in government innovation at Nesta, has suggested that councils need support from central government if they are to accelerate their use of data. He’s suggested that £4 million – just £1% of the Government Digital Service (GDS) budget – is spent on pilot schemes to embed data specialists into councils.

Mr Symons has also proposed that all combined authorities should develop Offices of Data Analytics, to support data analysis across counties. Over the past few months, Nesta has been working on this idea with the Greater London Authority, and a number of London boroughs, to tackle the problem of unlicensed HMOs (Houses in Multiple Occupation). Early insights highlight that data analytics could be used to show that new services would provide value for money.

Final thoughts  

After successive years of cuts, there has never been a greater need for adopting a data-led approach. Although there are undoubtedly challenges in using council data – including changing a culture where data sharing is not the norm, and data protection – the above examples highlight that overcoming these challenges is achievable, and that data analysis can be used to bring benefits to local councils.


Follow us on Twitter to see what developments in public and social policy are interesting our research team. If you found this article interesting, you may also like to read our other digital articles. 

How to become a more effective coach

Teacher talking with student

Coaching can be described as the use of positive support, feedback and advice to help improve personal effectiveness.

Its use within the work environment is not a new concept.  Indeed, according to the CIPD, 9 out of 10 organisations already use coaching by line managers, and 2 out of 3 use external coaches.

However, despite its prevalence, there is very little research evidence about what makes a ‘good’ coach and whether coaching actually works.

The Institute for Employment Studies (IES) are among those working to address this.  In August, they published a report which explored the factors leading to coaching success, from both the coach and the coachee perspective.  They also examined the nature of an effective coaching relationship and set out practical advice for organisations on how to improve coaching elements of everyday work.

The key to success

They found that, according to coachees, the most important characteristics of a coach were:

  1. Communicates clearly (including the ‘ability to listen’, ‘ask good questions’ and being ‘non-directive’)
  2. Displays emotional intelligence (e.g. ‘presence’, ‘emotionally involved’, ‘awareness’, ‘connection’, ‘sensitive’, ‘empowering’, and ‘authentic’)
  3. Has experience within the coachee’s industry
  4. Is challenging but supportive
  5. Displays acceptance of the coachee

In the context of achieving successful outcomes from coaching specifically, coachees felt that successful coaches:

  1. Displayed acceptance of the coachee
  2. Were calm
  3. Displayed self-confidence
  4. Were organised
  5. Had experience within their industry

The characteristic ‘has experience within my industry’ was of particular interest.  Whether or not coaches need experience of the industry in which their coachee works is a point of contention between different coaching researchers and practitioners.  Based on this research, the IES suggest that industry experience may help to improve coach credibility, but also note that who coaches are has importance to coachees, not just what they do”.

They conclude that “the key to effective coaching lies within the coachee having respect for the coach’s ability. A coachee can also derive comfort from the coach’s experience in dealing with situations, and in the coach’s confidence and manner”.

While the characteristics perceived as important by coach and coachee were broadly similar, it was noted that coaches being organised, calm and self-confident was considered very important to coachees – much more so than to the coaches themselves.

In terms of the coaching relationship, the coach having ‘similar values’ was considered the key to success.   It is thought that such shared values may promote the sense of connection between the coach and coachee.  The coach being the same gender, age or having a similar personality was less important to the development of a successful coaching relationship.

Addressing the barriers

The majority of coachees felt that their coaching was effective.  However, there is clearly room for improvement – around 1 in 10 people felt that their coaching was of limited or no effect at allPrevious research by the IES has also shown that as many as 84% of coachees have faced barriers to their coaching experience.  These include:

  1. Unclear goals
  2. Emotions getting in the way
  3. Lack of commitment
  4. Unsupportive boss
  5. Defensiveness

Coachees also faced difficulties with:

  • Their own readiness and engagement
  • The coaching model used
  • Organisational culture/boss
  • Coaches skills or manner
  • External events
  • The coaching relationship

Awareness of the barriers commonly experienced by coachees and the factors coachees perceive as contributing towards their success is a useful first step towards developing and adopting effective coaching practices.

Improving coaching practice

According to the IES, their research on coaching is a conscious attempt to “shift away from ‘guru’- led coaching practices to research-informed and evidence-based practices”.  Based on their research to date, they offer the following advice for coaches and line managers:

  • Not to worry about having less experience than coachees – that the coachee having respect for your ability is more important
  • Weave reflection into everyday coaching practice after each session/encounter – consider how to best help your coachee, how your coaching made a difference, and how your coaching compares to that of others
  • Obtain feedback from your coachee about what you did that made the coaching successful (or unsuccessful) for them, and ask them to contribute to collective feedback mechanisms such as evaluation surveys

Follow us on Twitter to see what developments in public and social policy are interesting our research team. If you found this article interesting, you may also like to read our briefing on coaching and mentoring.

Why a holistic approach to public health and social care needs a wider evidence base … and how Social Policy and Practice can help

SPP screenshot2016 has been described as “make or break time for the NHS”, and with pressures on finances increasing, social care and public health are in the spotlight. Around £1 in every £5 of NHS spending is estimated to be the result of ill health attributable to the big five risk factors of smoking, alcohol, poor diet, obesity and inactivity. Investing in prevention, and understanding the complex wider community and social factors that lead to poor health, is therefore important. In cash-strapped local authorities however, investment in preventative projects can be sidelined in the face of tackling acute issues.

Prevention and behaviour change are linked

Recent health policy has included an expectation that individuals should take greater responsibility for their own health. But where we are talking about behaviour change, there is no quick fix. Glib use of the term ‘nudge’ to promote change can suggest that laziness is the only issue. However, research such as that by the King’s Fund has highlighted that motivation and confidence are essential if people are to successfully modify their health behaviours.

Practitioners within the field of both public health and social care need help understanding what works – but as two great recent blogs from the Alliance for Useful Evidence noted, change can be achieved in multiple ways and evidence shouldn’t be used to prove a service works but as part of a journey of improvement and learning.

We talk about the “caring professions”, but it seems that it can be difficult to maintain a focus on the ‘person not the patient’ when budgets are being cut. Well-reported issues such as the rise in the use of 15-minute home care appointments are just one symptom of this. More generally, making time to consider alternative approaches or learn from good practice elsewhere can be hard. That is where having access to a trusted database can help.

Trusted source of research and ideas

The Alliance for Useful Evidence, most recently in its practice guide to using research evidence, has highlighted the importance of using trusted sources rather than “haphazard online searches”. One of these resources is Social Policy and Practice, a database which we have contributed to for twelve years.

“SPP is useful for any professional working in the field of social care or social work who can’t get easy access to a university library.” Alliance for Useful Evidence, 2016

The partners who contribute to the database – Centre for Policy on Ageing, Idox Information Service, National Children’s Bureau, the NSPCC and the Social Care Institute for Excellence – are all committed to sharing their focused collections with the wider world of researchers and to influence policy and practice.

Social Policy and Practice is the UK’s only national social science database embracing social care, social policy, social services, and public policy. It boasts over 400,000 references to papers, books and reports and about 30% of the total content is grey literature.

Social Policy and Practice has been identified by the National Institute for Health and Care Excellence (NICE) as a key resource for those involved in research into health and social care. And importantly, it supports the ability to take a holistic approach to improving outcomes, by covering social issues such as poor housing, regeneration, active ageing, resilience and capacity building.


Find out more about the development of the Social Policy and Practice database in this article from CILIP Update. Update is the leading publication for the library, information and knowledge management community and they’ve given us permission to share this article.

If you are interested in using the Social Policy and Practice (SPP) database for evidence and research in health and social care, please visit www.spandp.net for more information and to request a free trial.

Read some of our other blogs on evidence use in public policy:

Evidence use in health and social care – introducing Social Policy and Practice

SPP screenshotWith public sector austerity and the integration of health and social care, it seems as though the need for access to evidence-based policy and practice has never been stronger. Initiatives such as those from the Alliance for Useful Evidence, most recently its practice guide to using research evidence, have highlighted the importance of using trusted sources rather than “haphazard online searches”. One of these resources is Social Policy and Practice, a database which we have contributed to for twelve years.

“SPP is useful for any professional working in the field of social care or social work who can’t get easy access to a university library.” Alliance for Useful Evidence, 2016

The journey to a new resource

A recent article in CILIP Update has explored the background to the Social Policy and Practice database, and its contribution to ensuring the inclusion of grey literature and a UK-perspective within systematic reviews. Update is the leading publication for the library, information and knowledge management community and they’ve given us permission to share this article with our blog readers.

We’ve written before about the value of UK-produced databases within a global publishing context. The CILIP Update article makes the point that when conducting any search for evidence, it’s important to look beyond the major databases to more specialist collections, to source grey literature and to look beyond geographical borders.

This was one of the major drivers in 2003, when the heads of the Centre for Policy on Ageing, the Greater London Authority, Idox Information Service, National Children’s Bureau and Social Care Institute for Excellence agreed to pool their resources and create the first national social science database embracing social care, social policy, social services, and public policy.

The proposal was met with great enthusiasm by the sector which recognised a gap in provision. The new Social Policy and Practice database was launched in November 2004.

The strengths of consortium working

Bringing together these organisations was relatively simple. They were all striving to provide evidence and information to their staff, members or customers. They all also had a professional drive to share their focused collections with the wider world of researchers and to influence policy and practice.

Through developing best practice and troubleshooting problems together, we have improved not only the Social Policy and Practice database for users but also improved our own individual collections. All whilst remaining independent and focused on our individual specialities.

Continuing to evolve

The NSPCC joined the database consortium in 2015, bringing its collection of resources focused on child abuse, child neglect and child protection. The NSPCC library is Europe’s largest collection of publications dedicated to safeguarding children. It includes journal articles, books, academic papers, leaflets, reports, audio-visual material, websites and digital media on all subjects that help researchers, policy makes and practitioners protect children from abuse and neglect.

Social Policy and Practice now boasts over 400,000 references to papers, books and reports and about 30% of the total content is grey literature. Social Policy and Practice has been identified by the National Institute for Health and Care Excellence (NICE) as a key resource for those involved in research into health and social care.

We’re proud to be contributing to the knowledge base for social research, policy and practice!


To find out more about Social Policy and Practice (SPP) database for evidence and research in health and social care, or to get a free trial, please visit www.spandp.net

Read some of our other blogs on evidence use in public policy:

Budget cuts hit research in councils’ adult social care departments

 

By Morwen Johnson

The news that budget pressures have affected councils’ ability to carry out research in adult social care won’t come as any surprise to those working in the sector. Councils have cut £4.6bn from adult social care budgets since 2009-10, equivalent to almost a third of net real terms spend, according to Adass. And with research seen as non-essential, it will always lose out in favour of frontline services and care packages.

A recent survey carried out by the Social Services Research Group (SSRG) and commissioned by the Personal Social Services Research Unit (PSSRU), highlighted the scale of the problem though, finding that there were fewer staff to do research, and those who were left had fewer resources and less support.

Dr Chris Rainey, one of the report authors commented: “In-house research is critical to finding out what, how and why services are delivered and what difference they make. The survey points to the need to reinvest in local research capacity to ensure sound evidence is used”.

Barriers to the use of research

As well as low capacity to undertake their own research on local needs, the survey also identified restrictions on training and professional development.

Like other professions, those working in public health and social services face barriers to keeping up with the latest evidence and commentary. This includes lack of time but also the accessibility of much research (both in terms of knowing it is out there and being able to understand how research relates to practice).

The report highlighted the risk that “reliance on internet-based training and information … may result in a lack of exposure to critical debate and an over-reliance on ‘received wisdom’”.

What our members say

The findings reflect our own experience in meeting the information needs of council staff. We’ve a number of adult social care departments who use our Information Service and once staff realise the time savings we offer, they become champions of the service to colleagues. As our team is made up of information professionals and researchers, we offer experience that can be lacking internally. Resources include peer-reviewed journals, grey literature, books and practice-based case studies and evaluations – which won’t be found by searching Google.

Staff also use us for CPD purposes – nowadays spending on event and conference attendance is unlikely to be approved, but our briefings and current awareness services can help keep them up-to-date with essential topics. We also have a lot of resources on general management issues, such as managing teams, benchmarking, performance, equalities and communication.

“From time to time, we review this service and our last review showed that those who use it regularly either in a corporate capacity or in our major strategic services value it highly, describing it as quick, easy, and comprehensive. It gives staff access to a wide range of information and keeps them up-to-date across many areas that are of direct relevance.”

“I recently completed a Post Graduate course and used it as my first reference point at the beginning of each module. The service saved me a lot of time in searching for articles and books and the staff were extremely helpful. The library is well stocked and I didn’t need to purchase any books for the course.”

“Having access to the on-demand research service is a real plus, and most of our staff see real advantage to that. It saves them time in the long run and frees them up to do the day job.

The threat of short-termism

With resources in social care departments likely to remain very tight, but with practitioners under more pressure to deliver than ever, the question is how can local authorities retain and enhance the evidence base it needs to make decisions effectively?

And how can practitioners engage with the research and analysis on key developments in policy that affect social care services, such as demographic change, housing need, and independent living?

It’s worth remembering that local authority social services researchers were introduced as a result of a recommendation of the 1968 Seebohm Report. This report stressed the need for research and evaluation to be ‘a continuous process, accepted as a familiar and permanent feature of any department or agency concerned with social provision.”

But as we approach another Spending Review, it’s likely that adult social care services will face more cuts. This is despite national organisations representing the sector issuing a statement in October arguing that the sustainability of the sector has now reached a ‘crunch’ point.

Focusing on efficiency savings and short-term interventions may seem the only option at the moment, but we risk just patching up problems rather than delivering services which take a holistic and long-term view of outcomes. And that’s why recognising the value of research and evidence should be a key part of decision-making in every part of the public sector.


We are currently offering a free trial of our service for local authorities. Contact us for more information.

Follow us on Twitter to see what developments in public and social policy are interesting our research team.

Supporting professional practice within planning … how the Knowledge Exchange helps

wordle3By Morwen Johnson

The Idox Information Service has been offering an information service for planners and other built environment professionals for forty years. Our new individual subscription offer allows planners to benefit from our expertise and acess a wide range of resources for use in their work and continuing professional development.

Use of evidence within planning

“Research and theory can help to lift the perspective of practitioners beyond the day-to-day demands of the job, to provoke reflection and discussion about the wider social purposes and values of planning. It can also help us better to defend planning from those who would seek to erode it further.”

These words from Dr Mike Harris, Deputy Head of Policy and Research at the Royal Town Planning Institute, highlight the importance of ensuring that the planning profession is able to access and use evidence and research. Research is inherent to what it means to be a professional – to take informed, evidence-based decisions.

Barriers to the use of research

Like other professions, planners face barriers to keeping up with the latest evidence – most obviously lack of time and the accessibility of much research (both in terms of knowing it is out there and being able to understand how academic reserach relates to practice).

With resources in local planning authorities in particular likely to remain very tight, but with practitioners under more pressure to deliver than ever, the question is how can the profession retain and enhance the evidence base it needs to do its job effectively? And how can practitioners engage with the research and analysis on key developments in policy that affect planning, such as devolution, economic growth, sustainability and housing?

Cross-cutting issues

It was clear at this year’s RTPI Convention that the planning profession has a significant contribution to make to current policy priorities such as improving wellbeing, sustainability and economic growth. Understanding these wider debates and engaging with them effectively, in a way that positions planners as a solution to a problem rather than a barrier, is something that is crucial for the future of the profession.

Reflecting recently on this year’s Association of European Schools of Planning (AESOP) research conference, the RTPI’s Mike Harris suggested that “we urgently need the academic community to help us demonstrate how planning can produce better, more efficient markets for local development, in ways which promote growth and meet social and environmental objectives”.

New, exclusive offer for planners

In response to demand from planners to access research, evidence, opinion and commentary on both planning and wider cross-cutting issues, we have introduced a new individual membership offer to our Information Service, exclusively for RTPI Members.

Individual membership provides RTPI members with access to:

  • Searchable online database of planning research resources with over 200,000 records and with 10,000 new items added every year;
  • Free access to member-only briefings on key planning topics;
  • 50 current awareness bulletins per year;
  • Twice-monthly topic updates – choose from over 32 topics including planning, regeneration, economic development, management, architecture, transport and community development;
  • Free user support to help planners find and access the research they need;
  • Book loans and copyright material are available on request (subject to additional charges).

The annual subscription of £179 (plus VAT) represents a 30% saving on the cost of the normal rate for individuals.

We’re happy to be working with the RTPI to enhance professional knowledge and intelligence at a crucial time for planning, and we look forward to welcoming more RTPI members as subscribers to our service.


To find out more about what we cover, read our planning subject guide.

You can find out more about the offer for Institute members on the RTPI website.