Assessing information quality: sorting the wheat from the CRAAP

The rapid expansion of the internet has enabled users to access unprecedented amounts of information.  However, not all of this information is valid, useful or accurate. In the world of ‘post-truth’ and ‘fake news’, the ability to critically assess information and its source is an essential skill.  Let’s consider what this means for public policy.

Growth of evidence-based policy

The need to assess research evidence is no longer limited to academics and scientists. The shift towards evidence-based decision-making means that policy makers and decision makers at every level now need to incorporate evidence into policy and practice.

The growth of the randomised controlled trials movement in public policy also reinforces the need for decision makers to be familiar with a range of research approaches.

In the UK, the What Works Network and the Alliance for Useful Evidence both work towards encouraging and improving the use of evidence to improve public services. Similarly, the Evidence Matters campaign seeks to promote the importance of evidence in policymaking, and tackle the misuse of research findings.

Publication doesn’t guarantee quality

While most of us are aware of the risks of encountering ‘fake news’ online, relying uncritically upon Google as an information source can leave one falling foul of ‘predatory’ open access journals, which masquerade as legitimate, peer-reviewed publications.

In recent years, there has been a boom in articles being published open access. There are now a vast number of good quality, open access publications in just about every subject imaginable.  Overall, this has been a positive development – who can argue with making more research free and easily accessible?

Open access not only has an ethical dimension – in many situations, it is also an obligation.  The UK government has already committed to ensuring that all publicly funded research is made available via open access.

However, the proliferation of open access material has led to a new problem – that of predatory open access journals. These journals operate using a business model that involves charging publication fees to authors, without providing the editorial and publishing services associated with legitimate journals. They may even include fake editors or members of the editorial board. Librarian and researcher, Jeffry Beall, has compiled a rather impressive list of ‘potential, possible, or probable predatory scholarly open-access publishers’.

The quality of articles published in predatory journals is therefore questionable. Recent (rather entertaining) examples of how unreliable such journals can be include the neuroscientist who managed to trick a number of scientific journals into publishing a nonsensical piece of research complete with a number of Star Wars references, including the authors Dr Lucas McGeorge and Dr Annette Kin, and the article reporting the case of a man who develops ‘uromycitisis poisoning’, inspired by a 1991 episode of Seinfeld (the actual article is still online).

Is this information CRAAP?

So how do you assess the quality of a piece of information?

One way to do this is to ask yourself – is this information CRAAP? The CRAAP test was developed by Meriam Library at California State University to help students think critically about the sources of information they had identified.  Some of the key questions to consider when evaluating information sources are:

  • Currency
    • When was the information published?
    • Has it been revised or updated?
  • Relevance
    • Who is the intended audience?
    • Is the information at an appropriate level (i.e. not too advanced/basic?)
    • How well does the information relate to your topic?
  • Authority
    • Who is the author/publisher/source/sponsor?
    • Is the author qualified to write on this subject?
  • Accuracy
    • Is the information supported by evidence?
    • Has the information between reviewed or refereed?
    • Are there any spelling/grammatical errors?
  • Purpose
    • Is the information fact, opinion or propaganda?
    • Is it objective and impartial?
    • Are there any religious/political/cultural/personal biases?
    • Is it trying to sell a product?

How we can help

While the increasing availability of information via the internet and the growth of open access content has undoubtedly been a positive development, it goes hand in hand with the need for users to critically assess information sources.

Here in the Idox Information Service, we take pride in our database of high quality resources. Our Researchers select only the best quality resources from hundreds of verified sources to populate our database. Our research database has been recognised by the Alliance for Useful Evidence as a key tool within the UK.

Each week, we support policy and decision makers by providing the latest research and evidence on a range of public policy issues – both through our current awareness services, and through bespoke literature reviews. In doing so, we hope to contribute in our own small way to the wider drive to improve the use of evidence in public policy decision making.

As we have seen, the use of inaccurate or misleading information can have significant real-world consequences.  The need for authoritative, accurate and relevant research has never been greater.


Find out more about the Idox Information Service and our subscription services here.

Follow us on Twitter to see what developments are interesting our research team.

Denmark’s digital ambassador: should the UK be following suit?

 

By Steven McGinty

On 26 January, the Danish Ministry of Foreign Affairs announced that they would be appointing the world’s first ‘digital ambassador’ to act as the nation’s representative to major technology companies, such as Google, Apple, Facebook and Amazon.

At a conference on the future of the Foreign Service, the Foreign Minister, Anders Samuelsen, explained that:

Denmark must be at the forefront of technological development. Technological advances are making such a great impact on our society that it has become a matter of foreign policy. I have therefore decided to announce the appointment of a digitisation ambassador.

In a follow up interview with Danish newspaper Politken, Mr Samuelsen expressed his belief that multinational technology giants “affect Denmark just as much as entire countries”. He highlighted the examples of Apple and Google whose market values are so large that if they were countries they would only narrowly miss out from inclusion in the G20 – the global forum for cooperation between the world’s 20 major economies.

As a result of this economic strength, together with tech firms’ impact on the everyday lives of citizens, Mr Samuelsen argues that the technology sector should be treated as a form of ‘new nation’, which Denmark must develop closer relationships with.

Cooperation between nation states and the technology sector

Technology companies are becoming involved in activities that were once reserved for nation states. For example, Mr Samuelsen’s Liberal party accepts donations in Bitcoin – an online currency which challenges the state’s role as the only issuer of legal tender. And Microsoft have signed a partnership agreement with the French Ministry of Education to provide teacher training, in order to prepare teachers for running special coding classes.

The technology industry argues that it is better placed than national governments to provide effective digital services, at cheaper prices. In terms of national security, computer engineering expert and academic, Jean-Gabriel Ganascia, argues that this is probably the case. Mr Ganascia highlights that Google and Facebook have vast image databases that enable them to use facial recognition software far better than any national security service. Therefore, countries have started working with technology companies on a variety of crime and public safety issues.

Citizens are also spending greater amounts of time on social media platforms. In an interview with The Washington Post, Mr Samuelsen stated that more than half of the world’s data has been created in the past two years (much of this from major platforms such as Facebook). This trend has implications for the privacy of citizens and the spreading of false information, a phenomena that has been labelled ‘fake news’. These issues are fundamentally important for citizens and nation states, and are likely to increase cooperation between countries and the technology sector.

Australia’s Ambassador for Cyber Affairs

Although Denmark will be the first country to introduce a digital ambassador, another government has made a similar appointment. In January, Dr Tobias Feakin was appointment as Australia’s Ambassador for Cyber Affairs. His role focuses on cyber-security, but also includes issues such as censorship and promoting internet access. At this stage, it’s unclear whether Dr Feakin will have direct contact with technology companies and whether this relationship will involve discussions over economic issues such as taxation.

Is a digital ambassador necessary?

Not everyone, however, is buying into the appointment of a government representative focused solely on digital issues. Technology journalist, Emma Woollacott, believes that it’s a ‘terrible idea’.

According to Ms Woollacott, Denmark already has a good relationship with technology companies, highlighting that Facebook has recently announced plans to build a new data centre in Odense, creating 150 new permanent jobs. These views may have some merit, as Mr Samuelsen has confirmed that the deal between the Foreign Ministry and Facebook was the result of three years of behind-the-scenes work.

Ms Woollacott also argues that Denmark is setting a worrying precedent by equating a private company to a nation state.  In her view, the importance of the technology sector could have been acknowledged through hiring knowledge staff, rather than granting it a ‘unique political status’.

However, Professor Jan Stentoft, who researches the insourcing of technological production to Denmark, believes creating the ambassadorial post is a good idea. He explains:

We have much to offer these companies, but Denmark is a small country, and we obviously need to make ourselves noticed if we are to attract them to the country.

Marianne Dahl Steensen, CEO of Microsoft Denmark, also welcomed the creation of a digital ambassador position, but did acknowledge that the company ‘can hardly be equated with a nation’.

Should the UK introduce a digital ambassador?

By introducing a digital ambassador, Mr Samuelsen is taking a pragmatic approach to ensure Denmark is a key player in the international digital economy, as well as attempting to manage the impacts of an increasingly digital society.

Although appointing an ambassador for the technology sector poses philosophical and ethical questions, the UK should closely monitor how this new role develops and the potential benefits (and challenges) it brings for Denmark. In particular, if the new role is able to improve dialogue between technology companies and the security services on matters such as privacy, or help address the sector’s need for digitally skilled workers, then maybe introducing a digital ambassador is something worth exploring.


Follow us on Twitter to see what developments in public and social policy are interesting our research team. If you found this article interesting, you may also like to read our other digital articles

The Autumn Statement 2014

Pound coin

By Alex Thomas

Chancellor George Osborne’s 2014 Autumn Statement was delivered at 12.30 GMT today in the House of Commons. The Autumn Statement is an opportunity for the Chancellor to update MPs on the Government’s plans for the economy based on the latest forecasts from the Independent Officer for Budget Responsibility (OBR).

As many predicted the chancellor committed to further tightening public finances within the statement. He went on to say that if the Conservatives remain in government after the May General Election there will be substantial savings in public spending. Continue reading

Google’s driverless car may succeed where Sinclair failed

by James Carson

Readers old enough to remember the Sinclair C5 might have experienced a déjà vu moment at the news that Google is developing a driverless car. As with the C5 scepticism seemed to surround the story, with followers of the Financial Times Twitterfeed giving full vent to their doubts:

Continue reading