Assessing information quality: sorting the wheat from the CRAAP

The rapid expansion of the internet has enabled users to access unprecedented amounts of information.  However, not all of this information is valid, useful or accurate. In the world of ‘post-truth’ and ‘fake news’, the ability to critically assess information and its source is an essential skill.  Let’s consider what this means for public policy.

Growth of evidence-based policy

The need to assess research evidence is no longer limited to academics and scientists. The shift towards evidence-based decision-making means that policy makers and decision makers at every level now need to incorporate evidence into policy and practice.

The growth of the randomised controlled trials movement in public policy also reinforces the need for decision makers to be familiar with a range of research approaches.

In the UK, the What Works Network and the Alliance for Useful Evidence both work towards encouraging and improving the use of evidence to improve public services. Similarly, the Evidence Matters campaign seeks to promote the importance of evidence in policymaking, and tackle the misuse of research findings.

Publication doesn’t guarantee quality

While most of us are aware of the risks of encountering ‘fake news’ online, relying uncritically upon Google as an information source can leave one falling foul of ‘predatory’ open access journals, which masquerade as legitimate, peer-reviewed publications.

In recent years, there has been a boom in articles being published open access. There are now a vast number of good quality, open access publications in just about every subject imaginable.  Overall, this has been a positive development – who can argue with making more research free and easily accessible?

Open access not only has an ethical dimension – in many situations, it is also an obligation.  The UK government has already committed to ensuring that all publicly funded research is made available via open access.

However, the proliferation of open access material has led to a new problem – that of predatory open access journals. These journals operate using a business model that involves charging publication fees to authors, without providing the editorial and publishing services associated with legitimate journals. They may even include fake editors or members of the editorial board. Librarian and researcher, Jeffry Beall, has compiled a rather impressive list of ‘potential, possible, or probable predatory scholarly open-access publishers’.

The quality of articles published in predatory journals is therefore questionable. Recent (rather entertaining) examples of how unreliable such journals can be include the neuroscientist who managed to trick a number of scientific journals into publishing a nonsensical piece of research complete with a number of Star Wars references, including the authors Dr Lucas McGeorge and Dr Annette Kin, and the article reporting the case of a man who develops ‘uromycitisis poisoning’, inspired by a 1991 episode of Seinfeld (the actual article is still online).

Is this information CRAAP?

So how do you assess the quality of a piece of information?

One way to do this is to ask yourself – is this information CRAAP? The CRAAP test was developed by Meriam Library at California State University to help students think critically about the sources of information they had identified.  Some of the key questions to consider when evaluating information sources are:

  • Currency
    • When was the information published?
    • Has it been revised or updated?
  • Relevance
    • Who is the intended audience?
    • Is the information at an appropriate level (i.e. not too advanced/basic?)
    • How well does the information relate to your topic?
  • Authority
    • Who is the author/publisher/source/sponsor?
    • Is the author qualified to write on this subject?
  • Accuracy
    • Is the information supported by evidence?
    • Has the information between reviewed or refereed?
    • Are there any spelling/grammatical errors?
  • Purpose
    • Is the information fact, opinion or propaganda?
    • Is it objective and impartial?
    • Are there any religious/political/cultural/personal biases?
    • Is it trying to sell a product?

How we can help

While the increasing availability of information via the internet and the growth of open access content has undoubtedly been a positive development, it goes hand in hand with the need for users to critically assess information sources.

Here in the Idox Information Service, we take pride in our database of high quality resources. Our Researchers select only the best quality resources from hundreds of verified sources to populate our database. Our research database has been recognised by the Alliance for Useful Evidence as a key tool within the UK.

Each week, we support policy and decision makers by providing the latest research and evidence on a range of public policy issues – both through our current awareness services, and through bespoke literature reviews. In doing so, we hope to contribute in our own small way to the wider drive to improve the use of evidence in public policy decision making.

As we have seen, the use of inaccurate or misleading information can have significant real-world consequences.  The need for authoritative, accurate and relevant research has never been greater.


Find out more about the Idox Information Service and our subscription services here.

Follow us on Twitter to see what developments are interesting our research team.

Working longer – the reality ‘behind the headlines’

Senior businessman in office working on laptop

By Heather Cameron

With no shortage of headlines highlighting the record employment rate in the UK, and the increasing number of older workers widely reported, it may seem that the outlook for the ageing workforce is a rosy one. But do these headlines hide the reality?

Recent analysis from Age UK argues that the headline employment rate doesn’t tell the whole story about working longer, “making it an insufficient – and even misleading – tool for public policy decision-making”.

The statistics

The most recent official figures show that the employment rate (the proportion of people aged from 16 to 64 who are in work) is the joint highest since comparable records began in 1971, at 74.8%, while the unemployment rate is the joint lowest since 1975.

Data also shows that the employment rate for people aged 65 and over has indeed increased since the 2008 recession. It is currently at 10.4%, up from 7.3% in 2008.

Age UK has also recognised the increase in employment rates for older people, noting that, in fact, the older the age group, the greater the increase in employment. However, the average number of hours worked has declined since the recession, indicating a more complex and perhaps less reassuring situation than the one portrayed in the media.

The biggest drop was for 50-54 year old men, whose average hours declined by 29%. For men aged 60-64, the average number of hours declined by 8 hours (over 22%), while women aged 50-54 experienced a fall of 18%.

The only age group not to see a decline was women aged 60-64, which is likely to be as a result of the raising of the State Pension age.

Choice or necessity?

The change in the State Pension age was justified on the grounds that it gave people more choice and more scope to continue working if they wished to.

A recent CIPD survey found that the most common reason for wanting to work past 65 is that employees believe it will help keep them mentally fit, followed by wanting to be able to earn a sufficient income to continue to do the things they enjoy.

As Age UK suggests, it may be that the reduction in working hours is a good sign if it is due to older workers choosing to wind down their hours, maybe to enable them to juggle other responsibilities such as caring for their grandchildren, while still earning a wage.

However, the research suggests it may be less through choice and more as a result of the changing labour market such as increasing underemployment (working less hours than they would choose to) or increasing insecure working practices driven by the rise in self-employment and the ‘gig economy’.

As it is likely working fewer hours will mean less income, this could be a cause for concern since it will be more difficult for older workers to maintain their standard of living until they meet the State Pension age and for them to save enough for retirement.

Another issue highlighted by the CIPD, is that most employees don’t believe their organisations are prepared to meet the needs of the over 65s, suggesting that there is a need for employers to also review their practices in terms of managing older workers.

Final thoughts

It is clear that while, for some, choosing to work beyond the traditional retirement age will be a lifestyle choice, for many it will be a necessity. Any substantial reduction in working hours for these older workers could consequently pose a real issue.

It would therefore make sense for policy makers to heed the warning from Age UK not to rely on the headline rate of employment for older workers, and rather look beyond it to the reality of many struggling to get and keep the secure, well paid jobs they want and need.


If you enjoyed reading this, you may be interested in reading our previous post on the pros and cons of the gig economy.

Follow us on Twitter to see what developments in public and social policy are interesting our research team.

Christmas without a home

By Heather Cameron

Last week saw George Clooney launch a campaign to feed the homeless at Christmas by donating the first £5.

When visiting Edinburgh’s branch of Scotland’s not-for-profit sandwich shop, Social Bite, last month, Clooney filmed a video clip on a staff member’s phone in which he pledged the first £5 donation to Social Bite’s £5 Christmas dinner appeal.

First Minister Nicola Sturgeon, Olympic star Sir Chris Hoy, Labour leader Jeremy Corbyn, comedian Rob Brydon, broadcaster Chris Evans, and Scotland football manager Gordon Strachan have also pledged their support.

Last year’s campaign raised enough money to buy 36,000 meals to feed homeless people in Glasgow, Edinburgh and Aberdeen for the whole year. Just 24 hours after Clooney’s initial donation, £165,000 was raised with over 33,000 people donating.

How many homeless?

Considering that Scottish local authorities logged 35,764 statutory homelessness assessments in 2014/15, of which 28,615 were assessed as ‘legally homeless’, this figure is impressive.

Nevertheless, the actual number of homeless people is likely to be far greater.

The latest data for Scotland suggests that 50,000 adults experience homelessness each year.

Shelter has estimated that 109,000 children in Britain will be homeless this Christmas, with nearly 5,000 of them in Scotland. According to the Scottish arm of charity, this is a 15% increase on last year’s figure, which:

“is simply not good enough and a badge of shame for such a relatively wealthy country”…The increased number of homeless children indicates a growing bottleneck of families stuck in temporary accommodation due to the major shortage of affordable housing across Scotland.”

upset boy against a wall

Government figures show that the number of people in temporary accommodation has grown over the past five years despite more than £1bn being spent on homelessness since 2010.

And these figures don’t include the hidden homeless that evade official statistics. According to Crisis, “official homelessness figures are masking the true scale of the problem”.

People living in overcrowded accommodation, shared accommodation, young single people and those in ‘concealed households’ (including groups/families/single people who are unable to form separate households and forced to live with others) can all be hidden from the system. And as local authorities only have to accommodate ‘statutory’ homeless people, these people are often hidden from support and advice as well as statistics.

Positive practice

As Social Bite’s Christmas dinner campaign shows though, good work is being done. Many homeless charities work tirelessly across the UK to provide services for people at Christmas time and indeed throughout the year.

The Salvation Army provides support and friendship to the homeless and other vulnerable people and its Christmas appeal for donations of time, money and gifts has seen much success over the years.

Crisis runs their Crisis at Christmas event across the country providing hot meals, fun activities, entertainment, health care and advice for the homeless. This year they have Christmas centres in Birmingham, Coventry, Edinburgh, London and Newcastle.

A new community initiative led by students at Darlington College aims to give homeless people in the town a Christmas lunch at the college, a cooking demonstration and festive meal at a local restaurant.

And as well as providing dinners for homeless people in Scotland, Social Bite will also be using donations to provide food and clothing packs for refugees in camps in Calais, the Serbia/Croatia border, and Lesbos.

Final thoughts

With the sheer scale and complexity of the issue, of course it won’t be possible for such initiatives to reach every homeless person. And with the combination of cuts to welfare and a severe lack of affordable housing across the UK, many more families are likely to face a fight to keep roofs over their heads.

So while we settle down to enjoy the festive period with our nearest and dearest, perhaps we should all spare a thought for those who simply seek the gift of shelter.


Further reading: if you liked this blog post, you might also want to read our previous blog on Britain’s hidden homeless. 

Our popular Ask-a-Researcher enquiry service is one aspect of the Idox Information Service, which we provide to members in organisations across the UK to keep them informed on the latest research and evidence on public and social policy issues. To find out more on how to become a member, get in touch.

Follow us on Twitter to see what developments in policy and practice are interesting our research team.

Graduating into a brighter future?

Image from Flickr user Luftphilla, licensed under Creative Commons

by Stacey Dingwall

Post-recession, the employment situation for UK graduates has not been great. Following the economic crash, headlines and statistical releases alike screamed about how bad it was out there for the recently graduated. Graduates were portrayed as either unemployed or underemployed, i.e. forced to accept roles for which their qualifications were not required or unpaid internships. With the end of the recession however, has the situation improved?

The graduate job recession

In 2010, the number of graduates in full-time work, three months post-graduation was 51% – its second-lowest level since 2003 (57%). And in 2009 The Association of Graduate Recruiters (AGR) was reporting that the number of graduate vacancies being advertised had fallen by up to a quarter since before the recession.

With record numbers of graduates now competing for each vacancy, and competing not only with their own graduating class but also with earlier cohorts, it could have been concluded that the era of the traditional graduate employment route was on its way out.

A return to form?

According to recent figures, however, things are looking up. Previewing the second 2015 update of its Graduate Recruitment survey, AGR describes the current graduate market as ‘buoyant’, and notes that the findings of the previous survey indicated an 11.9% increase in graduate vacancies on the previous year. These findings are backed up by the September 2014 edition of the Higher Education Careers Services Unit’s (HESCU’s) What do graduates do, which described the employment prospects for 2012/13 graduates as ‘dramatically improved’ compared to those of their immediate predecessors, with their unemployment rate six months after graduating down at 7.3% from the previous year’s 8.5%.

Additionally, the most recent release of the High Fliers graduate recruitment study suggests that those graduating in 2015 are doing so into the “most attractive graduate market in a decade”, and predicts 8% more vacancies than the previous year. It also notes that the class of 2015 are the first to graduate having paid tuition fees of up to £9,000 per year; this has led to the end of the image of students merely partying their way through their time at university, with the majority now focused on securing a promising career for themselves from as early as first year.

The new face of the graduate job

The prospect of graduating with tens of thousands of pounds of debt appears to be proving quite the motivation for today’s students. Rather than waiting until their final year to seek out internships and careers advice, High Fliers reports that firms are now taking on first year undergraduates in placement roles. Building up a relationship with a desired employer as early as possible is now the key way of securing a job post-graduation according to the report, with those with little or no work experience described as having “no chance” of receiving the offer of a place on a firm’s graduate programme.

AGR’s chief executive Stephen Isherwood has also pointed towards this trend, suggesting that graduate recruitment is being replaced with ‘student recruitment’, as those leaving university face competition from those still at university who have already been hired by employers for apprenticeships or have succeeded in finding an employer to sponsor them through the rest of their studies.

Another issue, as highlighted by Gerbrand Tholen, is the changing definition of what constitutes a graduate job. He notes that the previous understanding of what made a graduate occupation (those that combined expertise, strategic and managerial skills and interactive skills) has been abandoned in favour of defining the extent to which the role utilises specialist, orchestration or communication expertise.

This has led to a blurring between the lines of graduate and non-graduate roles, and also issues with compiling official statistics on the number of graduates employed in each arena. In 2014, the director of High Fliers, Martin Birchall, criticised the Office for National Statistics for not updating their definition of a graduate job since 2002, after they released data which suggested that 47% of recent graduates were not working in jobs which required a higher education qualification. This issue is further compounded by the issue of ‘over-education’ and ‘under-employment’, and the question of whether employers have been able to benefit from a more highly skilled workforce.

The graduate class problem

An important thing to keep in mind is that reporting on graduate labour market trends tends to focus primarily on the most general of findings – considering graduates as a homogenous group. This is particularly true in terms of the social backgrounds of graduates: research has found, and is continuing to find, significant differences in the labour market experience for graduates from working class backgrounds and their more socially privileged backgrounds. Until this much wider issue of a lack of social mobility within the graduate labour market can be addressed, it is perhaps too early to describe the situation as ‘buoyant’ – at least for everyone.


 

The Idox Information Service can give you access to a wealth of further information on education and employment trends; to find out more on how to become a member, contact us.

Further reading on the topics covered in this blog *

‘Graduate jobs’ in OECD countries: development and analysis of a modern skills-based indicator (LLAKES research paper 53)

What do graduates do? Employment review, IN Graduate Market Trends, Autumn 2014, pp12-14

Graduates’ experiences of non-graduate jobs: stop gaps, stepping stones, or dead ends?, IN Graduate Market Trends, Summer 2014, pp6-8

‘You have to be well spoken: students’ views on employability within the graduate labour market, IN Journal of Education and Work, Vol 27 No 2 Apr 2014, pp179-198

The gap between the proportion of young graduates from professional backgrounds who go on to a “graduate job” six months after graduating and young graduates from non-professional background

We need to talk about graduates: the changing nature of the UK graduate labour market

*Some resources may only be available to members of the Idox Information Service

New Idox research – Data Mining to inform public policy

By Susan Lomax, Data Scientist, Knowledge Transfer Partnership placement

The latest “new” thing in the world of data mining is using “Big Data” to inform public policy. Using data mining methods, we can aid evidence-based decision making by learning what the data can tell us and using this to write or implement policy. Idox are now exploring these methods to look at opportunities for our public policy and research members.

Investigation indicates that using data in this way is in its infancy, where data mining methods are in the process of being used, but so far, very little is completed. Published examples include, London Borough of Newham’s property data, which has been combined with numerous other datasets and mined to examine change in property tenure in order to support, amongst other things, their housing management services. The University College London mined Oyster Card data in order to minimize cost for travellers using public transport and to encourage public transport use. The first stage of the research will be exploring what can be done and what would be useful to members.

As a new member of the Idox staff, I am on a scheme known as Knowledge Transfer Partnership (KTP), which helps companies engage in this type of research and development. The scheme is celebrating its 40th Anniversary this year, having first been formed in 1975 as the Teaching Company Scheme. The KTP program is funded by 17 public sector organisations and led by Innovate UK, formally the Technology Strategy Board. The aim is to support UK businesses wanting to improve their competitiveness, productivity and performance by accessing the knowledge and expertise available within UK Universities and Colleges.

Traditionally taking place in engineering and manufacturing industries, they have now branched out into ICT, looking at data analysis, and creative industries such as design, fashion, music and video games businesses. There are currently 800 partnerships across the UK.

Our research partnership includes an academic institution and The University of Salford, is on hand to provide support and guidance. It has an outstanding record with regard to innovation, enterprise and skills. The Informatics Research Centre builds on history, success and achievements of research in Computer Science and Information Systems over the last 30 years.

Data mining is a process to discover patterns in large datasets. Its roots are in disciplines such as artificial intelligence, machine learning, statistics and database systems. Its overall goal is to extract information from data and make this understandable, so that it can be used to make decisions. A popular book “Data mining: Practical machine learning tools and techniques with Java” has information about the most common data mining methods.

The three main data mining methods we will be trying are association rules, classification and clustering and we will be exploring these in the research.

  • Association rule learning searches for relationships between variables (or attributes) in the dataset. A most popular example is a supermarket finding out which products their customers buy together and use this information for marketing purposes. This is also known as market basket analysis.
  • Classification is when a dataset has examples grouped into known classes; the task is to assign a new example to one of these known classes. A well-known algorithm performing this task is the Decision Tree algorithm C4.5.
  • Clustering performs a similar task to classification but with clustering we don’t have an assigned ‘class’. A technique known as k-nearest Neighbour is a popular method. Other main tasks are regression, summarization and anomaly detection.

Although the research is explorative at the moment, I hope to keep you updated with our progress throughout the project. If you have any thoughts or want to find out more, please get in touch.


The Idox Information Service can give you access to a wealth of further information on data and knowledge management. To find out more on how to become a member, contact us.

Further recent reading*

Classification

Association rule

Measuring transit use variability with smart-card data

Digital councils

*Some resources may only be available to members of the Idox Information Service

Zero future for zero hours in a fair economy?

By Stacey Dingwall

The Office for National Statistics (ONS) has released its second annual update on the number of people employed on zero hours contracts, which suggests that in August 2014 UK firms were employing 1.8 million people on such contracts.

What is a zero hours contract?

According to Acas, the term ‘zero hour contract’ (although not defined in legislation) can be understood as “an employment contract between an employer and a worker, which means the employer is not obliged to provide the worker with any minimum working hours, and the worker is not obliged to accept any of the hours offered”.

Use of the contracts has been a highly controversial issue in recent months, with high-profile retailers such as Sports Direct (who employ 90% of their part time staff on zero hours contracts) coming in for criticism of their “exploitation” of their employees. The sports retailer is also facing legal action from hundreds of their workers due to their exclusion from the company bonus scheme, thanks to the nature of their contracts.

Increasing or not?

The ONS’ first Analysis of Employee Contracts that do not Guarantee a Minimum Number of Hours found that between January and February 2014, 1.4 million UK workers were employed on zero hours contracts. Despite the inevitable headlines depicting the new figure as a direct increase from the 2014 analysis, the ONS was careful to warn against this in its latest analysis, noting that it covers a different time of year than the first release therefore the number of contracts reported may be affected by seasonal factors.

The latest release also includes data from the Labour Force Survey (LFS), which indicates that the number of people employed on zero hours contracts in their main employment, between October and December 2014, was 697,000 or 2.3% of all people in employment. The figure for the same period the year before was 586,000 or 1.9% of people in employment although again, the ONS are careful to stipulate that they can’t be certain how much of this ‘increase’ is due to greater recognition of what constitutes a zero hours contract, as opposed to new contracts.

The Economic Research Council suggested that a lot of the jobs that have been created recently have come with much less security and guaranteed pay. And the UK Commission for Employment and Skills have noted that 33% of people on zero hour contracts would like to work more hours (either in their current job or in a different one), compared to just 13% of people not on a zero hour contract.

Zero hours and the general election

The issue of the use of zero hours contracts looks set to become a key feature of parties’ campaigns in the upcoming general election. Current Secretary of Business, Innovation and Skills – Liberal Democrat Vince Cable – has already put forward legislation (clause 151 of the Small Business Enterprise and Employment Bill, currently before the House of Lords) which would see exclusivity clauses in contracts (which prevent those employed on zero hours contracts from seeking additional work to supplement their income) banned.

The Conservative Work and Pensions Secretary, Iain Duncan Smith, has however defended the contracts, arguing that they “provide people with a flexible way of working and the freedom to arrange jobs around other commitments” and “allow employers to be competitive in response to market trends”.

What of the other parties? Labour has vowed to “end exploitative zero hours contracts” and introduce “new rights” to employees on such contracts, however has stopped short of proposing to ban employers from offering them altogether. Somewhat embarrassingly for the party, figures released by the Independent Parliamentary Standards Authority (IPSA) and seized on by the tabloids, have indicated that over 30 Labour MPs employed staff on zero hours contracts in 2014.

The Green Party is firmly against the use of zero hours contracts altogether: leader Natalie Bennett has been calling on the government to place an outright ban on them since 2013. UKIP leader Nigel Farage has also criticised the long-term use of zero hours contracts by employers, and has called for large employers to be subject to a code of conduct as to how they are applied.

Zero hours contracts aren’t a financial necessity

In these times of budget cuts, many local authorities have argued that they have no choice but to offer some of their workers zero hours contracts. One area in which this has been particularly prevalent is in the provision of social care, with some employees paid on a ‘time and task’ basis, i.e. only for the amount of time they actually spend with a client, which can be as little as 15 minutes in some cases.

In 2012, Southwark Council took the decision to move away from this approach, after feedback from care workers and service users indicated that it did not allow workers to carry out their duties with the required level of compassion. The Council carried out a review of their homecare services and found that extending the length of visits greatly helped in keeping service users healthy in their own homes and out of hospital and residential care. It also noted that the costs of providing longer visits had been ‘passed on’ to their care workforce over time through the use of zero hours contracts and, wishing to end this, announced that from October 2014 they would be eliminating their use altogether, and offering guaranteed hours of employment to their staff.

Immediate reaction to the release of the latest figures has been plentiful; it now remains to be seen whether it is reflected in party campaigns in the forthcoming general election.


The Idox Information Service has a wealth of research reports, articles and case studies on zero hours contracts and other employment issues. Items of interest include:

The decent jobs deficit: the human cost of zero-hours working in the UK

Give and take? Unravelling the true nature of zero-hours contracts

Zero hours contract: not all bad news

Zero-hours contracts: myth and reality

Flexibility or insecurity? Exploring the rise in zero hours contracts