A world of evidence … but can we trust that it is any good?

What is good evidence? And how can policymakers and decisionmakers decide what is working and what isn’t, when it comes to deciding where public money is spent and how?

These are the kinds of questions that models and tools such as randomised controlled trials and cost-benefit analysis attempt to answer. The government has also supported the development over the last five years of the What Works Network, which now consists of 10 independent What Works Centres. When talking about impact there’s also been a move to capturing and recognising the value of qualitative data.

As one of our key aims is to support and facilitate the sharing and use of evidence in the public sector, we were interested to read a new publication ‘Mapping the standards of evidence used in UK social policy’.

Standards of evidence

Produced by the Alliance for Useful Evidence, the research has found 18 different Standards of Evidence currently in use across UK social policy.

The report notes that over the last decade there has been increasing interest in grading effectiveness or impact against a level or scale. Typically, the higher up the scale, the more evidence is available. Theoretically this means that decision-makers can have higher confidence in deciding whether a policy or intervention is working.

While all the evidence frameworks generally aim to improve the use of evidence, the different goals of the organisations responsible can shape the frameworks in different ways. They can be used to inform funding decisions, to make recommendations to the wider sector about what works and what doesn’t, or as a resource to help providers to evaluate. And unfortunately this means that the same intervention can be assessed differently depending on which framework is used.

The Alliance for Useful Evidence concludes that while a focus on evidence use is positive, the diversity of evidence standards risks creating confusion. Suggested options for improving the situation include introducing an independent accreditation system, or having a one-stop shop which would make it easier to compare ratings of interventions.

Dissemination and wider engagement

The question of standardising evidence frameworks is just one part of a wider effort to increase transparency. As well as collecting evidence, it’s important that when public money has been invested in carrying out evaluations and impact assessments, that this evidence remain accessible over the longer term and that lessons are learned. It can often seem that government departments have very short organisational memories – especially if they’ve suffered a high churn of staff.

Two projects which we support in Scotland are focused on increasing the dissemination and awareness of evaluation and research evidence. Research Online is Scotland’s labour market information hub. Produced by ourselves and Skills Development Scotland, the portal brings together a range of statistics and research and acts as the centre of a community of practice for labour market researchers, practitioners and policy-makers.

Meanwhile Evaluations Online is a publicly accessible collection of evaluation and research reports from Scottish Enterprise. The reports cover all aspects of Scottish Enterprise’s economic development activities – some of the latest added to the site cover megatrends affecting Scottish tourism, innovation systems and the gender gap, and the commercial flower-growing sector in Scotland.

When working within the policy world it can be easy to suffer from fatigue as ideas appear to be continually recycled, rejected and then revisited as policy fashions change and political parties or factions go in and out of power. The spotlight, often driven by the media, will shine on one hot policy issue – for example, moped crime, cannabis legislation or health spending – and then move on.

Online libraries of evaluations and research reports are one tool which can help support a longer-term culture of learning and improvement within the public sector.

Evidence Week 2018

Inspired by similar objectives, Evidence Week runs from 25th to 28th June 2018 and aims to explore the work of parliamentarians in seeking and scrutinising evidence. It will bring together MPs, peers, parliamentary services and the public to talk about why evidence matters, and how to use and improve research evidence.

This may be the start of wider knowledge sharing about standards of evidence, to help those using them to improve their practice.


The Knowledge Exchange is a member of the Alliance for Useful Evidence. Our databases are used by government and the public sector, as well as private-sector consultancies, to keep abreast of policy news and research in social and public policy.

Research Online: an expert source of information on the Scottish labour market

ro-pic

One of the Knowledge Exchange’s key aims is to support the use of evidence and research in public policy and practice. Our Information Service database has been recognised by the Alliance for Useful Evidence in its Evidence Ecosystem which illustrated the diversity of organisations involved in supporting evidence use in the government and public sector. But we also support two other sector-specific research portals – Research Online and Evaluations Online.

Here we introduce Research Online, which first launched over 14 years ago and which we have worked with Skills Development Scotland to develop and update ever since.

Scottish labour market intelligence

Research Online is Scotland’s labour market hub. The portal provides an authoritative source of labour market research and analysis relevant to Scotland and supports evidence-based policy making in the Scottish labour market.

Before Research Online was created, research suggested that although useful labour market research and analysis was undertaken within Scotland by a large range of organisations, there was no single dissemination source.

Therefore, a requirement existed for a portal that clearly identified current labour market intelligence (LMI), provided a common understanding of current gaps and provision in areas including labour supply and skills, and focused action to ensure LMI met Scottish user needs.

Research Online was conceived to improve access to this wealth of intelligence.

The most comprehensive collection of labour market intelligence

The portal now contains thousands of documents on a range of labour market topics including:

  • Employment;
  • Skills and training;
  • Unemployment;
  • Entrepreneurship;
  • Vocational education and training;
  • Workforce development; and
  • Equal opportunities.

The material available on the portal includes research, policy, analysis, discussion and sectoral and geographic profiles. Our team sources the latest research and policy documents from a wide range of sources, including academic journals, government departments and agencies, labour market research centres and material sent in directly by key organisations in Scotland and the wider UK. The available material includes grey literature, government policy and up-to-date academic research.

Research Online also incorporates a current awareness service that alerts registered users to new material on a fortnightly basis. It also has integrated reading list functionality.

Free to access

Research Online can be accessed by anyone, free of charge. You can browse the material here without registering, as well as create reading lists to be accessed at a later date or shared with colleagues.

If you would like to sign-up for a range of current awareness alerts that keep you up to date on a variety of labour market topics, covering both Scotland and the wider UK, you can do so here.

Our shared vision is for Research Online to be recognised as a key dissemination mechanism by Scotland’s producers of labour market intelligence and to be at the centre of a community of practice for labour market researchers, practitioners and policy-makers.


You can find out more about the projects The Knowledge Exchange team has been involved in, and the consultancy services we offer, here.

Evaluations Online: evaluating economic development activity in Scotland

by Stacey Dingwall

Recently we profiled Research Online, one of the two research portals managed by the Knowledge Exchange team. In this blog, we focus on Evaluations Online.

Economic development activity in Scotland

Evaluations Online is a public portal providing access to a collection of evaluation and economic development research reports commissioned by Scottish Enterprise. Scottish Enterprise is Scotland’s main economic development agency and a non-departmental public body of the Scottish Government.

Idox won the contract to deliver Evaluations Online in 2007. The team developed a site which utilises a publishing platform designed specifically to deal with research material. Users can easily navigate to and assess the relevance of material thanks to specially-written abstracts and structured search functions based on a bespoke classification and record structure.

The site now contains over 500 evaluation and research reports commissioned by Scottish Enterprise, dealing with different aspects of economic development activity such as business support, investment, sector growth and improving skills. All of the reports are publicly accessible and free to access.

In 2011, the team won a further contract to refresh and improve the site, focusing on how the site could be refined to better meet the needs of key user groups including economic development policy-makers and practitioners across Scotland. In the last quarter of 2016, the reports hosted on the site were accessed over 30,000 times.

The importance of evaluation

One of the key reports hosted on Evaluation Online is the annual review of the risk capital market in Scotland. Scottish Enterprise commissions the report annually in order to consider the scale of new investment flows. The findings are also used to inform the nature of Scottish Enterprise interventions in the Scottish early stage risk capital market, such as the Scottish Co-Investment Fund and Scottish Venture Fund.

Scottish Enterprise commissions evaluations of projects and programmes each year in order to identify their contribution towards economic growth in Scotland, and particularly in terms of their impact on gross value added (GVA) and employment. As the findings of the evaluations inform decisions about public spending, it’s important that all of the appraisal and evaluation work is of a high technical standard.

We’ve highlighted the importance of evidence and evaluation on the blog several times before. It’s worth repeating that repositories of evidence can help bring about better policy in a number of ways:

  • improve accountability by making it easier for people to scrutinise the activities and spending of public sector organisations – this helps organisations meet Freedom of Information responsibilities;
  • improve the visibility and therefore the impact of evidence;
  • help identify gaps in evidence by making it easier to compare research findings; and
  • increase our understanding of what works (‘good practice’), not only in the activities covered, but also in evaluation and research methods.

We’re proud to support Scottish Enterprise in the dissemination of their evaluation and research output, through a portal which they believe increases the return on these activities.

You can find out more about the projects The Knowledge Exchange team has been involved in, and the consultancy services we offer, here.

 

Research Online: Scotland’s labour market hub

ro-pic

by Stacey Dingwall

As well as the Idox Information Service, the Knowledge Exchange Team manages two other research portals – Research Online and Evaluations Online.

This blog focuses on Research Online, which we developed over 13 years ago and have worked with Skills Development Scotland to maintain and update ever since.

Scottish labour market intelligence

Research Online is Scotland’s labour market hub. The portal provides an authoritative source of labour market research and analysis relevant to Scotland and supports evidence-based policy making in the Scottish labour market.

Before Research Online was created, research suggested that although useful labour market research and analysis was undertaken within Scotland by a large range of organisations, there was no single dissemination source.

Therefore, a requirement existed for a portal that clearly identified current labour market intelligence (LMI), provided a common understanding of current gaps and provision in areas including labour supply and skills, and focused action to ensure LMI met Scottish user needs.

Research Online was conceived to improve access to this wealth of intelligence.

The most comprehensive collection of labour market intelligence

The portal now contains thousands of documents on a range of labour market topics including:

  • Employment;
  • Skills and training;
  • Unemployment;
  • Entrepreneurship;
  • Vocational education and training;
  • Workforce development; and
  • Equal opportunities.

The material available on the portal includes research, policy, analysis, discussion and sectoral and geographic profiles. Our team sources the latest research and policy documents from a wide range of sources, including academic journals, government departments and agencies, labour market research centres and material sent in directly by key organisations in Scotland and the wider UK. The available material includes grey literature, government policy and up-to-date academic research.

Research Online also incorporates a current awareness service that alerts registered users to new material on a fortnightly basis. It also has integrated reading list functionality.

Free to access

Research Online can be accessed by anyone, free of charge. You can browse the material here without registering, as well as create reading lists to be accessed at a later date or shared with colleagues.

If you would like to sign-up for a range of current awareness alerts that keep you up to date on a variety of labour market topics, covering both Scotland and the wider UK, you can do so here.

Our shared vision is for Research Online to be recognised as a key dissemination mechanism by Scotland’s producers of labour market intelligence and to be at the centre of a community of practice for labour market researchers, practitioners and policy-makers.

You can find out more about the projects The Knowledge Exchange team has been involved in, and the consultancy services we offer, here.

 

Idox: enabling transformation, collaboration and improvement

Idox_logo 800 x 800 jpeg

If you follow this blog regularly then you’ll know that we write on all areas of public and social policy. What you might not realise though is that our Knowledge Exchange team is just one part of a much wider business – Idox – providing specialist information and data solutions and services.

I’ve been working with Idox for about four years, but I’m still topping-up my knowledge about the organisation. Last week, at the company’s end-of-year get-together, my brain was like an overworked sponge as it tried to absorb a multitude of facts, figures and achievements during two days of workshops and presentations (to say nothing of the informal chats in between the working sessions).

From this wealth of information, I’ve compiled a selection that I think conveys a flavour of the depth and diversity of Idox today.

Ten things you might not know about Idox…

  1. The Reading Room, which is the newest addition to the Idox family of companies, has developed digital solutions for a wide range of customers, including Porsche and Clarence House, and this year developed a virtual reality test drive app for Skoda.
  2. Idox’s recently-launched iApply service enables planning applications and building control consent to be applied for via a single source, streamlining the application process.
  3. The Idox GRANTFinder policy and grants database contains details of over 8000 funding opportunities.
  4. Real-time information delivered by Idox’s Cloud Amber keeps the travelling public up-to-date about transport services and helps manage traffic congestion.
  5. The Idox group currently employs almost 600 people in over 10 countries, including the UK, the Netherlands, Germany, the United States, India and Australia.
  6. The Idox Elections service not only ensured the smooth management of postal voting for the 2015 UK general election, but has also supported delivery of local authority and community council elections in the UK, as well as this year’s local elections in Norway.
  7. Idox has a strong presence in the compliance sector, raising awareness among managers and employees of the importance of complying with regulations, from corruption prevention and data privacy to occupational safety and cybersecurity.
  8. Idox Engineering Information Management, provides critical engineering document management and control applications to the oil and gas, mining, pharmaceutical and transport industries in 50 countries.
  9. CAFM Explorer, Idox’s computer aided facilities management software, supports building maintenance and property management for organisations in 45 countries, and recently partnered with the Hippodrome to help maintain one of London’s most popular attractions.
  10. From food safety monitoring to licensing taxis, Idox’s regulatory services help local authorities enforce the rules that keep us safe.

One more thing…

Finally, the meeting reminded me of one thing I already knew, and it’s to do with the part of Idox where I work – the Knowledge Exchange.

Over breakfast on the second morning, a colleague from McLaren talked about the difficulties in finding the right information on the web. Search engines only go so far, he said, providing too little or too much. This is where skilled intermediaries, such as Idox’s team of Research Officers, can make a difference, identifying, sorting and presenting information that people can use to make decisions, support arguments and advance their businesses.

The Idox event was an enjoyable, if exhausting, couple of days, and it demonstrated the many ways in which the company is supporting public, private and third sector work.

Clearly, there’s much more to learn about Idox.


Our popular Ask-a-Researcher enquiry service is one aspect of the Idox Information Service, which we provide to members in organisations across the UK to keep them informed on the latest research and evidence on public and social policy issues. To find out more on how to become a member, get in touch.

Follow us on Twitter to see what developments in public and social policy are interesting our research team.

The next big thing in management and organisational development … why your organisation should be considering reverse mentoring

A jigsaw of a handshake being completed ny two hands.

Think of the modern workplace and a number of features may spring to mind: technological innovations, flexible and remote working, to name a few. However, a more social factor is also at play: the multigenerational workforce. For the first time in history, it is now feasible that a workforce could comprise employees from four different generations, i.e. the World War II Generation (born 1929-1945), Baby Boomers (born 1946-1964), Generation X (born 1965-1979) and Generation Y/Millennials (born after 1980).

The characteristics of a generation

In 2012, Ashridge Business School identified that Generation Y (Gen Y) has grown up in an environment that is very different to previous generations. Additionally, their survey of managers from around the world found that Gen Y:

  • comes to the workplace with different skills;
  • is motivated by different things;
  • thinks differently about learning and development; and
  • approaches work relationships differently.

As noted by Steve Regus, writing for HR magazine in 2012, Gen Y also take a less hierarchical attitude towards work, and place a high importance on mentoring and feedback. The trick, according to Regus, is for organisations to utilise the strengths of this diverse workforce to their advantage, by creating opportunities to learn from each other.

The technological benefits of reversing

One way that some organisations have approached this is through reverse mentoring. Rather than following the usual path of older, more senior employees being assigned a newer colleague to mentor, reverse mentoring (unsurprisingly) sees younger or newer employees sharing their knowledge with company stalwarts. An early champion of this strategy in the 90s was Jack Welch, then CEO of General Electric. Welch recognised the importance of capitalising on the skills of the company’s younger employees, and instigated an initiative that saw older employees learn how to use Netscape. Today, reverse mentoring is commonplace in global companies including Microsoft and Cisco.

Technology is an area in which reverse mentoring is particularly valuable. Having grown up in an age of constant technological change and development, Gen Y are ideally placed to offer insight into how technological innovation can benefit an organisation and its processes. Crucially, technological innovation has also opened up the possibility of working more flexibly, something that is highly valued by Gen Y employees. Senior employees who have taken part in reverse mentoring programmes have also highlighted gaining an insight into the potential benefits of flexible working as one of the positive outcomes of developing a mentoring relationship with a younger employee. Thus, opening up this dialogue between generations can potentially diffuse conflict between the traditional 9-5 generations and the less hierarchical Gen Y.

A two-way street

In practice, reverse mentoring has been found to be less ‘teaching an old dog new tricks’ and more of an exchange of information and experience. At General Electric, one of the most basic benefits for the young mentors was simply the ability to gain contacts in the upper echelons of the company. The mutual benefits of the relationship can also be seen in terms of the insight it offers each party. The older participant gains in terms of gaining new perspectives on the company’s industry, and the thinking of its workforce, while the younger gains a better understanding of the company’s strategies and objectives, and becomes better placed to recommend actions or technologies that may support these.

Reverse mentoring – how to do it

In 2013, Boston College’s Sloan Center on Aging and Work published an evaluation of the implementation of a reverse mentoring initiative by The Hartford, a leading US insurance company. The company’s CEO had identified a need for the company to become more confident in its use of digital technologies, particularly social media, and recognised that its younger employees were best placed to drive this forward. Following a successful initial pilot that went onto become a national initiative within the company, The Hartford highlighted the following factors as crucial to the success of any reverse mentoring programme:

  • the creation of a project timeline;
  • identifying the business objectives – link the reverse mentoring programme to what the business is trying to achieve as far as possible;
  • ensure that mentors are fully informed of what mentees are expecting to get out of the exchange;
  • making sure the initiative has clear agendas and timelines;
  • using the mentor role as a way of keeping younger employees motivated; and
  • encourage both mentors and mentees to be open to the relationship and gaining new knowledge, and to respect that each other approaches learning differently.

The final point is echoed by the majority of companies who have used reverse mentoring within their organisation. Initially, Cisco had issues around more senior employees adapting to younger employees’ more informal way of working. As they, and other reverse mentoring adoptees have discovered, though, the key is commitment to the programme, in recognition of the value it can bring to the business.


Follow us on Twitter to see what developments in policy and practice are interesting our research team.

Further reading: if you liked this blog post, you might also want to read Heather Cameron’s post on how entrepreneurship drives growth in the UK.

Let’s get personal: top tips for using our information alerts

Personalised infoBy Heather Cameron

With the abundance of information available these days and with time becoming ever more valuable, finding the information you need when you need it can be an arduous task.

Whether you are interested in a particular topic in general or need to be kept abreast of latest developments, having tailored information sent to you at times that suit you is invaluable.

This is something the Idox Information Service can help with. As highlighted in our previous blog showcasing the service, current awareness services are a large part of our offering.

Current awareness tools

In addition to our weekly bulletin providing a selection of the latest policy, research and comment added to our database, and our more subject-specific fortnightly topic updates which include 29 topics to choose from, we also offer the option of information alerts that allow for an even greater degree of personalisation.

With the Idox Information Service’s email alert service, our users can create a customisable schedule of alerts on topics of specific interest. So if co-housing is an area of particular relevance but you don’t want to receive items covering the wider topic of housing (which you can find in our housing topic update) then an alert on co-housing will pinpoint only those items of interest.

Setting up an alert

If you are a member of the Information Service, all you have to do to set up an email alert is log in, run a search as normal and click on the ‘Save as alert’ link at the top of your results: alert1

This will then provide options for saving your search as an alert so that you receive an email update whenever any new content that matches your search criteria is added to the database. You can choose a name for your alert and decide how often you would like to receive it. Alerts can be sent to you either weekly, fortnightly or monthly. Alternatively you can run the saved search whenever you choose from the My alerts page:

Alert pageIf you have selected a daily, weekly, monthly or fortnightly alert, you will receive emails with a list of records that match your search as below:

email alert2If there are no items in the selected time period that match your results, you will not receive any emails. And if you are going on holiday, you can also suspend the alert while you are away and reactivate it on your return – no need to worry about your inbox being clogged up!

Editing your alerts

These tailored updates allow you to receive focused information on the specific subjects that you need for your work, and you can adapt your search criteria to reflect your changing information needs as often as you like.

For example, if you think you receive too many or too few records, changing your search terms in order to broaden or narrow your search may help.

To do this, click on the ‘Edit’ link on the relevant alert on the My Alerts page which will take you to the ‘Edit Alert Schedule’ page. Then simply click on ‘Edit Terms’ and you will be taken back to the main search box, where you can change your terms. To update your alert, run the search with the edited terms, click on ‘Save as Alert’, then use the ‘Update Existing Alert Schedule’ option:

updateMeeting changing information needs

We cover a variety of topics on our database ranging from economic development, regeneration and planning, to social care and health. Some recent alerts that have been set up include:

  • Local welfare
  • Disguised compliance
  • Children and families social work
  • Child poverty
  • Safeguarding adults
  • Youth crime
  • Scottish tourism
  • Poverty and social justice
  • Benefit cap
  • Business and development
  • Land value tax
  • Community engagement
  • Community safety/involvement
  • Sustainable development

Our information alerts provide an efficient way for users to receive the tailored information they need and expect. Hopefully this article has shown how easy they are to set up.


If you’d like to find out more about our email alerts, or any other aspect of the Idox Information Service, you can contact us.

Happiness and productivity, and how our Ask A Researcher enquiry service can help to increase at least one of these things…

Smiley face

Image created by Sergio Barros from the Noun Project

by Laura Dobie

It’s the International Day of Happiness today. To mark the occasion on the blog, we’re going to take a closer look at a recent literature search that we did on happiness and productivity, and how the service can help our members to be more productive in their work.

Ask A Researcher

The Idox Information Service offers an Ask A Researcher enquiry service, which is very popular with our members who need to source and synthesise evidence and policy documents to meet tight deadlines. We’re often told that our searches save our members a day’s work or more, were they to conduct the searches and synthesise the research themselves, and they free up our members to work on other areas and achieve more with their day.

They can ask us to search for information on their behalf, and our team of research officers will conduct complex searches of our in-house database (over 200,000 references across a broad range of subjects in relation to economic and social policy), and other sources, where appropriate, to compile lists of relevant references to send back to the enquirer.

We don’t just send on a list of references for you to sift through: our research officers will also produce a research summary to accompany the results, which provides an analysis of the references that we have retrieved. This highlights:

  • Trends;
  • Key findings;
  • Implications for policy and practice; and
  • Significant research reports and articles, which are particularly relevant to the enquirer’s needs.

If the enquirer has asked a specific question, we will do our best to find an answer in the documents that we have sourced and present this in the summary.

Literature search on happiness and productivity

We recently carried out a search on our database for research which examined the link between levels of happiness in organisations and productivity and organisational performance. You can view this sample search here.

This search provides an ideal example of what we’re trying to do with the Ask A Researcher service: rather than simply compiling references, we have specifically highlighted resources in the results (and key words in the abstracts) which help to answer the research question.

The results describe the search terms and date limits which were used, and provide an overview of the content of the resources which were retrieved.

The summary highlights key documents within the results which are particularly pertinent to the research question, including:

  • MacLeod and Clarke’s concept of employee engagement: an analysis based on the Workplace Employment Relations Study, which explores employee engagement and organisational performance. It found that high levels of employee engagement were strongly associated with both financial performance and labour productivity.
  • Healthy staff equal healthy profits, IN Management Today, Jul/Aug 2013, pp56-57, which observes that organisations which look after the wellbeing of their employees see a return in greater commitment and higher productivity. It stressed the importance of effective communication of employee benefits, which can have a significant impact on productivity.
  • A government literature review, which has investigated the business benefits of adopting work-life balance practices, highlighting the positive association between flexible working and productivity and reduced absences, and between family friendly policies and retention and reduced absences. It observes that “A large body of evidence demonstrates that effective outcomes at the level of the individual, including job commitment, ‘happiness’, satisfaction, engagement and, in turn, discretionary effort, are all associated with business benefits such as reduced leaving intentions, fewer absences, less tardiness and improvements to performance and productivity.” (p.viii)

In addition to the results sourced from our own database, we also highlighted research from the University of Warwick, retrieved online, which also demonstrates the link between happiness and productivity.

Hopefully this article has provided some useful insights into the links between happiness and productivity, and demonstrated how our Ask A Researcher service can help our members to source and synthesise research in a short space of time and be more productive at work.

If you’d like to find out more about our Ask A Researcher service, or any other aspect of the Idox Information Service, you can contact us.

Knowledge insider…. a Q & A with Jonathan Breckon

jonathan_breckon_150x150In the latest of our series of Q&As with leading advocates of the use of evidence in policymaking and practice, we talk to Jonathan Breckon. Jonathan is Head of the Alliance for Useful Evidence – a partnership which champions the need for useful evidence, providing a focal point for improving and extending the use of social research and evidence in the UK.

Jonathan, what led you to a role about promoting and improving knowledge development? 

There are two ways in which I am interested in knowledge development; professionally I have always worked around universities, loved doing and finding out new research and working within research in UK. I have always been conscious however, of the gap between research and front line services, even when research is relevant to the service, and felt this was a great loss and disadvantage to public services.

My personal interest, as a user of public services, with my kids going through services such as schools, health and sports, I have been desperately aware that things are business as usual rather than continuously striving for innovation and change. The debate is now all about money and reductions when it should be about improvement and future proofing.

We don’t always know what it takes to bridge the gap between what we need, and what services can provide; research can actually help that. The What Works approach is really important but very hard, as it’s difficult to stop doing things we have already invested in. An evidence-based research approach can challenge and support this evaluation and we have a moral duty to do it and not continue to invest in services which don’t work.

What do you think the main benefits of developing your knowledge are?

The challenge of seeing if things work or not, why they work, where they work and who they work for – developing your knowledge is the critical aspect of improving how you do things.

It’s also important for a whole host of other benefits. I particularly like Carol Weiss’ work, which is instrumental – this ‘enlightment’ operational research should not be dismissed. This approach can support the ideas of learning continuously through research; it implies a continuous review of theory, methods, practice and we should always be striving to improve our methods and outcomes.

When people are talking to you about evidence, research or knowledge, what do they most frequently raise as issues?

The most common one is that investing in evidence is just rhetoric; politicians, charities, parties etc will never really be informed by the research agenda, and I agree. We aren’t in a super-rational culture, it’s about our wider culture, values and beliefs as well. But it’s a fundamental misunderstanding that research trumps anything. It is part of the mix, part of the overall democratic and rational approach to doing anything.

The Behavioural Insights Team has a massive role to play in understanding the biases in how we make decisions, whether in prisons, police, policy etc. We don’t work rationally all the time and evidence can help us understand the messiness of policy making. We just need it to be a bigger part of the mix.

Everybody has this view that they use evidence but we don’t really understand how effectively they use it.

What are the hard to spot mistakes when it comes to developing your knowledge, which you really need to avoid?

The main one is that not all evidence is equal; that you have to make judgements about it. This is hard for those writing the research – it’s not about the quality of the research, and it’s about the point of view from demand. They need some things and not others.

The big challenge, if you are looking at impact, is you need different approaches, experiments, systematic reviews – one study is not enough. Such as when you see studies reported in newspapers – until replicated we don’t really know if it is robust. You need to avoid literature reviews where you cherry pick, go to something which is transparent and is systematic. This is true of both policy makers and researchers’ point of view; we underestimate the challenges facing both sides.

Need more about impact. We are very good at qualitative – world class – but we are behind in quantitative methods. It is being addressed but it will need to filter through.

How do you think people will be doing evidence, research and knowledge development in 5 years’ time?

What Works Centres will, I hope, be a key part of the evidence ecosystem, in the way that NICE have done, helping providers and policy makers make decisions. Although it doesn’t do research itself it sucks in research and uses it well.

There will always be critics of them, even of just the name, but they will change the system. Some have been around for a while and are well established, but others are new and are just about to be. As well as synthesising research they will commission new work. For instance in wellbeing, we know a lot about the correlation with health and wellbeing but don’t know a lot about what will work in improving it.

Technology makes it very difficult to guess about the future, who would have predicted the work in social media research? Big data is emerging now and in 5 years’ time might be a standard tool. The fundamental principles like statistics will be there but we will have to adapt to the possibilities offered by technology

If you had a list of ‘best-kept secrets’ about research, evidence and knowledge you would recommend, what would you include and why?

Just because you have done a social science masters and PhD, does not make you an expert in evidence, partly because people over-specialise. People need open their minds to different methods and how people do it in other places. The Department for International Development have an amazing range of techniques, nothing like anything you have seen, with a database of all the research they have funded or delivered.

Emerging opportunities such as social media research – still early days and fundamentally new, and could have a huge impact. Most people’s default though is to go to an expert and be frightened off journals and academics; I don’t think you always have to commission something new, it’s about variety, breadth and developing your understanding in as many ways as possible.


 

You can also read Q&As with Tim Allen, Local Government Knowledge Navigator; Clive Grace, Local Government Knowledge Navigator; Sarah Jennings of the Knowledge Hub; and Kim Ryley, recent Past President of the Society of Local Authority Chief Executives.

 

Knowledge insider… a Q & A with Tim Allen

tim allenIn the latest in our series looking at evidence based practice, I spoke to Tim Allen, co-owner of two research consultancies, previously research director for the Local Government Association and a senior civil servant for the Department for Environment, Food and Rural Affairs and the Treasury. Tim is also one of the Local Government Knowledge Navigators.

Tim, what led you to a role promoting and improving knowledge development?

My interest in the practical use of knowledge and evidence is long standing.

I started my career as a property professional and very quickly moved to the former Agriculture Development and Advisory Service, where my role was to encourage knowledge exchange and technology transfer in improving agricultural production and fostering environmentally sensitive farming.

As I moved on, I became a member of the executive board for the then Countryside Agency where I was responsible for a wide range of activities, including establishing a corporate research function to inform rural policy at a time when this was high profile, something that I subsequently followed through in roles in the Treasury and DEFRA by setting up a rural policy function in the early 2000s.

More recently, I moved to the Local Government Association as research director with a key role to support the case for local government by providing or sourcing evidence to inform policy development, on a wide range of topics from the impact on public services of migration through to climate change or encouraging supermarkets to reduce waste by reducing product packaging.

During this time, I also sponsored the LARCI Initiative, which brought together UK Research Councils, academics and local government on the basis that local government wasn’t benefitting from very substantial national investment in research and development.

A key lesson for me from that experience (which had it’s successes, but ultimately didn’t quite reach out to local government in the way that we hoped) was the importance not of the research report that may or may not sit on shelves, but of the knowledge and experience lodged in the heads of the researchers concerned – which can be a knowledge base built on years, if not decades, of working in that field.

To me, applying research to the practical problems of local public service policy and practice is about actively – and intelligently – bringing people together to collaborate, learn and exchange ideas and knowledge. Local Government has to tackle often complex and interconnected issues, such as public health and social care, supporting the most vulnerable in society, tackling waste, and planning for transport.

In our role as Local Government Knowledge Navigators, we passionately believe that as local government and local public services face eye watering cuts to funding and increased demand, we need to look for new sources of knowledge and innovation. Whilst some of this can and will come from within, when you are under pressure, you need to look more widely for wise and long term solutions, or the clues that can help you reach these such solutions.

What do you think the main benefits of developing your knowledge are?

To use an old term, Continuing Professional Development – the world does not stand still, people in policy need to keep up, to make sure we are on top of where the best knowledge is and to apply it. As resources are constrained at the moment, you really do need that knowledge even more.

What you do, why you do it and how, needs to be demonstrated and organisations need to be sure they are making good, sound decisions. If short term decisions made necessarily in haste are to be well informed and robust – and not cause regret later because they were sub-optimal – they require good quality evidence and knowledge. Otherwise they risk creating the catastrophes of tomorrow.

The benefits of knowledge based decisions include confidence in decisions – and crucially, political assurance that they are soundly based – even if this means exposing yourself to new, and potentially disruptive influences. For example, who would think that multivariate modelling that draws from the world of engineering has application in social care: well the answer is that it does, and colleagues in Southampton University are showing how this approach can inform demand management and care planning for the elderly.

However, we start from a very low base, with little systematic research and development serving local government despite the fact that the sector still spends over 20% of public expenditure. We are on a journey: Local Government doesn’t have the resources for this, yet there are beacons of exemplary good practice in applying research and research knowledge to impressive effect, but these examples are just that, episodic and without systemic adoption.

We need to connect local government to new, relevant knowledge, but not long academic tomes of research, it’s the connection between the researcher and the practitioner that is of value. One driver ought to be CPD and the need to connect with practitioners. This should be about knowledge exchange, based around people as well as the research, with approaches that foster ‘co-production’ of knowledge and the ‘co-definition’ of problems. Integrating the researcher into the system and creating a research loop, with dialogue during research.

Developments in policy around research funding are helpful in placing ever stronger emphasis on real world impact, but we need a shift in Local Government also to embrace new sources of knowledge. And this includes our local politicians who should see this as a means to have a more informed dialogue with citizens, not a threat because new knowledge may challenge pre-conceptions: austerity should leave no place for ill informed policies.

When people are talking to you about evidence, research or knowledge, what do they most frequently raise as issues?

How you respond to substantial reductions in public funding? How to know whether you are making the right decisions? Am I commissioning the right services, will it work and work for a period of years? What will local government look like in five years time? In a world where local government has shifted from an industrial scale service provider through to being a commissioner and, increasingly, a minor funder yet still accountable when things go wrong (e.g. care support for the elderly which is substantively in the private sector) – how do you deal with that? How do you deal with complexity? How do you manage within the commercial environment or commissioning framework? How do you avoid the major failures?

What are the hard to spot mistakes when it comes to developing your knowledge, which you really need to avoid?

Know where the expertise lies – to what degree can we rely on the resources we are being pointed to, and are those resources transferable? Are they robust enough to be reliable? Are we relying on unsubstantiated stories about good practice?

People often don’t know that useful evidence exists, and there is a huge disconnect between publicly funded research, which should be informing practice and local government. The drivers of success in academia are all too often publishing in academic journals that are peer reviewed and rigorous, yet ultimately often never reach practitioners.

There is also a disconnect between where research is published and where practitioners go for their information. As a result, we need structured facilitation to bring the two together (using the thrust for academic research to demonstrate real world impact) around the issues that practitioners face and help both sides come together. And before we become dispirited, there are many in the research world who want to get involved in public policy and practice, and see their research have real impact.

People often think of knowledge and research as an overhead. This is a false view and we can draw on lessons such as a recent piece by the former Swedish Prime Minister Goren Persson on the things you need to do to get yourself through severe cuts and change. He highlights how important it is to have clarity, vision and evidence, with a clear view that something at the end of the change will be better.

How do you think people will be doing evidence, research and knowledge development in five years’ time?

I would hope that we are beginning a culture shift, that will become more embedded and systemic across local government and local public services with Local Government people and researchers better connected, and with many more accessible routes and pathways to link the two.

We may not have nirvana, but we would have local dialogue with active research agendas under way which are delivering results and an equitable portion of national investment in research working for local government being steered or influenced by the sector.

The seeds are there, it just needs leadership, from those who fund research, and from local government and public services to push for this access to – and collaboration with – the research and development base, to turn what has been ad hoc and happenstance to something more systemic.

For example, in practical terms, this could include effective use of knowledge and evidence as part of peer review in local government, and professional societies and groups engaging in the agenda through CPD, but more fundamentally, fostering a thirst for knowledge rather than creating a compliance culture.

If you had a list of ‘best-kept secrets’ about research, evidence and knowledge you would recommend, what would you include and why?

  • Dialogue with your local university, find out how you can work together: there are a whole range of potential opportunities that might be a Vice Chancellor getting involved in strategy, or researchers working in partnership on a particular policy or project such as smart city development or demography.
  • Come to us as knowledge navigators to get help, make connections (which might be a about accessing national expertise.
  • Let’s exploit the recently announced ESRC ‘impact accelerator accounts’; funding is available to encourage 24 universities and research institutions to engage with external stakeholders, including local public services, to explore opportunies.

You can also read a Q&A with Clive Grace, Local Government Knowledge Navigator; a Q&A with Sarah Jennings of the Knowledge Hub; and a Q&A with Kim Ryley, recent Past President of the Society of Local Authority Chief Executives.