Category Archives: Data

Finding data in Scotland

I’ve delivered data journalism training in Scotland twice in the past few months, and thought I’d share some tips on what data is available there, given that most guidance on data journalism focuses on data in England or the US.

General statistics and data

Scottish Neighbourhood Statistics has a ‘data download’ page which allows you to download data on a range of topics, each of which has dozens or hundreds of indicators, from ‘Physical environment’ (24 indicators) to ‘Economic activity, benefits and tax credits’ (377 indicators).

Subjects covered include health, housing, access to services, community care, crime, education and ‘special interest’ reports (these include sea fisheries, poverty and cultural event attendance).

Once you’ve downloaded the data you may find that it doesn’t name each local authority but instead uses a series of codes. These are ONS (Office for National Statistics) codes – you can find a master list on Wikipedia.

The Scottish Government Statistics page also provides access to a range of topics, including agriculture, business, tourism, transport and travel.

The Equality Evidence Finder is particularly interesting: this is an attempt to show the equality of different industries across measures such as gender, disability, income and so on. This, for example, is the page summarising the picture for ethnicity in Scottish business, enterprise and tourism.

The General Register Office for Scotland has broad statistics about births, deaths and marriages (known as ‘vital events‘), life expectancy, migration and elections.

And of course don’t overlook Data.gov.uk, where you can subscribe to an RSS feed for new datasets mentioning ‘Scotland’.

Health data in Scotland

ISD Scotland – the Information Services Division of NHS National Services Scotland – provides a range of health data from the right hand column of its home page, from specific conditions including cancer and mental health, through to the NHS workforce, eye care and dentistry, maternity, and prescriptions.

Rather helpfully, NHS Scotland has its own domain at scot.nhs.uk, which means you can a search for Scottish health data on Google by adding site:scot.nhs.uk to any search terms. Using your particular health board (e.g. ‘Grampian’) will also help.

NHS National Services Scotland is another useful domain at nhsnss.org – adding site:nhsnss.org to your search (and, for example filetype:xls) might also bring up some useful data – I found data on gifts and hospitality, payments to GP practices, and cancer waiting times using those.

Health Protection Scotland has regular data reports and publications. And there’s the Scottish Health Survey and Inpatient Patient Experience Survey on the Scottish Government statistics pages.

Education data in Scotland

Education Scotland publishes school reports, publications, and provides school-level information through the Scottish Schools Online portal. Some of this data is available as datasets through the Scottish Government statistics page too, such as this page on school meals.

Environment data in Scotland

Scottish Environment Statistics Online (SESO) publishes datasets on a wide range of indicators from recycling and conservation to air quality and noise pollution.

The Scottish Environment Protection Agency monitors environmental conditions and waste, most of which is linked from here.

And Scottish Natural Heritage has a page linking to statistics and reports.

If you are interested in data which has any sort of environmental impact (this can include things like housing and construction) then it may be worth looking into the Environmental Information (Scotland) Regulations (EIR).

This is similar to Freedom of Information laws, but have fewer exceptions and apply to a wider range of public bodies. The book EIR without the Lawyer is a useful guide for journalists.

Data on crime, justice and fire

The Scottish Government statistics pages on crime and justice, covering everything from perceptions of crime to numbers of police, racist incidents, the prison population and re-offending.

They also have data on fires and fire and rescue servicesAudit Scotland also has performance data on the fire and rescue services.

The Judiciary of Scotland has been more open than its English counterpart, with data including judicial expenses, and judgments and sentences – although these are provided as PDFs and feeds rather than more easily downloadable data.

Local government data in Scotland

Audit Scotland’s Council performance pages are a fantastic resource if you’re interested in the performance of services under local authority control. This includes schools, housing, social work and hundreds of ‘performance indicators’, with a spreadsheet for every council.

You can also look at the same data for all councils by service on the services page.

Local government finance data can be found on the Scottish Government statistics site – including council tax collection and budgets.

On housing some local authorities have passed over some or all of their housing responsibilities to housing associations. You can find data on these on the Scottish Housing Regulator site including inspection reports, accounts, performance returns and performance profile, regulation and rules.

The Scottish Parliament

MPs’ allowances are published on the Scottish Parliament website. It’s not particularly easy to download and compare. In these situations it’s often worth searching to see if you can find someone who’s tackled the problem – and Owen Boswarva did (at least, he did in 2011/12), providing a link to the bulk data. If he did it once, he may do it again, if you ask nicely…

Freedom of Information in Scotland

Scotland has its own Freedom of Information Act and its own Information Commissioner, who has a reputation for enforcing the Act more strongly than his equivalent south of the border.

To complicate things, many bodies operating in Scotland will also be subject to the UK FOI Act and the UK Information Commissioner.

You can follow public FOI requests mentioning ‘Scotland’ made using WhatDoTheyKnow, or of course specify a different region like ‘Dundee’ or a related term like ‘Scottish’. And you can look for Scottish public bodies’ disclosure logs, where they may be publishing the latest FOI requests answered.

Scotland also has the INSPIRE (Scotland) Regulations, which provide access to mapping – “spatial” – data.

And more broadly you may be interested in Public Contracts Scotland, which provides access to data on contracts awarded by local authorities, health bodies, education agencies, emergency services and national government.

I’ll be adding to and updating this post as I think of other sources. If you know of any please let me know.

Reporting the water industry – key performance indicators

Severn Trent Water Key Performance Indicators 2012

Severn Trent Water Key Performance Indicators 2012

If you’re interested in learning more about how the water industry works, Ofwat’s pages on regulating the industry provide information on the industry itself, what water and sewerage companies have to do to keep their licences, and how they are regulated. You can also find links to the water companies’ annual reports, including their ‘key performance indicatorsThe section of Key Indicators explains these (also called KPIs): it includes a link to a spreadsheet (XLSX) which does the following: Continue reading

Mapping bus stops in your local area

Recently I helped Pupul Chatterjee map bus stops in Birmingham for the BrumTransport. I thought I’d share the process here as it demonstrates a number of techniques in filtering data that isn’t helpfully categorised.

Does the data exist? Searching in the right place

I started my search for bus stop data at Google’s Tables search engine – this allows you to search public Google Fusion Tables as well as ‘web tables’. I started with a broad search for “bus stops” – the top result looked promising: a Guardian datablog post on “Every bus stop, train station, ferry port and taxi rank in Britain“.

That post included a link to a fusion table that mapped those stops, as well as the original data on Data.gov.uk.

The question was: how easily could we extract Birmingham bus stops from that? Continue reading

Understanding company accounts: How to get the most of Companies House

companies house logo

At the CIJ Summer School this year Robert Miller, Martin Tomkinson and Raj Bairoliya explained how to access company accounts and how to get the most of Companies House.

In the UK, all limited companies and limited liability partnerships must file their accounts at Companies House, which has offices in London, Cardiff and Edinburgh. There are more than two million registered firms and over 300,000 new companies incorporated each year, and Miller feels that Companies House is one of the main skills needed by journalists. Continue reading

5 tips on mining and using big data for journalists

cijlogo

Matt Fowler is a freelance application developer and programmer who helps journalists understand and use big data. At the CIJ Summer School this year he gave some top tips in the field, which we have summarised below…

1. Double check privacy settings of your data

You don’t want private work being published on show for all to see.

2. Tidy up the data and make the structure simpler

This re-engineering effort can get details out and help you to discover information to turn into stories. Continue reading

Get the data: all MPs in the UK

Help Me Investigate has compiled a list of MPs who operate throughout the UK.

The list has been scraped from the Parliament website.

The dataset shows the name, political party and constituency of all 674 MPs in the UK. In addition to this, a URL link to each MP’s biography has been displayed.

This information is also available on a Google spreadsheet.

If you use this information, please let us know – especially if any have been changed since publication. Continue reading

HMI Health compiles clinical commissioning group FOIs for health sector magazine

Over on Help Me Investigate Health, we’ve published a list of Freedom of Information emails for clinical commissioning groups (CCGs) – the new bodies controlling local health spending in England.

The list was compiled two months ago for Health Service Journal by Tom Warren and Matt Burgess, shortly after the new bodies took control, but is only now being published by HMI.

As part of the process, Matt and Tom also compiled a spreadsheet of CCG websites and FOI webpages.

What data does the Fire Service collect? Notes from a datablog meetup

Cat stuck up a tree

Image by Jonas Lowgren

On February 28 the Birmingham Datablog Meetup hosted a visit from Vikki Holland and Debbie Whittingham from the West Midlands Fire Service. They answered questions on how the Service uses data in both fire prevention and monitoring its activities. Here is a summary of the meetup.

The Fire Service are one of the most advanced users of data in the public sector. With much of their work centred on fire prevention, they draw on a range of data sources to identify and prioritise different risks in different communities. Continue reading

Understanding probation and justice data – notes from a @bhamdatablog meetup

At a recent Birmingham Datablog meetup we hosted probation data analyst Jason Davies, who very generously spent time highlighting the mistakes that journalists should avoid in reporting the justice system, and useful resources for finding both data and context on crime and justice.

Jason began by making an important distinction between crime data and justice data. Crime data is typically handled by individual police forces and the Home Office.

Once someone is charged, however, they enter the justice system. Information about what happens next – in the courts, the prisons, community service and on probation – is handled by the Ministry of Justice and other related bodies. Continue reading