Tag Archives: EIR

Finding data in Scotland

I’ve delivered data journalism training in Scotland twice in the past few months, and thought I’d share some tips on what data is available there, given that most guidance on data journalism focuses on data in England or the US.

General statistics and data

Scottish Neighbourhood Statistics has a ‘data download’ page which allows you to download data on a range of topics, each of which has dozens or hundreds of indicators, from ‘Physical environment’ (24 indicators) to ‘Economic activity, benefits and tax credits’ (377 indicators).

Subjects covered include health, housing, access to services, community care, crime, education and ‘special interest’ reports (these include sea fisheries, poverty and cultural event attendance).

Once you’ve downloaded the data you may find that it doesn’t name each local authority but instead uses a series of codes. These are ONS (Office for National Statistics) codes – you can find a master list on Wikipedia.

The Scottish Government Statistics page also provides access to a range of topics, including agriculture, business, tourism, transport and travel.

The Equality Evidence Finder is particularly interesting: this is an attempt to show the equality of different industries across measures such as gender, disability, income and so on. This, for example, is the page summarising the picture for ethnicity in Scottish business, enterprise and tourism.

The General Register Office for Scotland has broad statistics about births, deaths and marriages (known as ‘vital events‘), life expectancy, migration and elections.

And of course don’t overlook Data.gov.uk, where you can subscribe to an RSS feed for new datasets mentioning ‘Scotland’.

Health data in Scotland

ISD Scotland – the Information Services Division of NHS National Services Scotland – provides a range of health data from the right hand column of its home page, from specific conditions including cancer and mental health, through to the NHS workforce, eye care and dentistry, maternity, and prescriptions.

Rather helpfully, NHS Scotland has its own domain at scot.nhs.uk, which means you can a search for Scottish health data on Google by adding site:scot.nhs.uk to any search terms. Using your particular health board (e.g. ‘Grampian’) will also help.

NHS National Services Scotland is another useful domain at nhsnss.org – adding site:nhsnss.org to your search (and, for example filetype:xls) might also bring up some useful data – I found data on gifts and hospitality, payments to GP practices, and cancer waiting times using those.

Health Protection Scotland has regular data reports and publications. And there’s the Scottish Health Survey and Inpatient Patient Experience Survey on the Scottish Government statistics pages.

Education data in Scotland

Education Scotland publishes school reports, publications, and provides school-level information through the Scottish Schools Online portal. Some of this data is available as datasets through the Scottish Government statistics page too, such as this page on school meals.

Environment data in Scotland

Scottish Environment Statistics Online (SESO) publishes datasets on a wide range of indicators from recycling and conservation to air quality and noise pollution.

The Scottish Environment Protection Agency monitors environmental conditions and waste, most of which is linked from here.

And Scottish Natural Heritage has a page linking to statistics and reports.

If you are interested in data which has any sort of environmental impact (this can include things like housing and construction) then it may be worth looking into the Environmental Information (Scotland) Regulations (EIR).

This is similar to Freedom of Information laws, but have fewer exceptions and apply to a wider range of public bodies. The book EIR without the Lawyer is a useful guide for journalists.

Data on crime, justice and fire

The Scottish Government statistics pages on crime and justice, covering everything from perceptions of crime to numbers of police, racist incidents, the prison population and re-offending.

They also have data on fires and fire and rescue servicesAudit Scotland also has performance data on the fire and rescue services.

The Judiciary of Scotland has been more open than its English counterpart, with data including judicial expenses, and judgments and sentences – although these are provided as PDFs and feeds rather than more easily downloadable data.

Local government data in Scotland

Audit Scotland’s Council performance pages are a fantastic resource if you’re interested in the performance of services under local authority control. This includes schools, housing, social work and hundreds of ‘performance indicators’, with a spreadsheet for every council.

You can also look at the same data for all councils by service on the services page.

Local government finance data can be found on the Scottish Government statistics site – including council tax collection and budgets.

On housing some local authorities have passed over some or all of their housing responsibilities to housing associations. You can find data on these on the Scottish Housing Regulator site including inspection reports, accounts, performance returns and performance profile, regulation and rules.

The Scottish Parliament

MPs’ allowances are published on the Scottish Parliament website. It’s not particularly easy to download and compare. In these situations it’s often worth searching to see if you can find someone who’s tackled the problem – and Owen Boswarva did (at least, he did in 2011/12), providing a link to the bulk data. If he did it once, he may do it again, if you ask nicely…

Freedom of Information in Scotland

Scotland has its own Freedom of Information Act and its own Information Commissioner, who has a reputation for enforcing the Act more strongly than his equivalent south of the border.

To complicate things, many bodies operating in Scotland will also be subject to the UK FOI Act and the UK Information Commissioner.

You can follow public FOI requests mentioning ‘Scotland’ made using WhatDoTheyKnow, or of course specify a different region like ‘Dundee’ or a related term like ‘Scottish’. And you can look for Scottish public bodies’ disclosure logs, where they may be publishing the latest FOI requests answered.

Scotland also has the INSPIRE (Scotland) Regulations, which provide access to mapping – “spatial” – data.

And more broadly you may be interested in Public Contracts Scotland, which provides access to data on contracts awarded by local authorities, health bodies, education agencies, emergency services and national government.

I’ll be adding to and updating this post as I think of other sources. If you know of any please let me know.

AUDIO: Investigating a bank using environmental information laws: #Dataharvest13

One of the most interesting talks at the data and investigative journalism conference #Dataharvest13 was Gavin Sheridan‘s talk about how they tackled an Irish bank using laws on environmental information.

The bank in question was not subject to the Freedom of Information Act, despite being Government-owned. But the Environmental Information Regulations (EIR) applies differently, and Sheridan, along with Fred Logue, used those to request – then fight for – information on their involvement in property development.

Sheridan and Logue’s talk is split over three audio clips – embedded below:

New guidelines on the ‘FOI for environmental reporting’ – EIR

Cleland Thom provides another of his useful roundups on ‘right to know’ laws in the UK. This time it’s the Environmental Information Regulations (EIR), a set of regulations which work in a similar way to the Freedom of Information Act, but specifically cover information about the environment.

The regulations aren’t just useful if you’re reporting on the environment – transport, energy and other issues all touch on the areas the EIR covers.

Now the Information Commissioner has issued guidelines on the regulations (PDF) for the first time. Cleland’s rundown of the basics goes as follows:

  1. The ‘default position’ is that authorities must disclose information if journalists ask for it.
  2. The regulations make it hard for an authority refuse a information request.
  3. Public authorities can release business information they hold – without the business’ knowledge or consent.
  4. Authorities cannot ask why a journalist wants information, or be obstructive or secretive because they know what they might do with it.
  5. The information must be accurate and up to date.
  6. They must respond in 20 working days.
  7. The media does not have automatic right to reproduce information they’re given. Check copyright first.
  8. Journalists can make requests by phone. But it’s best to provide written confirmation  to ensure accuracy.
  9. Authorities cannot ‘neither confirm nor deny’ they have information, unless it’s to do with international relations, defence, national security or public safety
  10. Authorities can refuse a request if it is too general – but they should then ask for a more precise request.
  11. Authorities can refuse to hand over information when It involves personal data; is against the public interest; is unreasonable; release WILL (not might) have an adverse effect on the course of justice, or someone’s intellectual property rights; the information is incomplete, or not available at all.

To give Cleland Thom a deserved plug, he is a consultant and trainer in media law, and is good.

More guidelines on EIR in this article in Local Government Lawyer from September 2011.

5 ways to simplify an investigation

If you are trying to investigate something – to get answers to a question – how do you make sure that you use your time most effectively?


Here are 5 ways to do just that:


1. Write a hypothesis


This is the advice of Mark Lee Hunter, explained in a free ebook called ‘Story-Based Inquiry’, and is probably the most important action in keeping you on track.



A hypothesis helps you clarify exactly what it is that you are gathering evidence for – and it helps you see when your hypothesis needs to change.


A good hypothesis should be specific – numbers are good, even if they are plucked out of the air as something to begin with (those investigations linked above may have begun with different hypothetical figures – the important thing is that you start with something you can test). Terminology is important, too – avoid generic terms, and know the jargon of the field you’re looking at.


2. Break the investigation down into discrete tasks


An investigation is much more manageable – and easier for others to collaborate on – if you have broken it down.


Typical tasks might include the following:
  • Find background information – e.g. news coverage, official reports, etc.
  • Find experts
  • Find witnesses
  • Find people who are affected by it (they may gather in online communities such as Facebook groups, mailing lists or forums)
  • Find laws and regulations relating to the issue
  • Find documents – e.g. internal reports, meeting minutes, declarations of interest, etc.
  • Find facts and data – these are often compiled in internal or external databases, research, etc.
  • Write up the story so far – this is particularly useful for providing context for those who come to the investigation later.


3. Keep a record of what you’ve done and need to do


The potential for distraction is only partly addressed by a good hypothesis. If you have numerous parts to the investigation then you need to keep track of those – but also avoid spending so much time on one avenue that you overlook others.


Blogging the results as you go – and including what needs to be done next – can help you keep track of your progress.


Using categories (for questions or types of query) and tags (for people, places and organisations) effectively will allow you to easily find that information by just looking within that category or tag. You can also use a bookmarking tool like Delicious to keep track of online material, using and combining tags when you need to find them again quickly.


Blogging also makes it easier for others to find you – if they are interested in the same area. If you don’t want others to see what you’re doing, however, you can make posts or entire blogs private or password-protected.


In addition to blogging, there are a range of free online project management tools that can help keep track of the tasks ahead of you (for individuals, Springpad is quite useful in being on hand when something occurs to you).


And the Story Based Inquiry website provides a range of templates for keeping track of your investigation too: http://www.storybasedinquiry.com/masterfile/


All of the above allows you to get things out of your head and onto paper, clearing your mind to take a step back and re-assess what should be the priority next.


4. Exercise your right to information – but use the phone first


The Freedom of Information Act, Data Protection Act, Audit Commission Act and Environmental Information Regulations require public bodies to supply information when requested, as long as they hold the information and no exemptions apply. It is very useful for getting hold of information – but too often it is used with no clear idea of what you are actually looking for.


Speaking to someone who deals with that information can help you clarify what you ask for. Knowing what information is held, what the jargon is surrounding it, and what policies and reports relate to it, can all influence what you eventually ask for.


It also helps if you pre-empt any excuses that may be used to avoid providing you with that information.


5. Use computers to drill into large amounts of data


If your investigation involves going through lots of tables, it may be worth investing some time in learning basic computer assisted reporting techniques.


This will save more time further down the line, as well as potential errors which can creep in when you’re doing things manually (although you should also check initial results manually too).


Do you have any other tips for using time effectively in an investigation?

Making FOI requests: pre-empting the ‘commercial interest’ excuse

Quite often Freedom of Information requests are refused on the grounds of ‘commercial interest’ – or specifically that “the commercial interests of a third party will, or are likely to, be prejudiced”. This might also be referred to as being refused “under Section 43 of the Freedom of Information Act”.

Note that this is separate – although similar to – “commercial confidentiality” which applies to information that has been provided to the public body by another company.

Sometimes the use of Section 43 will be justified. Where it is not, however, is when the public interest in disclosure overrides those commercial interests. And this is where it is useful to be proactive in your FOI request.

The Information Commissioner’s Office is very useful in explaining how public bodies must deal with your FOI request, and provides guidance on each of the exemptions.

If you expect that your request may be refused on the grounds of commercial interest, it’s worth addressing these up front as part of your request, after you have outlined the information that you require.

How to phrase your FOI request to address possible commercial interest objections

Typical phrasing might go as follows:

“Please note that Section 43 of the Freedom of Information Act states that any attempt to withhold information on the basis that it would prejudice commercial interests can be overruled if there is a public interest in releasing that information”

You can then further strengthen your request by specifying how the release of this information does indeed overrule commercial interests on the basis that it will do one or more of the following, which are specifically outlined in guidance by the Information Commissioner’s Office (PDF):
  • further the understanding of, and participation in the debate of issues of the day; 
  • facilitate the accountability and transparency of public authorities for decisions taken by them; 
  • facilitate accountability and transparency in the spending of public money; 
  • allow individuals to understand decisions made by public authorities affecting their lives and, in some cases, assist individuals in challenging those decisions; 
  • bring to light information affecting public safety

Typical phrasing, then, might go something like this:

“This information clearly qualifies as being within the public interest as defined by the Information Commissioner’s Office (Awareness Guidance No.5) on the basis that it “furthers the understanding of, and participation in the debate of issues of the day”; “facilitates the accountability and transparency of public authorities for decisions taken by them”; and “facilitates accountability and transparency in the spending of public money.”

This public interest test will probably also help you focus on what information you should ask for.

Some other notes:

“The price submitted by a contractor is likely to be commercially sensitive during the tendering process, but less likely to be so once the contract has been awarded.” 

“Where a company enjoys a monopoly over the provision of the goods or services in question it is less likely that releasing the information will have a prejudicial impact on that company.” (guidance by the Information Commissioner’s Office (PDF))

The public body cannot make any presumptions on the commercial interests of the company, but should ask the company to make its own case if it wants information to be withheld. This cannot be used as an excuse for delaying a response: “Failure to respond by those being consulted does not remove the obligation [for the public body] to respond within [the 20 day] time limit.” 

It should also be noted that the public body also does not have to adhere to the wishes of the company. Any case put forward by the company for withholding information is only that: a case. If that case is overruled by the public interest in disclosure, then the public body can still – and should – release the information.