Category Archives: Tips and tricks

Please stop wasting my time and money and just send me the data

Leave a reply
Web developer Adrian Short made a Freedom of Information request to Transport for London for journey data from the London Cycle Hire scheme on 8th October. In January TFL released data to some developers. Yet to date TFL have not complied with the FOI request and Adrian Short is one developer who is not able to make use of the data.
Adrian Short is a supporter of the Barclays Cycle Hire scheme. He created a free API service for live data, which helped developers produce the rich choice of cycle hire apps Londoners enjoy. His motive? “I’m keen to do what I can to help people use it and to make it work better.” 
What happened
08.10.2010 Adrian made an FOI request for records of the first million Barclays Cycle Hire journeys as a CSV file.
05.11.2010 TFL send the first 100 journeys as  an email attachment and say the full million will be available from their website
04.12.2010 London Open Data Hack day passes without the TFL data
27.12.2010 Adrian requests internal review by TFL of their handing of his FOI request
05.01.2011 TFL make the data available from their developer area, which requires registration and agreeing to TFL terms and conditions
TFL: “I note your preference to be sent this data directly without having to register. This will be considered as part of the internal review, which should take place next week”
Adrian: “ Please stop wasting my time and money and just send me the data.”
What Do They Know FOI documents
Million journey zip file
The data is now available to developers who accept TFL’s terms. Developers have been able to produce visualisations, for example these by Suprageography
and Steerdavies 

Media_httpwwwsteerdav_eisfe
You could also download the million  journey zip file from a link in this Guardian article Guardian Data Blog, but you would have no legal basis for distributing a service which used the data.
The Licence: some highlights
2.1.2 only use the Transport Data in accordance with these Terms and Conditions and the Syndication Developer Guidelines, and not use such information in any way that causes detriment to TfL or brings TfL into disrepute.
2.1.3 You shall not make the Transport Data feeds available to any third parties, save with TfL’s prior written consent
2.1.7 only display the Transport Data for the intended use as approved in the registration form
4.4 TfL may change, suspend or discontinue all or any aspect of the Service or Transport Data, including its availability, at any time, and may suspend or terminate Your use of the Service at any time and for any reason
Whose data?
The TFL licence is completely at odds with government open data directives, the policies of the Greater London Authority Datastore, of which TFL is a member, and guidance from the GLA Data Committee:
“Robust requirements in all contracts to include ‘allow for free commercial re-use of public data’. This is included in the Data Charter for London.”
Issues
1. FOI compliance.
In making the data available, but subject to a restrictive licence which negates your rights under the Freedom of Information Act, TFL are clearly in breach of the Information Commissioner’s guidelines. The offer to enter into a developer contract with TFL is not a response to an FOI request. Is it possible that an Information Governance Adviser in the Information Access & Compliance Team is unaware of  this?
2. Commercial development using Public Sector Information
The terms of the licence are so onerous for developers that it is difficult to see how significant commercial investment in TFL data can take place. Visualisations and weekend apps, yes, but try raising investment for a service where your use of data may be suspended or terminated at any time for any reason.
3. Is it the policy of TFL to delay compliance with this FOI request to encourage developers to sign up for their T&Cs? Whilst that may look like a strategy from inside the bunker it can only serve to undermine innovation in services using transport data.

How to get that data – new data Q&A site launched

Leave a reply

“Data” and “data journalism” are buzz words of the moment and many individuals, organisations and sites are using data in different ways and working out the best ways interesting information can be used and presented.

You know ‘its’ going on, you want to get involved but you have no idea what data you’re looking for or where to start in finding it. That’s where Get the Data comes in. This new Q&A site built by Rufus Pollock allows you to ask questions such as:

  • “where to find data relating to a particular issue;
  • “how to query Linked Data sources to get just the data set you require;
  • “what tools to use to explore a data set in a visual way;
  • “how to cleanse data or get it into a format you can work with using third party visualisation or analysis tools.”

The site encourages users to ask questions, respond to others, tag questions with keywords, comment on and vote for the questions of others. Its clear that the aim is to build up a real community amongst users like sites such as Quora and users’ contributions are encouraged with the prospect of earning ‘badges’ for highly voted questions or answers. 

There’s already lots of interesting questions and useful answers on the site such as Is UK Local Government Spending Data from different councils being aggregated in any queryable way?

So if you want to use data but have never done so before or are unsure about where you could find what you’re looking for, log on, post a question and see if someone else can help!

How are the BBC handling their FOI data…?

4 Replies

Now, more than ever, public organisations are trying to make themselves just that: public.

“The genie cannot be put back into the bottle, however hard authorities try,” writes John Kampfner in today’s Media Guardian. “The information relationship has shifted, but the power relationship has not. The Democracy recession is gathering pace.”

The general public now have a thirst for that most gritty and honest of information and journalists more than ever are gaining access to it.

This means that everyone is becoming more wary of the way in which both public and private organisations handle, store and release information for public consumption and record.

This is why we are mentioning onebillionpageviews: the anti-license fee website have offered a single download that allows access to all the Freedom of Information requests that the BBC received (and hosted on their site) before 2008, which were later removed.

First of all, it is shocking that whilst everyone is so tuned in to the way in which public organisations handle data that the BBC would simply remove a huge cache of data from their site.

Secondly, for the BBC to do anything this brave and seemingly careless with their data when websites like NoTVLicenseFee are willing to keep that store of data available for the foreseeable future seems counter to their nature as a ‘public’ organisation.

They have also chosen to host every freedom of information request that the BBC received since the big removal and any more that arise in the future.

It’s great that a site like this wants to hold organisations to account and make sure that data is readily available amongst the rise in public curiosity into how their money is being spent, but it is also important that massive organisations like the BBC are careful to not be caught in the crossfire that grows out of the “democracy recession”.

The legal issues around recording ‘public’ council meetings – investigation roundup

4 Replies

Users of Help Me Investigate have been exploring the various legal issues surrounding recording council meetings. Although these are supposed to be public, some councils object to any form of recording, while others are happy to have them streamed live, and still others sit somewhere in the middle.

A key decision that motivated the decision was Southwark Council’s to allow audio recording of its meetings – but not video without prior permission. 

The full ruling is particularly informative. It can be found at http://www.scribd.com/doc/44926864/Broadcasting-council-assembly-meetings-legal-implications 

It mentions 5 areas for consideration, which I want to explore as part of an attempt to put together a guide for bloggers wanting to record public meetings:
  • The Data Protection Act 1998
  • The Human Rights Act 1998 (privacy)
  • “Procedural matters”
  • Defamation law
  • Copyright
In terms of data protection the document suggests that those attending public meetings will be considered to have given implicit consent to being filmed, and that clear signage that the meeting is being filmed – and mention by the chair at the start – should reinforce this. This seems sensible and reflects the practice of most television producers.

I’m not clear how audio recording might be affected.

The privacy issue is quickly brushed aside as the meeting is a public one and any issues are covered by the signage mentioned above.

“Procedural matters” seems to be a mix of retaining existing powers to hold part of the meeting in private, and asserting new ones. One of these is to interrupt recording if there is a disturbance, another to interrupt if a defamatory statement has been made. This is a curious inclusion, as the document itself recognises that council meetings are covered by qualified privilege – unless, of course, the defamatory statement is made in malice.

The final power allows the mayor to terminate or suspend recording if they “on advice” consider that continued recording may infringe the rights of any individual. This is a rather vague line and does not include a public interest exemption. The Local Government Act 1972, when stipulating instances where a meeting may take place in private, lists a number of reasons why this might be, but makes it very clear that public interest must take precedence in most cases.

The addressing of defamation demonstrates the curious priorities of local government. Once qualified privilege and other defences are explained two main fears come through: 
  • That a member of the public will make a defamatory statement, and publication of this on the council website will leave them open to legal action
  • That a member of the council might make a defamatory statement which, despite being covered by qualified privilege, would also constitute a breach of the council’s own code of conduct, “and webcasts may be used as evidence”. It’s not made clear whether the emphasis here seems to be on destroying evidence, rather than the preventing such behaviour itself.
The final passage on copyright gives the council an opportunity to retain control, saying that if they do with to enforce copyright they need to publish terms and conditions that include “any download or upload of the footage is not permitted without the written permission of the council and those featured in the same” It also suggests that users would not be able to upload the footage to sites such as YouTube because “you must be the copyright owner”.

However, copyright is actually held by the content creator – the person making the recording. If that person is not employed or contracted by the council then they can do what they wish with the footage, subject to any agreements made – and terms such as those mentioned above would be overly prescriptive. 

This issue is fleshed out nicely in the comments to Jim Killock’s blog post on a similar issue in Brighton, where the exemption for fair dealing in reporting the news is also mentioned. Richard Cant puts the copyright point succinctly: 

“The only copyright here belongs to Jason (assuming that he made the recordings himself). The council meetings are not literary, musical or artistic works. They don’t fall under copyright. That should be the end of it.” 

There is a second issue here, which is that minutes of council meetings are normally not copyrighted (see this Birmingham City Council page for an example). There seems no reason why coverage of that meeting should be.

That’s as much as I can write for now. As we continue to flesh out the issues around recording public meetings I’ll post again. In the meantime, please help us out by joining the investigation here, or request an invitation by emailing paul at helpmeinvestigate dot com.

Why the Freedom of Information Act makes GPs vulnerable under the new NHS

1 Reply

by Chie Elliott

The National Health Service is about to undergo a major overhaul, with strategic health authorities and primary care trusts being abolished and GPs gaining control of the NHS budget and responsibility to commission healthcare services. 

Some GPs are not happy about the pressure imposed by their additional administrative duties, which could detract from their primary role as doctors. Others may see the change as empowering, but more power probably also means increased public scrutiny – for doctors are not immune to the spying eye of Freedom of Information Act.

The resolution of case reference FS50295954 of the Information Commissioner’s Office involving a general practice in County Durham illustrates the point. 

The case was referred to the Information Commissioner, when the complainant failed to obtain full disclosure of the information he had requested to the practice. The ICO ruled in favour of the complainant. 

What happened? 
1. The complainant wrote an FOI letter to Weardale Practice requesting information on eight points mainly relating to the Practice’s Health and Safety regulations. 
2. Some of the information was released after the Information Commissioner intervened but some remained undisclosed.
3. The Practice tried to justify withholding information by using section 40 (2) of the Freedom of the Information Act, which refers to exemption on the grounds of personal data protection.
4. The Commissioner did not think disclosure of the information requested would cause damage or distress to the individuals whose data would be made public through it.
5. The ICO considered that Weardale Practice was in breach of the Freedom of Information Act.

You can read the Decision Notice in its entirety here. 

The law is clear on doctors’ accountabilities. Schedule 1 of the Freedom of Information Act covers which public authorities the Act covers. Part III of Schedule 1 relates to organisations and individuals in the National Health Service, with paragraphs 44 and 45 referring specifically to GPs and other medics:

“44. Any person providing general medical services, general dental services, general ophtalmic services or pharmaceutical services under Part II of the National Health Service Act 1977, in respect of information relating to the provision of those services

45. Any person providing personal medical services or personal dental services under arrangements made under section 28C of the National Health Service Act 1977, in respect of information relating ot the provision of those services.” 

Under the Freedom of Information Act, not only a practice but individual doctors working at the practice are considered to be public authorities and can therefore be held accountable. 

Nothing new there. The novelty could be in the fact that we may be evaluating our GPs less as healers of our ailments, more as politicians in charge of public affairs that impact our welfare. 

As long as the Freedom of Information Act rules, it is the patients who will have their fingers on the doctor’s pulse.

 

Workplace bullying and harassment in universities: the findings

Leave a reply

As Ben already mentioned in a previous post, AcademicFOI.com is a great example of how FoI data can be published well. One of the site’s latest reports is the findings of an investigation into workplace bullying and harassment in UK universities and higher education institutions. The results of the 132 FoI requests are published in an Excel document for ease of use.

What’s great about this investigation is that every aspect of the process is very public, very clear and well explained with notes covering key findings, incomplete data and even the threats received by Ian Benson from some institutions who did not want to disclose information.

So, what did the investigation aim to find out? In his FoI request, Ian asked a series of 14 questions related to bullying and harassment in the workplace between 1 January 2007 and 31 December 2009. Interestingly, 132 institutions responded in full and 13 failed to do so. A number of institutions also cited privacy concerns as a reason not to disclose and two uni’s  – Bristol and Salford – claimed it would exceed the 18 hour time limit to address.

From the data received back from universities, the site was able to produce a summary of key findings:

 – 1,957 university staff asked for support or advice due to bullying or harassment during 2007-2009 (true figure likely to be higher do to informal complaints)

 – 998 formal investigations were made into complaints

 – Of those, 764 concluded that no bullying or harassment had taken place and the rest were upheld (23%)

 – Top 3 UK universities Cambridge, Oxford and Imperial had an uphold rate of 54%

 – 41 UK universities had a 0% uphold rate (430 staff sought help, 56 left citing bullying and 169 investigations found no evidence)

The findings also highlight that a total of £1.35million was spent on legal fees and Benson points out that an ‘astonishingly high number of instances where invoices from solicitors were not broken down by reference to specific case’ were found.

Only 20 staff were dismissed out of 234 proven cases.

So many stories can be found within this data; be it an overall look, a regional look or a specific university of choice. Regardless of the subject, the use of FoI and the presentation of findings in this case is exemplary and should be encouraged by other users and sites interested in the public process of investigative reporting.

The site also has two previous reports on employment and tribunal claims and TV interview permission rights which are equally worth a look.

 

FAQs about the background to Help Me Investigate

Leave a reply
Every so often I get asked to answer questions about Help Me Investigate for someone's research project or article. I try to answer these in public so that others can also find the answers if they need to.

If you are doing an article or research into crowdsourcing, you'll find answers to basic questions about the site in these audioboos:?http://audioboo.fm/tag/hmi

There are videos of presentations I've made about the site here:?http://www.bbc.co.uk/journalism/blog/2010/11/event-face-the-future-1.shtml

And here:?

And you can find interviews with me about Help Me Investigate on the Columbia Journalism Review:?http://www.cjr.org/the_news_frontier/q_a_paul_bradshaw_founder_of_h.php?page=1

And, going back a while, at MediaShift (just before the site was due to launch):?http://www.pbs.org/idealab/2009/06/help-me-investigate-paul-bradshaw-on-crowdsourcing-investigative-reporting153.html

How you should use the Freedom of Information Act…

Leave a reply

The Freedom of Information act, despite becoming common practice amongst the new wave of journalism, is still a very new concept to a great deal of the public, and finding the best way to present data that is acquired through this act can be a difficult task.

AcademicFOI are the perfect example of an organisation who are doing great things with useful data.

First of all, they ask questions and make sure everyone knows what questions they asked; the whole process not just public but easy to understand.

Take, for example, the most recent investigation into workplace bullying at UK universities.

The Freedom of Information request covered fourteen questions, covering all the universities.

This data is daunting, even to the most experienced, so for the public to even attempt to consume this raw data there must be a lot of work done.

They immediately outline the key findings, the mission groups and the extremes; the ‘newsworthy’ data that provides the key news angles and gives the most interesting answers. Further interesting findings are then outlined and explained, clearly.

The next section is where they really come into their own; they unveil every question asked, explain what they should have asked with hindsight, and where they could or have encountered problems.

This is not your standard public process as they make public every aspect of the investigation. The basic excel data is available for download and their findings are presented in tables categorised by the question they apply to, so nothing is hidden, their news values are openly identified and the data is there for you to find answers to any questions you may have.

Although, it would be useful if then information were available in Google Docs formats as well, to help move along the adaption of data for other journalists.

All in all, if there was a way to deal with data that best represented what the Freedom of Information Act is all about, it is this way. Making everything public is what the act was created for, and allowing your audience to interpret the data rather than consume it is something that more organisations need to become open to.

The Climate Camp story continues…

Leave a reply

On 23rd August last year, it was widely reported that “a substance similar to diesel or vegetable oil was poured onto the carriageways” that made up the A720 and A8 roads of Edinburgh.

As a result, protests took place where “hundreds of campaigners spent a week occupying the Gogarburn grounds of RBS’ headquarters protesting against what they believed were environmentally damaging investments”.

The Help Me Investigate investigation began with the legitimacy of the ‘oil slick’ claims, but eventually evolved, alongside The Guardian’s investigations, into the protests themselves.

Yet still, over four months later, Lothian & Borders Police have not released information relating to the costs of the protests, claiming Section 17 (information not found).

The first and second Freedom Of Information Requests were stalled, with Lothian & Borders Police originally claiming that “it will take some time before all costs (expenses etc) have been accounted for and I would therefore suggest that you re-apply for this information in about two month’s time”, and then, come October, announcing that “it is unlikely that this information will be fully collated until the end of October (at the very earliest) and I would therefore suggest that you re-apply after that time”.

The Guardian found similar roadblocks, and with this update last week, it seems that they are still no closer to finding any answers.

The Guardian tells us that “on three occasions since August we have used the Freedom of Information Act to ask the local police force to tell us the cost of policing the protests” but all have been refused.

The report also references the Help Me Investigate findings relating to traffic logs on the day of the alleged oil-spill.

The best quote to leave the story so far on is this:

“The Force Information Unit recommend you resubmit your FOI early in the New Year, as they hope to have a figure available in January.”

We’ll see.

Legal Leaks and toolkits for journalists…

Leave a reply

“Journalists play a central role in initiating and stimulating public debates but face constant challenges in accessing information from public bodies” and sadly, this is growing more and more difficult for the common journalist.

This is why people like Legalleaks are producing toolkits for journalists: “The Legal Leaks project not only explains what access to information laws are and how they should be used, but also creates an international network of journalists who could help each other access information”.

The toolkit itself consists of a series of step-by-step, easy to navigate instructions that try to provide solutions to all the problems you may have whilst conducting investigative journalism. It is designed as a cross-platform guide to finding information in your, and in detail for this guide other, countries.

They set out any differentiation in the laws of each respective country and allow for the practice of investigative journalism to go on without fear of retribution.

On a less fearful note, there is also a mass of information on how to speed up and simplify the commonly infuriating process of applying for (and then waiting for) information, as well as balancing projects to coincide with the waiting time.

It’s worth taking a look at if you work in any capacity as a journalist, because the advice will always be useful in obtaining, compiling and deconstructing data.