Home  |  Sarah  |  Services  |  Blog  Contact

Tuesday, 22 Jul , 2014

Government Forces Through the Data Retention & Investigatory Powers Act

Share this article

This article explores the history, content and criticisms of the Data Retention and Investigatory Powers Act (DRIP or DRIPA) which was pushed through parliament in 7 days using emergency powers normally only wielded when the country is at war. UPDATE 17th July 2015 The case brought by Tom Watson and Liberty has been won. The High Court says the DRIP […]

This article explores the history, content and criticisms of the Data Retention and Investigatory Powers Act (DRIP or DRIPA) which was pushed through parliament in 7 days using emergency powers normally only wielded when the country is at war.

UPDATE 17th July 2015

The case brought by Tom Watson and Liberty has been won. The High Court says the DRIP Act should be ‘disallowed’ giving the UK government until March 2016 to put a replacement together. A somewhat hollow victory as the DRIP ACT had a built in sunset clause which required repealing the act within similar timeframes. It does however avoid the same situation that arose with ‘temporary’ anti-terrorism legislation.  It was repeatedly renewed with little or no challenge.

UPDATE: 22nd July 2014

Liberty’s Press Release
More in the Guardian
Government Response (pdf – Looking at European Court of Justice finding on the Data Retention Directive and arguing DRIP meets those legal challenges)

The bill forces ISPs and telecommunications companies to retain communications data for up to 12 months and requires them to grant access to meta data and contents of communications (if requests are formally approved) to a range of secret service agencies and other bodies.

Most, but (controversially), not all of these data retention and access rights were already in place under the 2006 EU Data Retention Directive, but the European Court of Justice quashed that in April, citing breaches of human rights law.
MPs were given a brutally short time to review it.  The bill was published on Thursday 10th July, the House of Commons approved it on Tuesday and the Lords waved it through on Wednesday.

On Thursday 17th at 4.36pm it gained royal assent and became the DRIP Act. You can find details of the process, minutes, who voted which way and other information here. At the end you’ll find an assessment of key, bill related concerns, costs and benefits.

Just in case you’ve been in an isolation tank since Wednesday night, we’re talking about the surprise new law the UK government has pushed, as hard as possible through parliament. The government achieved it’s aim to have it approved within a week.  To make that happen, they have used emergency powers usually only wheeled out during wars or other times of national crisis. This article in the Telegraph provides some good historical context. They also timetabled it a week before the summer recess. An incredibly unpopular move as it squeezes time available, without a special extension of the parliamentary session, to raise challenges.

All of that led to MPs spending an almost unprecedented amount of time discussing the timetabling of the bill in advance of the debate proper.  David Winnick MP called it “A mockery of what parliament should be doing” a sentiment mirroring much of what was said. It is a replacement for the now defunct 2006 EU Data Retention Directive (DRD). The DRD was incredibly controversial. Privacy campaigners gained enough backing to have a case referred to the European Court of Justice, citing breaches of human rights legislation.  They won the case and the DRD was quashed in April this year. The Court listed three main objections to the directive.

  • Firstly, that it covers all individuals, all means of electronic communication and all traffic data “without any differentiation, limitation or exception being made”.
  • Secondly, that it failed to establish any objective criterion to ensure access to the data could only be used for such offences that were serious enough to warrant such an interference with fundamental rights.
  • Thirdly, in relation to the period of retention, that it made no distinction between the categories of data on the basis of the persons concerned or the possible usefulness of the data in relation to the objective pursued.
Bill picture
Click on the image to go to the government DRIP Act page

Version 2.0 (my link to the draft is now broken) removes some things from the original, but adds others. It proposes extra amendments to the RIPA or Regulation of Investigatory Powers Act,  which was, (before the the Data Retention Directive), the primary legislation setting out legal rules for how, when and why people can spy on you.

It also clarifies the type of data the government can order phone companies and internet service providers to retain, how long they can tell them to retain it (a maximum of 12 months) and the rules the secret service and others must follow to gain access to that data.

If you are not familiar with RIPA and how it relates to the new bill, this article looks at both. It also looks at whether DRIP is legal given the ECJ ruling on the DRD.  Well worth reading;

“What are Drip and Ripa – is the UK’s ’emergency’ new snooping law legal?” Expert Reviews, 11th July

UPDATE 11th July10 pages of accompanying regulations were published this afternoon. With 3 impact assessments that set out the government’s justifications in more detail and their cost/benefit statement.


Communications Data (CD) Retention Legal Interception (LI) Privacy  – No separate privacy impact statement was released, relevant words from the other statements are;

“In relation to data retention, in addressing the ECJ’s concerns, where possible, the new legislation will go even further in safeguarding privacy.

It is assessed that implementation of the proposed legislation is capable of being fully compliant with the Data Protection Principles and the Data Protection Act 1998.

With regards to interception, this legislation is clarifying the status quo. There will be no new privacy risks associated with the interception provisions in this legislation. The UK already has one of the most stringent oversight and authorisation systems for investigatory powers in the world.”

Update 15th July

Amendments to the bill, proposed by Labour (in the 47 minutes, on 14th July, they were given to do so) were defeated. Most notably the one proposing the bill is reviewed in 6 months, rather than the planned 2 years.


Ironically, given most MPs have only had 6 days to consider this bill, the justification was that 6 months is not long enough for a robust review of surveillance powers.

Much is made of the fact that data can only be accessed if necessary to safeguard national security, fight crime and protect the UK’s economic wellbeing (MPs demanded demanded a clarification of the latter yesterday. Home Secretary, Theresa May confirmed it is economic wellbeing, but only in so far as it can impact national security). The soundbite from the Prime Minister on justifications:

The #StitchUp hashtag is not a comment by me, it was the trending tag for related posts, prompted by a comment from MP Tom Watson, in a BBC Radio interview given when the story was breaking.

The Home Secretary added the below statement. Focusing on the risk that telcos, currently retaining data because of retention ordered under the Data Retention Directive, may now delete it). She is a staunch and long-time advocate of beefier investigatory rights for the police and security services and the person who would sign data access warrants under the new law.

“Without this legislation, we face the very real prospect of losing access to this data overnight, with the consequence that police investigations would suddenly go dark and criminals would escape justice.” The Guardian 10/07/2014

This legislation does not allow access to the content of calls, or other electronic communications unless a warrant is authorized by a secretary of state. Retention rules only apply to communications metadata. For example, who contacted who, when contact was made and how long contact lasted. That sounds very reasonable, but this slide puts it into perspective:
BcvKBVuIcAADm-K However, there are issues and critics that raise doubts about the restriction of access to just metadata;

  • The feasibility and cost effectiveness of telcos mining just metadata to retain, so they may, by default, store complete content.
  • Historical hedging about what, exactly, can and will be accessed. To quote Caspar Bowden;

“[Rules around access to content are] extremely devious, like Kafka + Poe….If you think you understand it, you probably don’t”

  • Legal and academic analysis of bill wording.  In particular this open letter from academics to parliament, stating that the bill clearly extends powers to intercept message content and this paper that looked at pre-existing surveillance powers (after all, DRIP powers are all supposed to be in place already), submitted as evidence to the joint committee considering the old Communications Data Bill.

In addition, while retention and control over access are specified in the bill, there are no restrictions specified for how data is analysed.  That raises concerns about validity and integrity of findings.  To again quote Caspar Bowden

“Data processing for national security purposes is exempted from almost all control under the Data Protection Act”. From this 6 page paper, produced in response to the same challenge about the 2012 Communications Data Bill.

Another change is to the definition of a “telecommunications service” which many are concerned is too broad.  It potentially puts any company who handles electronic user conversations involving UK citizens, into scope of the bill. Most recently, commentators agreed that includes Microsoft as owners of GMail, Facebook, Twitter and providers of any similar services. That conclusion looks solid, because the bill includes clauses explicitly extending retention and interception powers to overseas telcos servicing local users.

It also refers to a range of new oversight measures, designed to allay fears that this constitutes an ‘all you can sneak’ buffet of all our calls, and other digital exchanges, open to a range of covert and not so covert bodies.

More generally there is concern about the overall ambiguity and open endedness of the bill.  In the top section it appears to permit the government to change the retention period or change the bill whenever they deem it necessary.  Any changes would be subject to the normal judicial review and vote, but given the way this bill is being handled, that may not provide much comfort. This article gives an expert breakdown of government rights now and then if the bill passes:

“Dissecting the emergency Data Retention and Investigatory Powers Bill” by Graham Smith (@CyberLeagle) a private practice lawyer who specializes in internet and online communications law.

Below are some more good articles looking at various aspects of the bill.  I’ve included a link to Theresa May’s speech to parliament on 10th July.  They are all (except Theresa’s speech) robust critiques of the bill. Not my bias.  Those who support the bill and most media outlets are suspiciously quiet, as pointed out by Casper Bowden (independent privacy researcher, ex-Chief Privacy Adviser Microsoft, ex- )

More time is needed to drive out implications before the DRIP bill is passes into law, not least by members of parliament who have to vote on it. A situation made worse by the regulations published at 4pm on Friday 11th. Most backbenchers only became aware of the specific nature of this bill at 8am on Thursday morning, just 30 minutes before the story broke. They then had to wait with the rest of us to get a peek at the actual words.

A number of people, including some of those MPs, view this as a shocking show of disregard for some key principles of democracy:

Update 15th July

My MP Richard Bacon MP did not attend any bill debates and was absent for all votes. Like the vast majority of our elected representatives. How they all voted

DRIP Bill Debate
DRIP Bill Debate
Expenses Debate
Expenses Debate

I’ve been trying to apply the risk management skills I use in my day job to look at DRIP bill related risks, but there are still so many uknowns, especially around the real threat level and potential impact.  Nevertheless I’ve put some shape round the bigger concerns that can impact everyone. After each section I say what I think, which you may or may not agree with, but now is the time to work out how you DO feel.
I start with the heavy handed and simplistic justifications thrown out by the Prime Minister (from the soundbite tweet at the top of this post).  He heavily stressed these at the press conference held soon after the bill news broke.

Is the current threat of terrorism & pedophilia adequate justification?

Of course we must give the government all reasonable means to find and track terrorists, so they can minimize the likelihood of successful attacks on our people, infrastructure or institutions.  The same thing goes for other dangerous criminals. I can’t conceive of the anguish if anyone I love was harmed in a terror attack and as a Mum of two small girls, the thought of them being preyed on by a sexual predator makes me feel physically sick, BUT I don’t know how current, severe, or common those threats are and no-one is going to give me access to the reports I would need to find out….

….unlike Richard Dearlove, ex MI6 chief, who explains his understanding of the scale of the terrorist threat in this 7th July article: “Islamist terror threat to west blown out of proportion

Increasingly Sceptical – I, like most of the rest of the nation, am vulnerable to FUD  – my definition: Using Fear, Uncertainty and Doubt to deliberately make individuals feel insecure and therefore more dependent upon you. What is the realistic level of threat right now and does it match how scared we are made to feel? This Open Rights Group article on proportionality puts more context round this.

Will ISPs and Telcos delete all data stored in response to DRD retention requests, thereby grievously harming on-going counter-terror and criminal investigations?

This was the main concern raised by Theresa May.  According to the Independent on 11th July:

“Internet and phone companies, mainly in the United States, had warned they would soon start to delete such records following a ruling in April by the European Court of Justice (ECJ) that an EU directive infringed privacy”

In April a Swedish ISP made a high profile statement about their intention to delete retained data after the ECJ overruled the legality of data retention requests.

Possibly, but do two wrongs make a right? – It depends if you think the ECJ ruling was wrong and/or doesn’t apply to the UK.  Even outside the remit of RIPA and the DRD, a main pillar of UK data protection legislation is the need to delete any personal data, as soon as you have no further need to retain it. There are reportedly some key on-going cases that need to sift through retained data, but does that justify ignoring the breach of human rights mass indiscriminate data storage represents?

Why can’t the needs of those investigations be served by the pre-existing data access rights embodied in RIPA? This goes to the heart of the debate.  Is the additional loss of control over our data and the data retention burden to be placed on a wider range of companies, a reasonable price to pay for the advertised reduction in risk? Are we confident that the risks (breaches of data stores, misuse of data access powers, misuse of legally accessed data, potentially inadequate oversight) don’t outweigh advertised crime fighting benefits?

Is this less about current threats and more about closing down legal challenges about prior illegal surveillance?

In the wake of Edward Snowdon’s revelations, it has become more and more apparent that governments may have been bypassing pre-existing surveillance laws or approval processes.

“The jointly backed powers will confirm that foreign-based firms should hand over limited data harvested in Britain – a move seen as a tacit admission that Edward Snowden, the former National Security Agency contractor, may have revealed surveillance work that did not enjoy international legal backing in his high-profile leaks” Independent 11th July.

There is a high profile case challenging GCHQ surveillance practices in court at the moment

“The case – also brought by Amnesty International and the American Civil Liberties Union and other groups – centres on the alleged use by UK intelligence and security agencies of a mass surveillance operation called Tempora.

The UK government has neither confirmed nor denied the existence of the operation.

But documents leaked by whistleblower Mr Snowden and published in the Guardian newspaper claimed the existence of Tempora, which the paper said allowed access to the recordings of phone calls, the content of email messages and entries on Facebook” BBC News 14th July

Don’t Know – I haven’t been able to get hold of a legal opinion on how DRIP becoming law might impact the GCHQ case, or others that might be brought.  Are there likely to be further Snowdon revelations that open the door for more legal challenges about abuse of surveillance powers? Cases that might be weakened by DRIP making practices legal, that were not so at the time?

Is the ECJ ruling a political excuse to give the secret service the RIPA amendments they’ve always wanted?

This point came out of a number of online articles and discussions, but needs some background to consider properly. The Communications Data Bill (Wikipedia gives a good rundown on history, contents and challenges) was christened the “Snooper’s Charter” by Nick Clegg. On 23rd August 2012, Caspar Bowden was invited to submit evidence to the joint parliamentary committee considering the draft Communications Data Bill.  You should read it, it is informed, impactful and equally relevant to the DRIP bill debate. The CDB was sent back to the drawing board in December 2012 after Nick, other Lib Dems, MPs from other parties, the public and much of the mainstream press condemned it.  Subsequently there was some discussion of a redraft which led to Nick Clegg re-stating his condemnation of the Bill:

“What people have dubbed the snooper’s charter – I have to be clear with you, that’s not going to happen.

“In other words the idea that the government will pass a law which means there will be a record kept of every website you visit, who you communicate with on social media sites, that’s not going to happen”.  April 2013 BBC

Still the secret services were loathe to let it go, as illustrated in this article: “Counter terror chief renews fight for “Snooper’s Charter” (The Guardian November 2013) and in the same month, an Intelligence and Security Committee call for evidence for a review of existing surveillance rights.

Dr Paul Bernal responded to that call with the following paper in December 2013 “Communications Surveillance – A Miscast Debate – again very much worth reading as it provides excellent context for the current debate.

It looks suspicious – The extension in the scope of surveillance rights (in particular the redefinition of a telecommunications service provider and inclusion of oversees telcos) is significant.  Justifications may not quite stack up (the impact statements are concerningly generalist and the terror threat, according to Richard Dearlove, may not be such a clear and present danger).

Did a securocrat note the potential to piggyback off the ECJ ruling to get those RIPA amendments? This time forcing it through too fast to let a head of critical steam build up among MPs. That article, showing ex terror chief Farr’s defense of the need for DRIP-like rights seems to back this up.  Or maybe I’m mistaken, after all, quoting Mr Clegg during Thursday’s press conference:

Is planned oversight enough?

Having slept on it, I’m still not sure whether the government could always withstand pressure to share our data from other nations, or our own secret services. However, I choose to believe that their intention is not to flagrantly abuse our right to privacy.  This is based on what I initially read about planned controls, but since additional regulations were published on Friday, without any explicit detail about most of the promised controls, I’m feeling less comfortable.

Not Sure – So much depends on the advertised oversight being implemented, rigorously operated and transparent about all accesses requested. At the moment we’re being asked to take it on faith that those things will happen.

Is stored data safe?

Am I confident all that data stored by ISPs and telephone companies won’t get inappropriately accessed or stolen? We have, after all,  created and advertised a mile-wide personal data target for the internet underworld to aim at (with the added motivation that a breach would embarrass the government).

Not Sure – That view is colored by my current day job – overseeing assessment of supplier security, but I’m not the only one who will think it.

Back when the Communications Data Bill was still alive and kicking the guardian published this article about the risks associated with mass data retention; “MPs call communications data bill ‘honeypot for hackers and criminals’ I wonder what’s changed? I did see, in part of the bill, that there are likely to be government mandated standards for security of stored data.  That’s yet to be confirmed and if it is, I will have a raft of questions about how effectiveness of those controls will be assessed and reviewed.

Are legal implications clear?

Can I draw robust conclusions about the legality and longer term implications of the bill without expert legal input?

No – And more to the point, neither can MPs.  That analysis is starting to come out (in particular, on 14th July, this superb and comprehensive analysis by Tom Hickman for the UK Constitutional Law Association), but it may well be too late to overcome political pressure on MPs who would, if better informed, vote against the bill.

Will there be more legislation and regulation to come?

Will more rules come out of the post-recess look into other new protections that might be deemed necessary (work Mr Cameron alluded to repeatedly during Thursday’s press conference). Will that mean I will have to reassess how I feel about implications of the bill?

Very Possibly – Next parliamentary session work will begin to review all surveillance legislation as the current Bill has a formally limited life and is due to be repealed in 2016.  A replacement bill or fundamental change to RIPA is likely to be the outcome. There are also critics who highlight the risk that this “sunset clause” will be overturned, the same as it was for the Prevention of Terrorism Act. Another piece of legislation supposed to be temporary, but renewed year on year by parliament.

Does this impact on personal data ownership rights & privacy?

Do I accept there is some trade-off between national security and individual rights?  Does the bill impact your right to control access to your personal data? Am I still doubtful about the amount of control I would cede in return for an uncertain reduction in risks to my family and the country as a whole? The government impact assessment for privacy, reproduced again below, gives robust assurances that this legislation is in line with all existing privacy legislation and refers again to planned oversight.

“In relation to data retention, in addressing the ECJ’s concerns, where possible, the new legislation will go even further in safeguarding privacy.

It is assessed that implementation of the proposed legislation is capable of being fully compliant with the Data Protection Principles and the Data Protection Act 1998. With regards to interception, this legislation is clarifying the status quo.

There will be no new privacy risks associated with the interception provisions in this legislation. The UK already has one of the most stringent oversight and authorisation systems for investigatory powers in the world.”

However the analysis of extras in DRIP (e.g. telco redefinition and overseas extension of retention and access powers) doesn’t seem to bear out the “no new privacy risks” statement and the reference to compliance with the ECJ ruling is qualified:  The DRIP bill will “where possible” safeguard privacy to address the ECJ’s concerns.

Yes to all – But I’m reserving judgement.  I’m pragmatic enough to accept that law enforcers and spies sometimes have to sift through irrelevant data to find evidence needed.  I know, from my work in the IT and InfoSec, trade that no clever algorithm is going to surgically target and spit out JUST confirmed terror suspects when data is reviewed. The crucial point is whether good quality oversight will be implemented and how well that would work, in the long term, to prevent abuses of access to data belonging to both the innocent and guilty.

Does the use of emergency powers set a worrying anti-democratic precedent?

Has the use of emergency powers set a precedent for eroding parliamentary accountability? Will there be more related laws, or amendments to existing laws, pushed past the usual consultation with our elected representatives? Do I think it’s part of a slippery slope trying to desensitize us to a persistent increase in state powers? I’m not the only one asking these questions.  In the article by Tom Hickman referenced above, there’s fairly damning evidence of a an upswing in the amount of fast track legislation:

“When the Government was fast-tracking through Parliament legislation overruling a court judgment that found that thousands of benefits sanctions had been unlawfully imposed, the House of Lords Constitution Committee lamented that it was the latest in “an undesirably long line of recent fast-track legislation” and registered its concerns with the House (a strong thing for that Committee to do – see 12th Report Session 2012-13 HL 155 §6)” For the UK Constitutional Law Association

I sincerely hope not – Of all the concerns about the bill, this is probably the one that scares me the most. Just as I choose not to believe the government will willfully abuse their data access rights, I choose not to believe we are being subtly (or not so subtly) manipulated to tolerate more and more autocratic control from the government.
What I DO believe is that the current ’emergency’ looks less and less like a crisis and more and more like a disturbing abuse of parliamentary privilege. It’s clear that many MPs do not grasp the finer points of DRIP, RIPA and other associated legislation (see Dr Bernal’s post for one example).  Are you comfortable having such a significant change to surveillance rights rushed through without giving MPs time to understand what they’re voting on?

Now it’s time for you to make up your own mind. Not that it matters for a while.  The DRIP Act will stand until it expires in 2016.  The first time the public will get to formally consult is just prior to that expiration.

Opinion: Paying to play with our personal data – is it ok?

We’ve migrated from ‘Hot or Not?’ to being held virtually hostage by many of the digital platforms we rely on today. In the midst of that a new processing paradigm has emerged. Myriad startups want to pay to play with your personal data. Can this tackle on-going...

In AI we will blindly trust…

...and the architects, designers, data scientists, and developers will think we are nuts I've been driven back to the blog to talk about one very specific aspect of privacy, data protection and Artificial Intelligence (exchange for Machine Learning or Algorithms as...

Data Protection, Security, and the GDPR: Myths and misconceptions #2

Welcome back! This is a shamefully delayed sequel to my first instalment of security themed GDPR thoughts: Data Protection, Security, and the GDPR: A fraught and fuzzy relationship. Here I look back again over my pre-privacy IT and InfoSec career to spot things likely...

Where and to whom does the GDPR apply?

Yeah, I doubted my sanity going at this one too, but here I am, because working out whether or not the GDPR would apply in different practical and geographical circumstances is proving harder than it really should...for everyone. This regulation has been my almost...

GDPR – You’ve analysed the gaps, but can you close them?

  There is a critical gap for most firms: An inability to interpret and leverage gap analysis, data discovery, and mapping output to actually implement technical data processing change. This article is about the challenges most large firms are facing when trying...

GDPR – The Compliance Conundrum

There is one question related to the General Data Protection Regulation that will arguably cause more ulcers than any other: How much is enough? In some portions of the GDPR 'good' is straightforward. In many others we are asked to respect principles of fairness and...

Opinion: The role of automated data discovery in a GDPR programme

Do you have any online profiles or posts featuring those 4 magic characters: G D P R? If so, whether you are a business decision maker, IT body, security body, charity boss, employed data protection pro, or job seeking data protection pro (less and less likely), you...

When Business Culture Eats Cybersecurity For Breakfast – Part One

A four-part story of budget cuts, blamestorming, breaches and massive bumps in the road to mature security. Wild Speculation & IT Transformation Do you remember Nick Leeson? On February 23rd 1995 he sent a fax telling bosses at Barings Bank he was ill and wanted...

Cyber Insurers Dictating Cybersecurity Standards?

A run down of the key challenges with choosing and using cyber insurance called out in the last few months. It looks entirely possible you will have 'adequate' security dictated by your insurers, so it is your job to understand the risk based yardstick they're using...

There Is No Such Thing As Information Security Risk

Having worked in IT and Information Security for 13 years, I've come to the conclusion that there is no such thing as information security risk. There are just business risks that have one or more security or IT related causes. There is a fundamental and persistent...