Corporate security

Ohhhhh P M

Head in HandsI have been watching coverage of the OPM breach(es) with consistently weary interest. Why then haven’t I offered an opinion? As a security blog writer surely that breaks some kind of tacit contract with my modest, but fabulously loyal band of followers?

Firstly I didn’t want to shout above the noise, secondly, just like Sony, this is taking time to distil down to some verifiable facts and lastly I was trying to find a constructive, non-repetitive angle. Then it struck me. Everything about this is repetitive. The only question is which brand of repetition are we talking about for both OPM and their suppliers?

  • Common or garden neglect of security over years
  • Patches of good security where they got a chance to embed it and run budget survived security spending boom/bust cycles
  • ‘Great’ security by some out-dated yardstick that bears little resemblance to their real risk landscape, and/or
  • Really well chosen, well implemented and risk relevant controls…that were still powerless to stop (or more realistically, detect and mitigate) sophisticated and patient attacks.

It’s the same question every time there’s news of a breach. Some folk will have months of fun unpicking the who and how of the attack, but so much of what goes wrong is down to folk banging on doors until they find the one that’s unlocked. They may be armed with professional social engineering lock picks or whizzy 0day door code crackers. The fact is most cool toys are never needed, cos even if a door IS locked, the frame is often rotten, or it’s opened for them by a hapless user.


There’s clues to OPM’s context for this in one of the federal flash alerts linked to the breach. This time from the independent Inspector General’s office within the Office of Personnel Management: “There is a high risk that this project will fail to meet the objectives of providing a secure operating environment for OPM systems and applications,” it said. The background assertion being that upgraded security systems that did eventually detect one of the breaches, are themselves at risk of failure due to dramatically underestimated network scope and related costs. In response to that Katherine Archuleta, OPMs now ex-Director, said that legacy systems were a big part of the problem.


The only significant difference between OPM and most large companies is the number and type of attackers interested in what they’ve got and the potential impact causable with data grabbed. Oh and the fact they haven’t had a breach yet that deserves this much coverage.

Maybe it was Chinese state actors, maybe it wasn’t (Vocativ have found what looks like stolen OPM data on darknet websites – about $140m worth at advertised prices – state actors typically don’t sell off their spoils). So why not have a go with the Duo Labs attribution 8 ball and see which common blame-recipient floats to the top because…

…no matter who it was, key points don’t change

Both Tripwire’s blog and Brian Krebs have good breach timelines. I’ve pulled out bits that cut to the heart of it from my perspective:

  • Someone got themselves and malware in. Perhaps using data harvested during the Anthem or A. N. Other breach. Perhaps sneaking in through an unpatched vulnerability, riding in on a compromised mobile device, landing on an endpoint after some phishy clicking, or knowingly carried in by a coerced or malicious insider.
  • Attackers reportedly camped on the network for over a year, did their reconnaissance and harvested data too quietly to set off most alarms. On the other hand, perhaps there weren’t any effective detection tools and alarm settings at the time.
  • OPM reportedly detected the March 2014 hack and thwarted efforts to exfiltrate data using their IDS capability (Intrusion Detection Systems).  They gave no-one cause to believe data had been compromised, but despite this ‘win’ the end game and other linked naughtiness was still going on all over the place.
  • Unsurprisingly key third parties (who handled huge quantities of the more and less sensitive types of personal data being targeted) were also being attacked. The mothership reportedly declined to notify USIS (United States Investigation Services) and KGS (KeyPoint Government Solutions), of their problems, despite contracts stating that threat and incident information should be shared.
  • USIS subsequently reported a June 2014 hack, then two hacks of KGS were reported (one that looks to have happened in September and one in December). The former appears to date back to about the same time as the original OPM breach. The subsequent December 2014 OPM breach has since been linked to credentials stolen from KGS.
  • After USIS spotted and notified relevant agencies about their June compromise, all technical and contractual ties with them were cut. That cost 2500 USIS staff their jobs and ultimately pushed the firm into bankruptcy.
  • No matter what security holes OPM had or hadn’t plugged, no matter how well or poorly they collaborated with 3rd parties to define a consistent, worthwhile level of security, no matter how lax or diligent 3rd parties were implementing and maintaining that security, it wasn’t their fault…

…it was an APT,

I’m not sure you got that.

It was an APT.

or

Advanced Persistent Threat

or

Absolution for Persistent Thoughtlessness

What can you do if someone super motivated and able directs long-term effort at finding a way into your network and has the patience and means to identify, navigate to and extract the juicy prizes? Who can defend against that? There’s been much written about APTs and some of the best minds in the trade say there’s little defence against the really clever, sneaky and patient stuff….

….except when it might not be so clever and sneaky. Could it possibly be that a door or window was left unlocked and the alarm system was missing, not fit for purpose, set up badly or just not switched on?

Of course, being the US, that whole APT thing isn’t an absolution. There’s already a class action lawsuit being prepared, which may or may not force out facts about negligence by OPM and/or their associates.

In my opinion there’s not really enough info to start waving around ‘Things that could have prevented the OPM breach” or “How to avoid being the next OPM” articles with various silver bullety solutions, although there’s plenty out there (the KERCHING as dollar signs appeared in the eyes of vendors and consultants was audible). The full facts may come out in time, or as it’s the US government we’re dealing with, lots may remain foggy.

Just in case the veil is lifted, here’s the stuff on my ‘want to know’ list:

  • What allowed the original compromise and let attackers poke around nicking this and that until someone noticed months later? Was it some master cyber ploy? Sakula malware – fairly decisively associated with China – seems the most likely vehicle, but it is a vehicle. It needs to get in, be allowed to move around and get left to talk to and fire data off to remote servers. Could it also be (like 99.9% of system vulnerability related breaches reported in the Verizon DBIR) down to known and unpatched holes, plus or minus inadequate monitoring, detection, alerting and incident response capability?
  • What part did people play? Was there an insider who could have been screened, trained, access limited or activity monitored and stopped? Were attackers helped knowingly or unknowingly with a click, download, plugged in device or ill-advised conversation?
  • Were previously harvested credentials and personal information implicated? There are the reports of KGS credentials being used to facilitate the subsequent OPM breach and other articles pointing to user data that’s been available to hackers since the Anthem breach compromised tens of millions of user records. Were sensible precautions taken after other associated breaches? How did OPM and third party privileged access control, authentication standards, security system enhancements and potential fraud flags stack up to combat information stolen from other domains and sites? On the other hand you can’t go changing your social security number or address in a hurry.
  • Was OPM’s reported detection and halting of the earlier attack really value-add activity? Was their sudden detection of bad stuff to do with that upgraded security, or had things got so loud and messy someone hard of hearing would have asked for the volume to get turned down?
  • Was a decision to take down the online background check system E-Qip really non-breach related? OPM director Katherine Archuleta announced on Monday it would be taken offline to ‘proactively’ fix a discovered vulnerability.

“The actions OPM has taken are not the direct result of malicious activity on this network, and there is no evidence that the vulnerability in question has been exploited,” an OPM statement said.

That’s causing a  lot of concern about what exactly the vulnerability is, whether it has been exploited and how the rapidly growing backlog of federal employee checks will be dealt with. A fun potential chicken and egg with insider risk there and one can’t help wondering whether security staff who flagged vulnerabilities before the breaches could have persuaded OPM management to delay system changes going live, let alone take the live system down.

  • Was security at USIS and FGS as bad as OPM are making out? There’s obvious political mileage in banging that drum despite some bodies saying they had only recently proved they met defined and required standards.

In other words, are OPM helpless victims of a virtually impossible to stop sophisticated and determined attack, OR are they masking persistent muppetry and inadequacy of the security they required internally and from third parties by buck passing and crying wolf?

That plays to the bigger debate that’s being had around security…

If a determined attack will always work and there’s no way to be 100% secure, what’s the point? Surely it’s better to sidestep the big capital spend needed to sort out years of slapdash development, cludgey fixes and out of step policies? Better to invest in detection and post-incident impact mitigation, both directly and by transferring risk to an insurer?

Spending loads of cash on creating a useful level of security, to then not mitigate all risk, is the toughest of tough sells to the business. Far more rational to fly the defeatist flag…

…UH No!

Giving up just doesn’t make sense to me personally or professionally, it won’t make sense to regulators and it sure as heck doesn’t make sense to users and customers. Perhaps ask the originally estimated 4 to 18, now 21.5  MILLION odd federal employees (a number OPM wern loathe to confirm) who were put through standard HR checks or security screening by OPM et al. Senator Mark Warner has already written to the IRS asking them to help affected people avoid or mitigate identity theft. So what could be done?

If this was a soccer game, you wouldn’t stop playing because there was an off chance of a stray ball or kick to your soft places. You rely on team mates and opponents knowing and adhering to the rules of the game, a fair and observant ref who spots and acts upon bad and dangerous behaviour and strict sanctions for fouls. You also wear appropriate protective clothing, always keep your eye on the ball, keep your eye on the guys who have a tendency to play rough, swerve or turn when something looks to be headed that way and yep, occasionally spend half an hour in the foetal position wishing you were dead and trying not to throw up…

…but you don’t stop playing.

Just because some firms (perhaps including OPM and USIS) stand in the wall waiting for a penalty with hands dismissively and arrogantly behind their heads, or looking in completely the wrong direction, doesn’t mean you or team mates should too.

My turn to bang a drum for the umpteenth time.

  1. You can’t stop state sponsored or other brands of expertly targeted attacks, but you can up your chances of making success harder, easier to spot and easier to mitigate. If you’re not a desirable target for the big bad guys, work out who is keen on having a go and what their MO typically is.
  1. Find your data crown jewels (it really is, as I said here, mostly and mainly about the data), wrap them in good tech, process and people security and restrict/monitor ways to get to them.
  1. If you’ve spent years neglecting the security foundations (97% of the previously mentioned vulnerability related incidents in the Verizon DBIR were reportedly down to just 10 vulnerabilities, some of which you were notified about over a decade ago), find out where things are shakiest and how directly shakey stuff impacts security of high value (mainly data) assets. Use that information to prioritise which bits you shore up first.
  1. Yep, outsourcing (to cloud vendors or others), can be a cost effective way to buy some mature security, but only if you know what you need, what you get for your money (functionality, run cost and on-going dev/support-wise) and you keep an eye on how that fits your risk profile now and over time.
  1. For suppliers providing other stuff, know each other. Build in formal expectations, on-going checks and free flowing prompt notification both ways about incidents and changes to mutual risk profiles.
  1. In parallel, bring in the folk who understand the threat landscape. A bit of that is machines that go ‘ping’, but more is about long and deep experience of what motivates the bad guys, how they operate, how your specific business fits into that picture, what the pingy machine output says about real, business relevant risks. Then beyond the threat data there’s subtler tech detection resistant signs of impending or in-progress attacks.
  1. Why not also get your people involved? A few hundred/thousand extra eyes and spidey senses better tuned to spot anomalies in system, site and user behaviour.
  1. Oh and don’t swallow standards whole. Prioritise required control based on your real risk. Dare to propose risk acceptance if standards demand gold-plating where you know local risk is negligible and spend could derail work that can make a big difference.

So it turns out I had quite a bit to say. What do you think? Am I making sense? Is it a worthwhile addition to the reams of OPM related reporting? In many ways I don’t really care, because it’s a long-term investment. I’ll probably be able to recycle some or all of this post for years if we don’t start being honest about all the basic stuff we haven’t fixed, our ignorance about threats and the need to change how we communicate and treat security in the context of business/government culture and objectives

1 reply »

Want to add to the discussion?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s