Archive for 2018

“The Big Hack” – what does it really mean?

Thursday, October 4th, 2018

Bloomberg published an interesting article today, about a purported spying program by which the Chinese state injected trojan components into motherboards manufactured by SuperMicro. The claim is that Amazon, Apple and others had been compromised (or at least compromised hardware had been found in their data centers).


Did this really happen?

The Bloomberg report is credible for a few reasons:

  1. They are a serious news organization.
  2. They claim to have corroborated the story through 17 separate sources working for a variety of organizations, including the target companies and security agencies.
  3. The attack vector is technically feasible, though very challenging to pull off.
  4. The Chinese state, like the American state, has both the resources and motivation to do something like this. Neither China nor the US has the good sense to avoid doing something like this, due to the commercial damage that discovery would cause.

Amazon, Apple and Supermicro all strongly deny the report. That does not mean it’s not true, but it does raise the possibility that some or all of the details are incorrect.

Supermicro’s stock is down 41% today. Ouch. They aren’t even alleged to have done anything wrong – they are just the channel through which malicious or compromised subcontractors injected bad hardware destined for various US corporations. Apple and Amazon stocks are both down about 2% today as well.

Is this attack something new in the world?

In one sense: yes. Injection of malicious hardware components into motherboards is seriously difficult to do and I haven’t heard of anyone pulling this kind of thing off before.

In other senses: no. Injection of malicious components or software into otherwise innocent hardware has been done before, including by the US government:


Bad firmware in hardware products is nothing new. Bad security measures in CPU design are also a serious hardware problem that has been in the news lately:


As a security community, we should all know by now that neither hardware platforms nor supply chains can be blindly trusted.

Is this good for China?

Not at all. Disclosure of this attack means that manufacturers will scramble to move their fabrication to other, less aggressive jurisdictions.

Is this good for the US?

Not really. They have been caught doing much the same thing to in-transit network devices in the past, which has already caused many buyers around the world to avoid US products and services, causing billions of dollars of economic harm. Moreover, the US legal system provides little or no privacy protection to foreign citizens or firms, which likewise causes many foreign governments and corporations to procure products and services elsewhere.

This is one area where government action has the ability to either be benign or harmful, and both the US and Chinese security establishments have put their own, short-sighted, intelligence-gathering priorities ahead of the broader economic interests of their nations, via the revenue streams of corporations in their own jurisdictions. These are smart-yet-stupid governments.

Are we affected?

These malicious chips could be in any motherboard of any computer. Of course, attention will first be on motherboards manufactured by (or really on behalf of) Supermicro, and that’s a lot of motherboards. Hopefully, over the next few weeks, someone will figure out how to detect that a given motherboard is affected and publish that. If and when that happens, we should all crack open our PCs and servers and see if they are affected.

Then what? In most cases, I expect we’ll find nothing and carry on with business as usual. In some cases, we might find compromised hardware, in which case we’ll need to figure out how to mitigate the hardware or quickly replace the tainted machines.

Watch your news feed for an indication of how to identify compromised systems.

Impact on OEMs and future supply chains

Hardware OEMs like Dell and HP and SaaS operators like Amazon and Microsoft need to figure out how to vet incoming hardware. That’s an extremely advanced sort of analysis — I’m not aware of any way to automate it and only a very few people in the world have the required skills. Globally, anyone who procures a large amount of hardware needs to get into the business of pulling a statistical sample of devices out of their supply chain and subjecting them to deep analysis to see if a bad actor somewhere upstream in their supply chain has done something naughty.

This is an expensive process. Are we, as organizations and consumers, willing to pay for that?

Also, where does this leave the Chinese-domiciled OEMs like Lenovo? Do we trust them to do this kind of supply chain validation? What if they are also compromised by their own state? What about Taiwanese OEMs like Asus and Acer? How immune are they from influence, given the cultural and linguistic proximity to China and the fact that almost all of their manufacturing is on the mainland?

This little news story could send big ripples through global supply chains, presumably to the detriment of the Chinese economy.

IAM versus PAM: understanding the differences

Tuesday, September 25th, 2018

I frequently talk to people tasked with “securing privileged accounts” who struggle to understand the roles of different product categories — separating identity and access management (IAM) from privileged access management (PAM).

In the interests of clarity, I thought I’d try to clear the air about what these distinct product categories have in common and where they diverge.

First, lets consider what problems we are trying to solve:

  • There is at least one login ID with elevated security privileges (entitlements) on every system and application that one can sign into. This includes hardware — servers, iLO or DRAC cards, desktops, laptops, network devices, etc.; operating systems and hypervisors; applications and databases; on-premises and cloud services and more. If it’s a physical or logical computing system, there is at least one administrative account on it and in almost all cases, this account only supports password-based logins.
  • These accounts are normally shared by multiple people and sometimes also accessed by software processes (scripts and applications). This makes them shared accounts, which leads to accountability problems — who used this account? What did they do? Was their activity appropriate?
  • The passwords to these accounts are static by default and because the accounts are shared, coordinating password changes is difficult. Static passwords represent an elevated security risk, because the window of opportunity for a bad actor to guess or steal the passwords is large.
  • The passwords to these accounts are often stored in plaintext — in scripts, applications, registry entries, configuration files, spreadsheets or printed on paper. Plaintext passwords are a clear security problem.
  • As these accounts represent an elevated security risk, they should be subject to closer scrutiny: are they still required? Do they really need all the entitlements they are assigned? Who owns them?

    High risk accounts that aren’t really needed should be disabled or deleted, to reduce unnecessary risk. Likewise, entitlements that are not actually used should be revoked. Accounts without clear owners are especially vulnerable to abuse, because (a) nobody can make a determination about their continued need or appropriate entitlements and (b) it’s likely that nobody would notice successful compromise of the account in question.

Alright – so what do we do about it? This is where two product categories come into play:

  • Identity and access management (IAM) systems are used to manage the lifecycle of accounts and their entitlements. Specifically, we can use an IAM system to:
    • Create privileged accounts when needed (often this is not necessary as most systems and applications come with these accounts already present).
    • Disable or delete no-longer-needed accounts.
    • Periodically review these accounts and update information such as who owns them or what access rights they have.
  • Privileged access management (PAM) systems are used to control access to the accounts that already exist:
    • Periodically randomize and vault passwords.
    • Identify, authenticate and authorize people before brokering their access to shared accounts. In other words: directory integration to authenticate users, MFA to authenticate users and single sign-on to launch logins.
    • Record and enable search/playback of login sessions, to create strong, forensic-level accountability.
    • Enable scripts and applications to fetch the passwords they use to connect to services in a secure manner, rather than from plaintext local storage.
    • Periodically change the passwords for service accounts, used to launch scheduled jobs, services and other unattended processes.

In other words, IAM systems manage the lifecycle of accounts while PAM systems act as a gate to gain access to the accounts at runtime.

Incidentally, this is why the industry seems to have finally settled on the terminology “privileged access management” and deprecated older terms such as “privileged identity management,” “privileged user management,” or “privileged account management.” These older terms might lead you to think that a PAM system creates and manages accounts, rather than brokers access to existing accounts, which is wrong in most cases.

In some organizations, there are additional differences, because IAM systems have not (yet?) been integrated with privileged accounts. When IAM systems are not used to manage privileged accounts:

  • The IAM system will typically have a few (or even a few hundred) integrations, as compared to thousands of integrations for the PAM system.
  • It is reasonable to onboard managed systems and applications into the IAM application manually, one at a time. This does not scale for PAM systems, where automation is required to onboard thousands of endpoints at once.

IAM systems also have more normal “uptime” requirements than PAM systems. If an IAM system is shut down for a few hours for maintenance, the few people who needed to request access, approve requests, perform reviews or change their own password during that time window can wait a bit or call the help desk. The impact is not catastrophic. On the other hand, if a PAM system goes down for a few hours, the entire IT operation grinds to a halt as nobody can access the accounts they need to do their jobs.

To summarize:

  • IAM systems:
    • Are used to create, update and delete accounts.
    • Grant and revoke entitlements and track account ownership.
    • May only have a modest number of integrations.
    • Can tolerate modest outages without severe business impact.
    • Are always integrated with major systems and applications that thousands of users sign into.
    • May not be integrated with the thousands of bits of hardware and software that only IT staff sign into.
  • PAM systems:
    • Broker access between strongly authenticated and authorized users and already-present high-privilege accounts.
    • Address the problems of static, plaintext passwords.
    • Are used to introduce strong accountability.
    • Require extreme scale, including in how endpoints are onboarded and removed.
    • Cannot tolerate outages.
    • Typically have thousands of integrations.

Organizations don’t get to choose one or the other — IAM or PAM. All organizations need solutions from both categories.

SaaS: Be careful what you sign up for

Wednesday, September 12th, 2018

There is no question that the world is moving from on-premises software to cloud-hosted, vendor-managed alternatives. Software-as-a-Service (SaaS) means that applications are professionally deployed and managed by specialists and hosted in someone else’s data center. Success stories including, Microsoft Office 365 and ServiceNow are just examples of this macro trend.

When organizations move applications from on-premises to SaaS, there is a risk of losing significant capabilities. This happens because legacy software has evolved over decades to handle complex edge cases, while SaaS alternatives are relatively new and often missing key features.

This is not always the case, but seems to be quite common in the IAM segment. While solutions such as Salesforce and Office 365 are “feature complete” in the sense that they offer materially the same capabilities as on-premises alternatives, services that are branded as “Identity as a Service” (IDaaS) from Okta, Azure AD and others are much thinner, offering some combination of directory services, single sign-on (mainly via federation), basic (de)provisioning to SaaS applications and in some cases 2FA or simple account creation and deactivation.

In other words, IDaaS is often a misnomer, referring instead to single sign-on as a service (SSOaaS) plus some very limited IAM features. There is nothing wrong with these services — but calling them IDaaS is not quite truth in advertising as customers may get less than they expect.

There is no technical barrier preventing vendors from offering more feature rich IAM systems as a service. Indeed, IAMaaS is probably a better acronym than IDaaS for solutions that include connectors to manage both on-premises and SaaS applications, both automated and request/approval based access provisioning and governance features such as access reviews, approval workflows, analytics, segregation of duties policy and role based access control.

Hitachi ID, and doubtless some competitors as well, offers exactly this: full-featured IAM hosted in the cloud and managed by the software vendor.

One of the expectations that organizations bring to SaaS is shorter, less costly implementations. In practice, this has nothing to do with where the application is hosted (on-premises versus cloud) and everything to do with standardized configurations and frequent, automated version upgrades. In this regard, Hitachi ID may be unique, as we offer “Identity Express” — a set of standardized business processes for workforce IAM and for partner portal IAM, which helps both to speed up IAM implementations and provide more feature-rich joiner/mover/leaver process automation than would be possible with a custom approach to system deployment.

For those of you with a mandate to “move to the cloud,” the guidance is two-fold: (a) understand the difference between IDaaS (really SSOaaS) and IAMaaS (hosted, leased, managed IAM) and (b) focus on adopting best practices business processes, not just moving the IAM system from your data center to someone else’s.

A practical use for blockchain: disarming censors

Tuesday, July 24th, 2018

An interesting news story out of China:

The predictable, sad but not really noteworthy bit: a vaccine manufacturer saved money by making bogus vaccines and its executives were hauled away by police.

The predictable government response: delay publication of the news, as it implies a corrupt regulator and suppress subsequent social media posts on the subject.

The interesting and innovative bit: repost the story as metadata in an Ethereum transaction, such that it gets replicated across all participants in that blockchain and becomes impossible to tamper with or delete.

This is fascinating! An interesting way to totally defeat, once and for all, state authorities bent on censorship. Just encode public messages in blockchain transaction metadata. Brilliant, though dangerous (what if the state figures out who issued/signed the transaction? jail time at a minimum).

Most of the interest in blockchain is conjecture about hypothetical use cases, but this use case is legitimate and practical – a rare thing.

Privileged Access in an IoT world

Tuesday, June 19th, 2018

While it may seem like securing IoT is an abstract problem, in reality it is anything but.

Apparently Tesla (and possibly also SpaceX, separately, a couple of years ago) has been hit by internal sabotage to critical code and possibly other processes.

SC Magazine.

If you thought PAM was purely about administrator access to IT systems, you were wrong. How about PAM access to code check-ins? PAM access to factory control systems? PAM access to sensors (cameras?) and actuators?

This is reality, not theory.

Did your company spend millions on “access governance” and fail to automate anything?

Friday, June 8th, 2018

Imagine that you’re a mid-sized organization (say 2,500 employees in the case of a particular, real-world organization I have in mind) and a few years ago your company took the advice of a leading systems integrator / consulting firm and purchased licenses for the market leading IAM product.

You have since invested over $4 million in software, runtime platform, maintenance fees and lots (!) of implementation services. You have purchased as much consulting time as seemed appropriate, and you still have projects on the go.

By this time, after this much time and money spent, you would expect to have a lot of integrations and process automation in place, wouldn’t you? Several years and 4 million dollars should buy a lot, you’d think.

Unfortunately, if you were convinced to buy SailPoint, you probably haven’t got much to show for it. You have an excellent platform for running access certification campaigns, and you have built a system that integrates directly with your AD domain, plus consumes all sorts of CSV files from many applications, but that’s about it.

You haven’t automated any joiner/mover/leaver processes.

You don’t have a request portal rolled out to your users.

The SoD enforcement is sketchy at best — there are definitely ways to submit requests that should but do not trigger SoD violations.

Integrations are mainly via “flat files” — not real connectors.

But hey – there are pretty dashboards! Those must add value, right?

That’s not a great way to spend $4 million, but the integrator keeps proposing more — though every deployment phase seems to amount to just more CSV file “integrations.”

Maybe it’s time to get off the treadmill and look at better solutions, that can actually automate IAM processes, rather than delivering “access governance” but only promising (and never delivering) IAM process automation?

I can name a few organizations in this boat. Are you stuck in this situation too?

Please ‘like’ or ‘share’ if this scenario sounds all too familiar.

PC, Smart phone shipments declining

Thursday, March 1st, 2018

I noticed two apparently-unrelated news articles today:


One talks about declining unit sales of phones and the other talks about the same thing for PCs.

Reading these articles, you’d think the sky is falling. Less phones are selling, on an annual basis! Less PCs too! Oh no! These industries are in trouble!

I think the reality is actually much simpler: PCs have been quite good for years now. Smart phones have likewise reached a plateau in terms of functionality and performance. Sure, new PCs and phones are nicer than old ones, but the extra screen pixels, camera resolution and compute power are not solving any new problems – they just meet the same old needs slightly better than before.

More than that, both PCs and phones are increasingly used to access cloud platforms. The heavy computation, storage, etc. are done on someone else’s server, not on personal devices any more. This makes local compute and storage even less meaningful.

So when a hardware manufacturer asks for $1000 for a shiny new phone or $1500 for a shiny new PC, most consumers compare the extra value of the new toy to the functionality of their existing PC or phone and choose to just leave the old device alone. It’s not broken. It’s working fine. Why upgrade?

I think this is the new normal. If you write software, or provide on-line services, then you shouldn’t care about this. People are on-line pervasively. They use browsers to access everything. There is no problem here.

If you are a phone or PC manufacturer, you just have to reset your expectations. The hardware replacement cycle is lengthening, not because consumers don’t want your product any more, but because your last generation was very good and the current generation is not so much better as to justify a new purchase. You’ll still sell lots of units, but the age of rapid growth is over. Heck, the only way you’ll grow your shipments is to win more market share from competitors. Get used to flat sales and razor thin margins folks. That’s the steady state of selling hardware.

Will this situation change? I doubt it. I have a hard time imagining what new capabilities phone or PC makers will be able to invent tomorrow, to excite consumers into a big hardware refresh cycle.

So a tough situation for hardware makers and a good situation for everyone else. That’s life.

Groups versus roles

Monday, February 12th, 2018

Periodically, I’m asked what the difference is between groups and roles. This is coming up even more frequently now that we’ve released version 11.0 of the Hitachi ID Identity and Access Management Suite, which includes full lifecycle management of groups (i.e., create, delete, update group objects on systems like AD).

This is actually a harder question to answer than one might think, because different systems and applications use the terms interchangeably. The object that I think of as a group might be called a group on one system (e.g., AD) and a role on another (e.g., SAP).

Ambiguous and overloaded terminology is very common in the IAM business, unfortunately.

Still, there’s a difference, once you define the two terms. To understand it, we should start with definitions. I’ll use Hitachi ID’s standard nomenclature here:

  • A group is an object that exists on a target system (LDAP, AD, SAP, RAC/F, etc.). Accounts are attached to the group as members. The group is assigned security privileges. This way, it’s not necessary to assign individual access rights to each user — which would be painful for a variety of reasons.Note that this is an approximate definition. For example, on some systems, groups can contain other groups as members (AD, RAC/F come to mind). On other systems, groups are also used as mail distribution lists (AD, LDAP). In some cases, a group on one system can contain among its members accounts or groups from another system (AD with cross-domain groups, for example).
  • A role is an object that exists on the IAM system. It’s used to collect entitlements (mainly groups, but also accounts) on target systems for more convenient assignment. Roles are typically nested — i.e., roles can contain other roles.Roles may be “technical” which just means that they represent collections of entitlements that are normally assigned together, but might not make much sense to business users, or they might be “organizational” which means that they represent business functions that requesters, authorizers or reviewers are more likely to understand. The difference here is a bit fuzzy, and that’s OK.

Beyond groups being on the target system and roles being in the IAM system, how are roles and groups different?

It turns out that how these objects are used in practice is quite different!

While IAM software vendors and implementation consultants strive to make role management a business-friendly task that managers will participate in, I’ve found that this rarely comes to pass. Managers rarely have the motivation, skills or time to participate meaningfully in defining or updating role definitions.

This has practical consequences: an IT team is ultimately responsible for defining and maintaining roles. They have limited visibility into business requirements and limited time to spend on any given role. These constraints make role definitions fairly static and formal.

Static, formal IAM roles usually wind up representing business functions (think job code). With static, formal roles it is natural to automatically assign roles: For example, birthright access for employees and contractors, departmental level roles for common access rights, job-code driven roles for jobs that are shared by large numbers of workers, etc. As HR systems or requests update identity attributes, the IAM system recalculates eligibility criteria and automatically assigns or revokes roles.

The consequence of all this is that roles are used to assign highly predictable, very regular access rights to users. If your location, department code, job code or cost centre can be used to predict your access rights and if your access rights are the same as many other people, then roles are the right tool to assign those rights to you, without extra manual intervention.

In practice, some access rights are simply hard to predict based on available data. For example, projects start up and wind down all the time. There is usually nothing in the HR database to indicate that a given user is involved in a project, so automation driven by HR data cannot grant or revoke project-related access rights.

This is part of a general pattern: automatic assignment of rights, via roles or not, only works well if we have both a sufficiently detailed, timely and reliable data source and a matching model that maps from identity attributes to entitlements.

Where this pattern ends is where directly assigned privileges begin. For directly assigned privileges, users could create new roles and request role assignment, but in practice users will skip the role definition step and just request the rights they want. i.e., users will request membership in the security groups that grant access to the share, folder or other item they wish to access, or ask to join a given mail distribution list.

There are two patterns for group membership management: groups may be included in roles, in which case they tend to be automatically assigned to appropriate users, or groups are not part of a role, in which case they tend to be requested on an as-needed basis.

Put another way: roles are formal and static, while groups may also be ad-hoc and dynamic.

What does ad-hoc mean?

  • Ad-hoc groups won’t be automatically assigned, so users must be able to sign into a portal and find and request them.
  • Users often don’t know what group to ask for (or even where to find the request portal), so significant usability support is required.
  • Ad-hoc may also mean creating and deleting the groups themselves, rather than just just requesting membership.
  • Anyone could request anything, so an approval workflow is required to prevent inappropriate requests from being completed.
  • Nobody ever asks for access rights to be reduced, so a periodic access review and revocation process is essential.

And that’s it:

  • Roles are IAM constructs used to build a formal model of predictable, static access rights that are usually automatically assigned,
  • Groups are objects on existing systems and applications. They may either be parts of roles, or may be requested, approved, reviewed and revoked in an ad-hoc manner.

Bitcoin Hype

Tuesday, January 16th, 2018

Blockchain has become quite the hyped technology over the past year.

Consider this: it’s a distributed, validated ledger of transactions. It’s useful to transfer value between parties who may not know each-other, who may wish to remain anonymous and who wish to ensure that transfer is singular — no “double payment.”

That’s the high level picture.

On the other hand, it’s got some draw-backs:

  • It’s computationally expensive, by design.
  • It’s a profligate user of electrical energy.
    Bitcoin alone is estimated to consume over 41 terawatt-hours per year:
  • It’s slow.
    The entire Bitcoin network only processes about 300,000 transactions/day
    or about 3.5 transactions/second).

People have lots of wild theories about where this technology will bear fruit, but in practice the only major use case today is for criminals to pay for goods and services. Awesome.

The other “use case” today is a speculative market in exchanging Bitcoin for hard currency, like USD. Here’s the chart:

I recently read a great write-up comparing the hype of blockchain to the reality:

So what’s the reality?

  • Is it a safe place to store value? Only if you are sure that your private key and/or password will never be compromised and that you’ll never make a mistake when transferring funds. If you don’t trust yourself enough for that, with large value wealth storage, then it’s a bad platform for you.
  • Are bitcoin exchanges a safe place to store money? Given their history of hacks and shady business, I’d say no.
  • Is it good for funds transfer? Not really, given the cost and delay of transactions. Only criminals who really, really need anonymity would put up with the awful process.
  • Is it good for small payments? Not really. Too slow and costly.
  • Is it a good alternative for hyper-inflationary currencies issued by idiot regimes? US Dollars or Euros are a more practical choice there.
  • Does it have stable value? If you read the news, you know that Bitcoin/USD exchange rates have behaved more like a bubble than an investment-grade product.
  • Would it be a good alternative to inter-bank transfers? Maybe, but not anytime soon.
  • What about smart contracts? Sounds good, but people engaged in legal contracts generally want to understand what they are signing up for, and few can read algorithms. Moreover, how do you tie a smart contract to real-world trigger events?
  • What about using it to manage identities? The only plausible use case here is for citizens or consumers (forget about identities within the enterprise) and in any case it’s not clear how to link such a blockchain to physical things like birth records or driver licenses.

In other words, it’s a cool way for criminals to transfer funds anonymously and a plausible way for institutions to handle large but infrequent transfers among themselves. For everything else, the cost, inconvenience, non-repudiation, delay and even anonymity look like problems rather than advantages.

But everyone is making noises about “doing blockchain,” like this soft drink company:

Long Island Iced Tea?

So why is everyone jumping on the blockchain bandwagon? Because it’s the latest fad, and if your company tells the world that it’s researching the latest blockchain technology, your stock price will probably get a bounce. But don’t hold your breath for results.