Trade wars, mobile phones and supply chains

May 27th, 2019

The US and China are embroiled in a huge trade war and it’s likely that most people don’t really understand the gravity of the situation.

US complaints about China are legitimate: it’s a closed market, with state-funded firms who export abroad but where foreign firms cannot really import goods and instead are required to setup joint ventures with local companies, in a sort of forced technology transfer. This is on top of a clear pattern of IP theft by the Chinese state to benefit local companies in their competition with off-shore firms.

None of this is to suggest that the Trump administration’s approach to resolving unfair Chinese trade practices will work. On the contrary, a key element in most Asian cultures is the idea of “face” and the Chinese government has to project a narrative of “winning” to its domestic audience in any trade agreement. Trump’s approach is combative and public, which makes compromise essentially impossible.

It seems that a part of the US strategy in its trade war with China is to attack Huawei — China’s leading mobile phone and mobile infrastructure manufacturer. By recently classifying Huawei as a threat to its national security, the US has brought to bear the full weight of its export control regime, which extends well beyond the US to all its trading partners, to block Huawei from trading with most Western firms. It seems unlikely that Huawei can survive this.

The Americans assert that Huawei is a threat to national security and may in future engage in spying. This is technically true, but no more so than any firm subject to the coercive power of its own government. I don’t think anyone has offered or even implied the existence of evidence that Huawei has engaged in any material acts of espionage against foreign states. On the contrary, the sort of espionage implied, using back doors in telecom equipment, is precisely what the US has been caught doing — we know this courtesy Snowden’s leaks.

In short, US worries about Huawei boil down to “these guys are dangerous because they might in future do to us the things that we’ve been doing to our adversaries for years.”

It seems likely, therefore, that the US attack on Huawei is less about security and more about attacking China’s leadership. There is ample circumstantial evidence that Huawei and powerful figures in China are closely connected. For example, consider the case of Meng Wanzhou — Huawei’s CFO currently under house arrest in Canada due to an extradition request from the Americans. China’s reaction to what would otherwise be a minor commercial squabble has been ferocious — two Canadians are arbitrarily detained in China, multiple products normally exported from Canada to China are blocked for arbitrary and unsupported reasons and even a Canadian citizen already in Chinese custody for drug offenses is retried and sentenced to death. This in addition to frequent complaints about Canadian policy by Chinese officials.

This is not a normal response by a state to what is fundamentally litigation against one of its citizens for white collar crime (Meng Wanzhou is accused essentially of an export control violation).

That China responded so strongly to this case, and that the US has tried to convince its allies to exclude Huawei from 5G infrastructure contracts and now has essentially cut off Huawei’s access to Western technology suggests that the US is trying to hurt the Chinese leadership personally, perhaps through their ownership interests in Huawei. If you poke a bee hive the bees are likely to come out and sting you, and that’s exactly what’s been happening.

The trouble with successfully hurting the Chinese leadership is that China will inevitably respond. The West needs China just as much as China needs the West. Sure, Huawei will go out of business without access to Google Services, patches to Android, access to ARM CPU designs and more, but the West needs things just as badly from China — consider rare earth metals, which are mainly sourced from China and required to make all sorts of electronics. Or large-scale assembly of computers and phones (remember where all those iPhones are made?). And China is already retaliating against agricultural products, switching to other suppliers abroad.

Trade wars hurt everyone. Hopefully the amateurs in Washington and the kleptocrats in Beijing can work something out soon!

Is GDPR infectious?

May 24th, 2019

With the General Data Protection Regulation now in effect in Europe, we’re starting to see other jurisdictions make noises about adopting similar rules.

Most recently I noticed one from the Canadian government and another from the Microsoft CEO in regards to US regulations:

While privacy protection sounds great – there are often unintended consequences with regulations and this is no exception. There is definitely a chilling effect on trade here and an uptick in investment in likely redundant internal controls.

If other jurisdictions are smart, they’ll take a wait and see approach to this – to gauge the impact on the EU and on firms trading in the EU before adopting their own rules. They might also look at homogenized regulations, to reduce the total regulatory burden on corporations and other organizations. Then again, governments are not usually too smart. 🙂

“The Big Hack” – what does it really mean?

October 4th, 2018

Bloomberg published an interesting article today, about a purported spying program by which the Chinese state injected trojan components into motherboards manufactured by SuperMicro. The claim is that Amazon, Apple and others had been compromised (or at least compromised hardware had been found in their data centers).


Did this really happen?

The Bloomberg report is credible for a few reasons:

  1. They are a serious news organization.
  2. They claim to have corroborated the story through 17 separate sources working for a variety of organizations, including the target companies and security agencies.
  3. The attack vector is technically feasible, though very challenging to pull off.
  4. The Chinese state, like the American state, has both the resources and motivation to do something like this. Neither China nor the US has the good sense to avoid doing something like this, due to the commercial damage that discovery would cause.

Amazon, Apple and Supermicro all strongly deny the report. That does not mean it’s not true, but it does raise the possibility that some or all of the details are incorrect.

Supermicro’s stock is down 41% today. Ouch. They aren’t even alleged to have done anything wrong – they are just the channel through which malicious or compromised subcontractors injected bad hardware destined for various US corporations. Apple and Amazon stocks are both down about 2% today as well.

Is this attack something new in the world?

In one sense: yes. Injection of malicious hardware components into motherboards is seriously difficult to do and I haven’t heard of anyone pulling this kind of thing off before.

In other senses: no. Injection of malicious components or software into otherwise innocent hardware has been done before, including by the US government:


Bad firmware in hardware products is nothing new. Bad security measures in CPU design are also a serious hardware problem that has been in the news lately:


As a security community, we should all know by now that neither hardware platforms nor supply chains can be blindly trusted.

Is this good for China?

Not at all. Disclosure of this attack means that manufacturers will scramble to move their fabrication to other, less aggressive jurisdictions.

Is this good for the US?

Not really. They have been caught doing much the same thing to in-transit network devices in the past, which has already caused many buyers around the world to avoid US products and services, causing billions of dollars of economic harm. Moreover, the US legal system provides little or no privacy protection to foreign citizens or firms, which likewise causes many foreign governments and corporations to procure products and services elsewhere.

This is one area where government action has the ability to either be benign or harmful, and both the US and Chinese security establishments have put their own, short-sighted, intelligence-gathering priorities ahead of the broader economic interests of their nations, via the revenue streams of corporations in their own jurisdictions. These are smart-yet-stupid governments.

Are we affected?

These malicious chips could be in any motherboard of any computer. Of course, attention will first be on motherboards manufactured by (or really on behalf of) Supermicro, and that’s a lot of motherboards. Hopefully, over the next few weeks, someone will figure out how to detect that a given motherboard is affected and publish that. If and when that happens, we should all crack open our PCs and servers and see if they are affected.

Then what? In most cases, I expect we’ll find nothing and carry on with business as usual. In some cases, we might find compromised hardware, in which case we’ll need to figure out how to mitigate the hardware or quickly replace the tainted machines.

Watch your news feed for an indication of how to identify compromised systems.

Impact on OEMs and future supply chains

Hardware OEMs like Dell and HP and SaaS operators like Amazon and Microsoft need to figure out how to vet incoming hardware. That’s an extremely advanced sort of analysis — I’m not aware of any way to automate it and only a very few people in the world have the required skills. Globally, anyone who procures a large amount of hardware needs to get into the business of pulling a statistical sample of devices out of their supply chain and subjecting them to deep analysis to see if a bad actor somewhere upstream in their supply chain has done something naughty.

This is an expensive process. Are we, as organizations and consumers, willing to pay for that?

Also, where does this leave the Chinese-domiciled OEMs like Lenovo? Do we trust them to do this kind of supply chain validation? What if they are also compromised by their own state? What about Taiwanese OEMs like Asus and Acer? How immune are they from influence, given the cultural and linguistic proximity to China and the fact that almost all of their manufacturing is on the mainland?

This little news story could send big ripples through global supply chains, presumably to the detriment of the Chinese economy.

IAM versus PAM: understanding the differences

September 25th, 2018

I frequently talk to people tasked with “securing privileged accounts” who struggle to understand the roles of different product categories — separating identity and access management (IAM) from privileged access management (PAM).

In the interests of clarity, I thought I’d try to clear the air about what these distinct product categories have in common and where they diverge.

First, lets consider what problems we are trying to solve:

  • There is at least one login ID with elevated security privileges (entitlements) on every system and application that one can sign into. This includes hardware — servers, iLO or DRAC cards, desktops, laptops, network devices, etc.; operating systems and hypervisors; applications and databases; on-premises and cloud services and more. If it’s a physical or logical computing system, there is at least one administrative account on it and in almost all cases, this account only supports password-based logins.
  • These accounts are normally shared by multiple people and sometimes also accessed by software processes (scripts and applications). This makes them shared accounts, which leads to accountability problems — who used this account? What did they do? Was their activity appropriate?
  • The passwords to these accounts are static by default and because the accounts are shared, coordinating password changes is difficult. Static passwords represent an elevated security risk, because the window of opportunity for a bad actor to guess or steal the passwords is large.
  • The passwords to these accounts are often stored in plaintext — in scripts, applications, registry entries, configuration files, spreadsheets or printed on paper. Plaintext passwords are a clear security problem.
  • As these accounts represent an elevated security risk, they should be subject to closer scrutiny: are they still required? Do they really need all the entitlements they are assigned? Who owns them?

    High risk accounts that aren’t really needed should be disabled or deleted, to reduce unnecessary risk. Likewise, entitlements that are not actually used should be revoked. Accounts without clear owners are especially vulnerable to abuse, because (a) nobody can make a determination about their continued need or appropriate entitlements and (b) it’s likely that nobody would notice successful compromise of the account in question.

Alright – so what do we do about it? This is where two product categories come into play:

  • Identity and access management (IAM) systems are used to manage the lifecycle of accounts and their entitlements. Specifically, we can use an IAM system to:
    • Create privileged accounts when needed (often this is not necessary as most systems and applications come with these accounts already present).
    • Disable or delete no-longer-needed accounts.
    • Periodically review these accounts and update information such as who owns them or what access rights they have.
  • Privileged access management (PAM) systems are used to control access to the accounts that already exist:
    • Periodically randomize and vault passwords.
    • Identify, authenticate and authorize people before brokering their access to shared accounts. In other words: directory integration to authenticate users, MFA to authenticate users and single sign-on to launch logins.
    • Record and enable search/playback of login sessions, to create strong, forensic-level accountability.
    • Enable scripts and applications to fetch the passwords they use to connect to services in a secure manner, rather than from plaintext local storage.
    • Periodically change the passwords for service accounts, used to launch scheduled jobs, services and other unattended processes.

In other words, IAM systems manage the lifecycle of accounts while PAM systems act as a gate to gain access to the accounts at runtime.

Incidentally, this is why the industry seems to have finally settled on the terminology “privileged access management” and deprecated older terms such as “privileged identity management,” “privileged user management,” or “privileged account management.” These older terms might lead you to think that a PAM system creates and manages accounts, rather than brokers access to existing accounts, which is wrong in most cases.

In some organizations, there are additional differences, because IAM systems have not (yet?) been integrated with privileged accounts. When IAM systems are not used to manage privileged accounts:

  • The IAM system will typically have a few (or even a few hundred) integrations, as compared to thousands of integrations for the PAM system.
  • It is reasonable to onboard managed systems and applications into the IAM application manually, one at a time. This does not scale for PAM systems, where automation is required to onboard thousands of endpoints at once.

IAM systems also have more normal “uptime” requirements than PAM systems. If an IAM system is shut down for a few hours for maintenance, the few people who needed to request access, approve requests, perform reviews or change their own password during that time window can wait a bit or call the help desk. The impact is not catastrophic. On the other hand, if a PAM system goes down for a few hours, the entire IT operation grinds to a halt as nobody can access the accounts they need to do their jobs.

To summarize:

  • IAM systems:
    • Are used to create, update and delete accounts.
    • Grant and revoke entitlements and track account ownership.
    • May only have a modest number of integrations.
    • Can tolerate modest outages without severe business impact.
    • Are always integrated with major systems and applications that thousands of users sign into.
    • May not be integrated with the thousands of bits of hardware and software that only IT staff sign into.
  • PAM systems:
    • Broker access between strongly authenticated and authorized users and already-present high-privilege accounts.
    • Address the problems of static, plaintext passwords.
    • Are used to introduce strong accountability.
    • Require extreme scale, including in how endpoints are onboarded and removed.
    • Cannot tolerate outages.
    • Typically have thousands of integrations.

Organizations don’t get to choose one or the other — IAM or PAM. All organizations need solutions from both categories.

SaaS: Be careful what you sign up for

September 12th, 2018

There is no question that the world is moving from on-premises software to cloud-hosted, vendor-managed alternatives. Software-as-a-Service (SaaS) means that applications are professionally deployed and managed by specialists and hosted in someone else’s data center. Success stories including, Microsoft Office 365 and ServiceNow are just examples of this macro trend.

When organizations move applications from on-premises to SaaS, there is a risk of losing significant capabilities. This happens because legacy software has evolved over decades to handle complex edge cases, while SaaS alternatives are relatively new and often missing key features.

This is not always the case, but seems to be quite common in the IAM segment. While solutions such as Salesforce and Office 365 are “feature complete” in the sense that they offer materially the same capabilities as on-premises alternatives, services that are branded as “Identity as a Service” (IDaaS) from Okta, Azure AD and others are much thinner, offering some combination of directory services, single sign-on (mainly via federation), basic (de)provisioning to SaaS applications and in some cases 2FA or simple account creation and deactivation.

In other words, IDaaS is often a misnomer, referring instead to single sign-on as a service (SSOaaS) plus some very limited IAM features. There is nothing wrong with these services — but calling them IDaaS is not quite truth in advertising as customers may get less than they expect.

There is no technical barrier preventing vendors from offering more feature rich IAM systems as a service. Indeed, IAMaaS is probably a better acronym than IDaaS for solutions that include connectors to manage both on-premises and SaaS applications, both automated and request/approval based access provisioning and governance features such as access reviews, approval workflows, analytics, segregation of duties policy and role based access control.

Hitachi ID, and doubtless some competitors as well, offers exactly this: full-featured IAM hosted in the cloud and managed by the software vendor.

One of the expectations that organizations bring to SaaS is shorter, less costly implementations. In practice, this has nothing to do with where the application is hosted (on-premises versus cloud) and everything to do with standardized configurations and frequent, automated version upgrades. In this regard, Hitachi ID may be unique, as we offer “Identity Express” — a set of standardized business processes for workforce IAM and for partner portal IAM, which helps both to speed up IAM implementations and provide more feature-rich joiner/mover/leaver process automation than would be possible with a custom approach to system deployment.

For those of you with a mandate to “move to the cloud,” the guidance is two-fold: (a) understand the difference between IDaaS (really SSOaaS) and IAMaaS (hosted, leased, managed IAM) and (b) focus on adopting best practices business processes, not just moving the IAM system from your data center to someone else’s.

A practical use for blockchain: disarming censors

July 24th, 2018

An interesting news story out of China:

The predictable, sad but not really noteworthy bit: a vaccine manufacturer saved money by making bogus vaccines and its executives were hauled away by police.

The predictable government response: delay publication of the news, as it implies a corrupt regulator and suppress subsequent social media posts on the subject.

The interesting and innovative bit: repost the story as metadata in an Ethereum transaction, such that it gets replicated across all participants in that blockchain and becomes impossible to tamper with or delete.

This is fascinating! An interesting way to totally defeat, once and for all, state authorities bent on censorship. Just encode public messages in blockchain transaction metadata. Brilliant, though dangerous (what if the state figures out who issued/signed the transaction? jail time at a minimum).

Most of the interest in blockchain is conjecture about hypothetical use cases, but this use case is legitimate and practical – a rare thing.

Privileged Access in an IoT world

June 19th, 2018

While it may seem like securing IoT is an abstract problem, in reality it is anything but.

Apparently Tesla (and possibly also SpaceX, separately, a couple of years ago) has been hit by internal sabotage to critical code and possibly other processes.

SC Magazine.

If you thought PAM was purely about administrator access to IT systems, you were wrong. How about PAM access to code check-ins? PAM access to factory control systems? PAM access to sensors (cameras?) and actuators?

This is reality, not theory.

Did your company spend millions on “access governance” and fail to automate anything?

June 8th, 2018

Imagine that you’re a mid-sized organization (say 2,500 employees in the case of a particular, real-world organization I have in mind) and a few years ago your company took the advice of a leading systems integrator / consulting firm and purchased licenses for the market leading IAM product.

You have since invested over $4 million in software, runtime platform, maintenance fees and lots (!) of implementation services. You have purchased as much consulting time as seemed appropriate, and you still have projects on the go.

By this time, after this much time and money spent, you would expect to have a lot of integrations and process automation in place, wouldn’t you? Several years and 4 million dollars should buy a lot, you’d think.

Unfortunately, if you were convinced to buy SailPoint, you probably haven’t got much to show for it. You have an excellent platform for running access certification campaigns, and you have built a system that integrates directly with your AD domain, plus consumes all sorts of CSV files from many applications, but that’s about it.

You haven’t automated any joiner/mover/leaver processes.

You don’t have a request portal rolled out to your users.

The SoD enforcement is sketchy at best — there are definitely ways to submit requests that should but do not trigger SoD violations.

Integrations are mainly via “flat files” — not real connectors.

But hey – there are pretty dashboards! Those must add value, right?

That’s not a great way to spend $4 million, but the integrator keeps proposing more — though every deployment phase seems to amount to just more CSV file “integrations.”

Maybe it’s time to get off the treadmill and look at better solutions, that can actually automate IAM processes, rather than delivering “access governance” but only promising (and never delivering) IAM process automation?

I can name a few organizations in this boat. Are you stuck in this situation too?

Please ‘like’ or ‘share’ if this scenario sounds all too familiar.

PC, Smart phone shipments declining

March 1st, 2018

I noticed two apparently-unrelated news articles today:


One talks about declining unit sales of phones and the other talks about the same thing for PCs.

Reading these articles, you’d think the sky is falling. Less phones are selling, on an annual basis! Less PCs too! Oh no! These industries are in trouble!

I think the reality is actually much simpler: PCs have been quite good for years now. Smart phones have likewise reached a plateau in terms of functionality and performance. Sure, new PCs and phones are nicer than old ones, but the extra screen pixels, camera resolution and compute power are not solving any new problems – they just meet the same old needs slightly better than before.

More than that, both PCs and phones are increasingly used to access cloud platforms. The heavy computation, storage, etc. are done on someone else’s server, not on personal devices any more. This makes local compute and storage even less meaningful.

So when a hardware manufacturer asks for $1000 for a shiny new phone or $1500 for a shiny new PC, most consumers compare the extra value of the new toy to the functionality of their existing PC or phone and choose to just leave the old device alone. It’s not broken. It’s working fine. Why upgrade?

I think this is the new normal. If you write software, or provide on-line services, then you shouldn’t care about this. People are on-line pervasively. They use browsers to access everything. There is no problem here.

If you are a phone or PC manufacturer, you just have to reset your expectations. The hardware replacement cycle is lengthening, not because consumers don’t want your product any more, but because your last generation was very good and the current generation is not so much better as to justify a new purchase. You’ll still sell lots of units, but the age of rapid growth is over. Heck, the only way you’ll grow your shipments is to win more market share from competitors. Get used to flat sales and razor thin margins folks. That’s the steady state of selling hardware.

Will this situation change? I doubt it. I have a hard time imagining what new capabilities phone or PC makers will be able to invent tomorrow, to excite consumers into a big hardware refresh cycle.

So a tough situation for hardware makers and a good situation for everyone else. That’s life.

Groups versus roles

February 12th, 2018

Periodically, I’m asked what the difference is between groups and roles. This is coming up even more frequently now that we’ve released version 11.0 of the Hitachi ID Identity and Access Management Suite, which includes full lifecycle management of groups (i.e., create, delete, update group objects on systems like AD).

This is actually a harder question to answer than one might think, because different systems and applications use the terms interchangeably. The object that I think of as a group might be called a group on one system (e.g., AD) and a role on another (e.g., SAP).

Ambiguous and overloaded terminology is very common in the IAM business, unfortunately.

Still, there’s a difference, once you define the two terms. To understand it, we should start with definitions. I’ll use Hitachi ID’s standard nomenclature here:

  • A group is an object that exists on a target system (LDAP, AD, SAP, RAC/F, etc.). Accounts are attached to the group as members. The group is assigned security privileges. This way, it’s not necessary to assign individual access rights to each user — which would be painful for a variety of reasons.Note that this is an approximate definition. For example, on some systems, groups can contain other groups as members (AD, RAC/F come to mind). On other systems, groups are also used as mail distribution lists (AD, LDAP). In some cases, a group on one system can contain among its members accounts or groups from another system (AD with cross-domain groups, for example).
  • A role is an object that exists on the IAM system. It’s used to collect entitlements (mainly groups, but also accounts) on target systems for more convenient assignment. Roles are typically nested — i.e., roles can contain other roles.Roles may be “technical” which just means that they represent collections of entitlements that are normally assigned together, but might not make much sense to business users, or they might be “organizational” which means that they represent business functions that requesters, authorizers or reviewers are more likely to understand. The difference here is a bit fuzzy, and that’s OK.

Beyond groups being on the target system and roles being in the IAM system, how are roles and groups different?

It turns out that how these objects are used in practice is quite different!

While IAM software vendors and implementation consultants strive to make role management a business-friendly task that managers will participate in, I’ve found that this rarely comes to pass. Managers rarely have the motivation, skills or time to participate meaningfully in defining or updating role definitions.

This has practical consequences: an IT team is ultimately responsible for defining and maintaining roles. They have limited visibility into business requirements and limited time to spend on any given role. These constraints make role definitions fairly static and formal.

Static, formal IAM roles usually wind up representing business functions (think job code). With static, formal roles it is natural to automatically assign roles: For example, birthright access for employees and contractors, departmental level roles for common access rights, job-code driven roles for jobs that are shared by large numbers of workers, etc. As HR systems or requests update identity attributes, the IAM system recalculates eligibility criteria and automatically assigns or revokes roles.

The consequence of all this is that roles are used to assign highly predictable, very regular access rights to users. If your location, department code, job code or cost centre can be used to predict your access rights and if your access rights are the same as many other people, then roles are the right tool to assign those rights to you, without extra manual intervention.

In practice, some access rights are simply hard to predict based on available data. For example, projects start up and wind down all the time. There is usually nothing in the HR database to indicate that a given user is involved in a project, so automation driven by HR data cannot grant or revoke project-related access rights.

This is part of a general pattern: automatic assignment of rights, via roles or not, only works well if we have both a sufficiently detailed, timely and reliable data source and a matching model that maps from identity attributes to entitlements.

Where this pattern ends is where directly assigned privileges begin. For directly assigned privileges, users could create new roles and request role assignment, but in practice users will skip the role definition step and just request the rights they want. i.e., users will request membership in the security groups that grant access to the share, folder or other item they wish to access, or ask to join a given mail distribution list.

There are two patterns for group membership management: groups may be included in roles, in which case they tend to be automatically assigned to appropriate users, or groups are not part of a role, in which case they tend to be requested on an as-needed basis.

Put another way: roles are formal and static, while groups may also be ad-hoc and dynamic.

What does ad-hoc mean?

  • Ad-hoc groups won’t be automatically assigned, so users must be able to sign into a portal and find and request them.
  • Users often don’t know what group to ask for (or even where to find the request portal), so significant usability support is required.
  • Ad-hoc may also mean creating and deleting the groups themselves, rather than just just requesting membership.
  • Anyone could request anything, so an approval workflow is required to prevent inappropriate requests from being completed.
  • Nobody ever asks for access rights to be reduced, so a periodic access review and revocation process is essential.

And that’s it:

  • Roles are IAM constructs used to build a formal model of predictable, static access rights that are usually automatically assigned,
  • Groups are objects on existing systems and applications. They may either be parts of roles, or may be requested, approved, reviewed and revoked in an ad-hoc manner.