Security keys, meet the real world

David Huerta 2019

Digital Security Trainer

Security keys on a cold, wet NYC sidewalk

One of the most common questions we get in training journalists on two-factor authentication (2FA) is: How hard are these hardware security keys exactly? Our security training team has plenty of anecdotes to support their durability, but we've decided to methodically put them to the test.

Screenshot of FreeOTP running on Android

Two-factor authentication (2FA) is a security feature available in many websites and apps which allows you to protect your login by requiring an additional piece of information beyond your password. Usually, this comes in the form of a one-time-use code sent to your phone through a text message or generated in an authenticator app, such as Authy or FreeOTP.

Authenticator apps and text messages are good enough in most situations. However, using a hardware security key for 2FA removes many of the risks associated with the security of the phone, helps prevent regular old human error inherent to using an authentication code received through a text message or authenticator app. Additionally, security keys which feature modern security standards offer robust protection against phishing attacks by automatically verifying the authenticity of the site you’re trying to log in to.

Although the cryptographic strength of the security standards used are well-established, the durability of the hardware they're implemented on is less known. Journalism happens in the real world outside of technical whitepapers, and the ability to securely do our work depends as much on the ability for these tools to survive not only cryptographic attacks, but the kinetic attacks of everyday life. In order to test for this, we threw three security key products on concrete, put them through a wash cycle and ran them over with a car to prove the mettle in the silicon for our favorite phishing-prevention tool.

The contestants

There are several security key manufacturers making a variety of security keys, but we decided to use the more common ones we've encountered: The Yubico Security Key NFC, Nitrokey FIDO2 and Google Titan Keys (commonly used for Google's Advanced Protection program).

Photo of the four security keys used in testing after the tire test

A set of four new keys was used for each test to control for order effects, testing a total of twelve keys. We set up each security key for a test Google account. It passes the test when we can log in with it, and doesn't pass the test otherwise. An up-to-date Google Chrome web browser on macOS Mojave was used for testing the key with a standard USB-A connection. We wanted to see how well the standard USB components could stand up to physical strain; we did not test NFC and Bluetooth features.

The drop test

Unless you’ve been living on a space station, you and many others have dropped their keys on a hard surface at some point. Most security keys are designed to live alongside your house and car keys on your keychain, so it's important that they can survive an unexpected drop.

Security keys are typically designed to go on a keychain, right next to your house, car or bike lock keys. To add a little realism to the scenario, we attached our security keys to a keychain with some house keys [1].

The floor we chose is solid concrete, in this case a sidewalk at Columbia University, where actual science happens all the time, presumably like this but with more methodical testing and more serious writing. We stood on top of a two-foot tall bench to drop our keys, simulating the vantage point of a particularly tall person.

After being thrown on the ground, each key survived with no visible damage and worked just as well as it did when it was brand new.

Drop test results

✅Nitrokey FIDO2

✅Titan Key (Bluetooth)

✅Titan Key (NFC)

✅Yubico Security Key NFC

A perfect score across all four keys.

The wash cycle test

We have no shortage of anecdotes about someone leaving their keys in a coat pocket that ends up in a washing machine. No security key in our list is advertised as waterproof, but we've been surprised at our personal security keys' ability to survive rainy weather and late night drink spills. As with the drop test, we attached our security keys to a keyring with house keys for optimal realism (and to make it easier to fish them out of a washing machine afterwards). We set a washing machine to cold water and a “medium” 30-minute cycle, subjecting the keys to full water immersion, centrifugal force and scent-free detergent. Will our tiny defenders survive the tumultuous maelstrom of deep water immersion and unrelenting spin cycles?

Security key rice tomb

Following a commonly known procedure used to dry out accidentally moistened electronics, we placed each key in a container of rice to dry them out. Other desiccants, such as silica gel packets that come with new shoes, would be as effective or better, but we stuck with using rice to replicate a more realistic scenario. All keys were placed in a 16 oz container with at least a 1" margin of rice above and below each key, and left to dry for 60 hours.

At the 60 hour mark, we checked each key for dryness, and tested each with a routine log-in. Although there was one very tiny six-legged insect that found a home in the rice jar, each key otherwise performed bug-free. When plugged in, each key blinked its lights as cheerfully as if it had been treated to a spa visit, but one where you get tossed into a cold wet centrifuge and entombed in a jar of grain.

Wash cycle test results

✅Nitrokey FIDO2

✅Titan Key (Bluetooth)

✅Titan Key (NFC)

✅Yubico Security Key NFC

Another perfect score.

The tire test

Although cars are not as popular for day-to-day commuting in the media capital of New York City or FPF's home base in San Francisco, they're still very popular in many other cities, especially in North America. Keys get misplaced, sometimes end up dropped on a driveway or even an open road. Even those of us commuting on bikes or public transit travel alongside heavy motorized vehicles racing right next to us and our loose pockets. We've already tested what would happen if the keys were to merely fall, so we figured we would also test what would happen if they were subsequently run over by a moving vehicle, by doing exactly that.

Cybersecurity student Scott Hodnefield volunteered his Toyota Corolla for the effort at scenic Papago Park in Phoenix, a city with no shortage of car commuters. At roughly five miles per hour, we ran over each key once going forward and a second time in reverse.

Which keys survived our fury road, forever shiny and Chrome-compatible?

Tire test results

❌Nitrokey FIDO2

✅Titan Key (Bluetooth)

✅Titan Key (NFC)

✅Yubico Security Key NFC

Although every key looked worse for wear after this test, all security keys except for the Nitrokey FIDO2 worked flawlessly after we ran them over with a car.

Now imagine your smartphone going through the same tests

Although not all keys survived our reign of destruction, it's important to remember that almost any smartphone you can get will probably not fare nearly as well, making hardware security keys a better option than relying solely on authenticator apps or text messages for 2FA. If you'd like to learn more about how to get started with two-factor authentication for your website or app accounts, check out Martin Shelton’s guide to 2FA on Medium.

Note: Each key used in our testing that didn't survive our testing has been e-cycled through the Lower East Side Ecology Center in New York. Look for local ordinances on e-waste disposal and de-list affected security keys from any associated accounts before throwing them away.

[1] These are not the house keys of anyone we know. We advise against sharing photos of your house keys on the public internet; due to 3D printing it’s now very possible to reproduce keys with just an image.

Donate to protect press freedom.

Your support is more important than ever.