One of the most common questions we get in training journalists on two-factor authentication (2FA) is: How hard are these hardware security keys exactly? Our security training team has plenty of anecdotes to support their durability, but we've decided to methodically put them to the test.
Two-factor authentication (2FA) is a security feature available in many websites and apps which allows you to protect your login by requiring an additional piece of information beyond your password. Usually, this comes in the form of a one-time-use code sent to your phone through a text message or generated in an authenticator app, such as Authy or FreeOTP.
Authenticator apps and text messages are good enough in most situations. However, using a hardware security key for 2FA removes many of the risks associated with the security of the phone, helps prevent regular old human error inherent to using an authentication code received through a text message or authenticator app. Additionally, security keys which feature modern security standards offer robust protection against phishing attacks by automatically verifying the authenticity of the site you’re trying to log in to.
Although the cryptographic strength of the security standards used are well-established, the durability of the hardware they're implemented on is less known. Journalism happens in the real world outside of technical whitepapers, and the ability to securely do our work depends as much on the ability for these tools to survive not only cryptographic attacks, but the kinetic attacks of everyday life. In order to test for this, we threw three security key products on concrete, put them through a wash cycle and ran them over with a car to prove the mettle in the silicon for our favorite phishing-prevention tool.
The contestants
There are several security key manufacturers making a variety of security keys, but we decided to use the more common ones we've encountered: The Yubico Security Key NFC, Nitrokey FIDO2 and Google Titan Keys (commonly used for Google's Advanced Protection program).
We conducted three durability tests on our security keys. We tested a set of four new keys each test, for a total of twelve keys.
We set up each security key for a test Google account. If we can still use it to log in, it passes the test. We used an up-to-date version of Google Chrome on macOS Mojave for testing the key with a standard USB-A connection. For keys with wireless NFC support, we used iOS 14.4 and the latest Safari on an iPhone SE (2020).
The Bluetooth-enabled Titan key did not have its Bluetooth tested, since the need for that feature was made obsolete for iPhones when iOS enabled support for NFC-enabled security keys, and the same key also has NFC support.
The drop test
Unless you’ve been living on a space station, you and many others have dropped their keys on a hard surface at some point. Most security keys are designed to live alongside your house and car keys on your keychain, so it's important that they can survive an unexpected drop.
Security keys are typically designed to go on a keychain, right next to your house, car or bike lock keys. To add a little realism to the scenario, we attached our security keys to a keychain with some house keys [1].
The floor we chose is solid concrete, in this case a sidewalk at Columbia University, where actual science happens all the time, presumably like this but with more methodical testing and more serious writing. We stood on top of a two-foot tall bench to drop our keys, simulating the vantage point of a particularly tall person.
After being thrown on the ground, each key survived with no visible damage and worked just as well as it did when it was brand new.
Drop test results - USB
✅ Nitrokey FIDO2
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
✅ Yubico Security Key NFC
Drop test results - NFC
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
✅ Yubico Security Key NFC
A perfect score across all tested keys.
The wash cycle test
We have no shortage of anecdotes about someone leaving their keys in a coat pocket that ends up in a washing machine. No security key in our list is advertised as waterproof, but we've been surprised at our personal security keys' ability to survive rainy weather and late night drink spills. As with the drop test, we attached our security keys to a keyring with house keys for optimal realism (and to make it easier to fish them out of a washing machine afterwards). We set a washing machine to cold water and a “medium” 30-minute cycle, subjecting the keys to full water immersion, centrifugal force and scent-free detergent. Will our tiny defenders survive the tumultuous maelstrom of deep water immersion and unrelenting spin cycles?
Following a commonly known procedure used to dry out accidentally moistened electronics, we placed each key in a container of rice to dry them out. Other desiccants, such as silica gel packets that come with new shoes, would be as effective or better, but we stuck with using rice to replicate a more realistic scenario. We placed all keys in a 16 oz container with at least a 1" margin of rice above and below each key, and left to dry for 60 hours.
At the 60 hour mark, we checked each key for dryness, and tested each with a routine log-in. Although there was one very tiny six-legged insect that found a home in the rice jar, each key otherwise performed bug-free. When plugged in, each key blinked its lights as cheerfully as if it had been treated to a spa visit, but one where you get tossed into a cold wet centrifuge and entombed in a jar of grain.
Wash cycle test results - USB
✅ Nitrokey FIDO2
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
✅ Yubico Security Key NFC
Wash cycle test results - NFC
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
❌ Yubico Security Key NFC
Another perfect score for USB tests and a reminder from NFC tests that none of these keys market themselves as waterproof.
The tire test
Although cars are not as popular for day-to-day commuting in the media capital of New York City or Freedom of the Press Foundation’s founding office in San Francisco, they're still zipping through the roads in both cities. Keys get misplaced, sometimes end up dropped on a driveway or even an open road. Even those of us commuting on bikes or public transit travel alongside heavy motorized vehicles racing right next to us and our loose pockets. We've already tested what would happen if the keys were to merely fall, so we figured we would also test what would happen if they were subsequently run over by a moving vehicle, by doing exactly that.
Cybersecurity analyst Scott Hodnefield volunteered his Toyota Corolla for the effort at scenic Papago Park in Phoenix, a city with no shortage of car traffic. At roughly five miles per hour, we ran over each key once going forward and a second time in reverse.
Which keys survived our fury road, forever shiny and Chrome-compatible?
Tire test results - USB
❌ Nitrokey FIDO2
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
✅ Yubico Security Key NFC
Tire test results - NFC
✅ Titan Key (Bluetooth and NFC)
✅ Titan Key (NFC)
❌ Yubico Security Key NFC
Although every key looked worse for wear after this test, all security keys except for the Nitrokey FIDO2 worked flawlessly over USB after we ran them over with a car. The Titan keys, living up to their name, were the only ones to continue to work over NFC.
Now imagine your smartphone going through the same tests
Although not all keys survived our reign of destruction, it's important to remember that almost any smartphone you can get will probably not fare nearly as well, making hardware security keys a better option than relying solely on authenticator apps or text messages for 2FA. If you'd like to learn more about how to get started with two-factor authentication for your website or app accounts, check out our guide to two-factor authentication.
[1] These are not the house keys of anyone we know. We advise against sharing photos of your house keys on the public internet; due to 3D printing it’s now very possible to reproduce keys with just an image.