PRISON OF MIRRORS We're not here because we're free. We're here because we're not free.

An Introduction to Password Hashing and Storage

July 19, 2012

With all the recent password leaks and other security-related blunders, I decided to talk about password hashing and explain it so non-computer savvy people can understand. Hopefully people will gain a better understanding about storing passwords and how to create good ones.

Plain Text

You might have heard the term “plain text” when hearing about passwords or one of the recent leaks (or the mega-Sony leak of last year). This is sometimes referred to as clear text. So what does this mean? It simply means the password is stored so any person can read it. Like so:

Passw0rd

The dangers of this should be pretty obvious: anyone who has access to the password knows what it is, and considering that in a database an user name — or even worse, an email address — is usually accompanying the password, the dangers are even more paramount (especially if you’re one of those people who reuse passwords. Hint: don’t do that). And if a database is leaked… game over. So how do we prevent someone who has access to the password from knowing what it is? This where hashing comes in.

Hashing

Hashing is a way of obfuscating text. It can be any text, but it’s commonly used for passwords. It’s a one-way algorithm that literally transforms the password into a fixed-length string of random letters and numbers (in hexadecimal, meaning each “digit” goes from 0 to 15, which is represented by 0 to 9, and a to f). As just mentioned, this algorithm is one-way, so there’s no way to reverse the hash. Passwords at the bare minimum should be stored as hashed values in the database and the plain text versions should never be stored under any circumstances.

There are many different hashing algorithms. Some are better than others. The two most common ones are MD5 and SHA-1. Neither of these algorithms are considered secure anymore but they are still widely used. Each hashing algorithm is one-way and always produces a fixed-length string of characters, known as a string. MD5 outputs a 32 character string and SHA-1 outputs a 40 character string.

Let’s take a look at the MD5 output for Passw0rd, the example used above:

d41e98d1eafa6d6011d3a70f1a5b92f0

As one can see, there’s no real way to tell what password this is just by looking at it. It is important to note that each algorithm always produces the same output for the same input. In other words, every time one enters Passw0rd, the result will always be what’s above. This is how websites and other applications are able to authenticate users. If the user enters in an incorrect password, the hashed output will be different than what’s in the database and therefore will not be authenticated.

Another important thing to note is that input that is sequential or closely related will result in a drastically different output. Here’s a quick example using SHA-1:

Passw0rd is ebfc7910077770c8340f63cd2dca2ac1f120444f in SHA-1
passw0rd is 7c6a61c68ef8b9b6b061b28c348bc1ed7921cb53 in SHA-1

As you can see, changing just the first letter from uppercase to lowercase has a drastic effect on the output, and the two do not look similar at all. This helps make the algorithms stronger.

Predictable Output

While the output isn’t predictable to humans, it certainly is to computers. In an effort to “crack” hashed passwords people have created “dictionaries” and what’s known as rainbow tables to cover as many hash possibilities as possible. These dictionaries can number in the billions, and you can bet that the common passwords people choose on a regular basis are in there. There are rainbow tables that cover all combinations of upper and lowercase letters, numbers, and symbols up to eight characters for the MD5 algorithm. This is why it’s so important not to use some simple password like J@son123.

This suddenly doesn’t sound very secure. How do we mitigate the effects of precomputed hash dictionaries and rainbow tables? We use what’s known as a salt.

Salting

Salting is adding an additional string to the password, and then hashing the new string. Salts are usually appended to the front or end of the password. There are a few caveats about salts, though. The same salt should never be used more than once and should always be unique, changing every time the user changes his password. The salt should also be long and random, probably the same length as the hash algorithm’s output. The longer and more random, the stronger it is. I personally like to use a combination of PHP’s mt_rand() function (without arguments), which generates a random number between 0 and 2147483647, and the current date and time, formatted in a specific way, accurate to the second.

Adding a salt has two major strengths: 1) it makes the password much harder to guess, and a good salt would render hash dictionaries and rainbow tables useless, forcing an attacker to use brute-force techniques which could take centuries or longer, and 2) if two users are using the same password, the hashes that are stored in the database will look completely different. Since it’s expected that a lot of common passwords are reused, many unsalted hashes in a large database would be the same. An attacker would only have to crack it once, though, for all of the same password to be revealed. Adding a salt makes this impossible.

Simply put, adding a salt is just adding a string that’s so long, so ridiculously random, that it’s improbable to appear in any hash dictionary or rainbow table. Take a look at the following two passwords. Which do you think is more likely to appear in someone’s list?

Passw0rd
d32c00e3c7935e76da471babeda400c902903ee0Passw0rd

And what if three people in the database are all using the password Passw0rd?

The SHA-1 hash values of the following passwords

4e4940aa6f9df8148aafa6ed458d583091b6c162Passw0rd
79564deffc86af62e08f90d5cc432880357f5773Passw0rd
b5c4f7b8919242ed4316e302deb50748eb235812Passw0rd

Result in:

2a22aeaf4e0a6f10f475effa1256feff3b39b328
23da0509aaebb5b0e5911f9e1a5fb624f3bc6b0a
7e130f3cbb36cb86cd3caef24c554605c363012f

It’s not obvious that all three of those users are using the same password. Salting is essential to password security and every website should use it. Salts are usually stored in the database, and while this may seem detrimental, keep in mind that for an attacker to break the hash, they would have to apply the salt to every single entry in their dictionary, which would take an enormous amount of time — per password! Also don’t forget that the attacker has no way of knowing (unless they are able to get a hold of the application’s source code, which has happened before) whether the salt is appended to the front, end, or something else entirely before being hashed. Even if they did know, the amount of time required to break even a single password becomes infeasible.

Conclusion

I hope password hashing and storage has been explained simply enough for everyone to understand, and I hope after reading this post, you reevaluate your passwords. One other thing to keep in mind: hashing algorithms can handle any kind of string input, and do not require limitations; websites that impose arbitrary restrictions on your passwords should set off red flags because it’s an indicator that they’re storing the passwords in plain text. Think twice about the passwords you use for these websites and services.

Posted in Security

The Price of Security

July 15, 2012

There has been quite a few data breaches in the last month or so. LinkedIn, eHarmony, and Last.fm were all breached in early June and had user credentials leaked. Just a few days ago Yahoo!, Billabong, Formspring, Phandroid, and Nvidia were all breached as well with user information including passwords leaked. These breaches vary in severity; some of these sites were storing their passwords in plain-text (why are companies still doing this? Haven’t we learned from the Sony debacle?) and others were hashed. Storing passwords in plain-text is the biggest (and simplest to avoid) security mistake a company can make. Hashing the passwords is definitely better, but they’re not infallible as there are sites out there with massive hash dictionaries.

I don’t have much else to say about these breaches except that if you have an account with one of the above websites, I suggest you change your password immediately, as well as any other account where you may be using the same user name/email address and password combination (you’re not doing that, though, right?).

I’m no expert when it comes to security, but I like to think about it a lot, especially when I read about these high-profile breaches. I also like to think I’ve learned quite a bit over my years of programming that I could make a pretty secure web application. It may not be impenetrable, but it would be enough to repel opportunists. And that’s really what it’s about, isn’t it? Preventing “crimes of opportunity”?

I liken web and computer security in general to the security of a home, and what the price of it is. The price of security is convenience. Imagine your home with little to no security: you leave your doors unlocked or open, or heck, you may not even have a door! Anyone could come and go as they please. It’s very insecure but also very convenient. You never have to worry about forgetting your keys, or losing them, or locking yourself out. It’s a dream! Ever come home and your arms are full of groceries and you’re struggling to get your keys and open your front door? No need for that, just walk in!

Of course, no one lives like that, so we put some security on our homes: we have doors with locks, we have garage doors with keypads, we have locked windows. Some homes even have security alarms or security monitoring. Others may even own a dog or two (though I hope they have the dog for companionship rather than for security).

Pretty standard security. But houses get broken into all the time. It’s a very common occurrence. Imagine upping your home’s security. Imagine every room in your home had a locked door. An intruder could break in but he would be very limited on what he could do. Your house is very secure. On the flip side though it’s incredibly inconvenient. Every time you want to go to another room you have to bring a key. What if you lose a key? Uh oh.

So instead we choose to have a certain level of security because too much of it is too inconvenient. The same applies to web applications. It applies to both the user and the developer. A huge amount of forethought has to go into designing, testing, debugging, and fixing a secure system (as well as maintaining, and hopefully documenting). Users have to be able to use the system easily enough so they don’t become frustrated and leave. Imagine having to enter your user name and password every time you wanted to access a page of a site, or having different passwords for different areas. It would be secure because an attacker would need to know the specific password to the specific area he wants to access, or try to get them all. But it would be a huge pain to actually use.

In the end we have to decide what is an acceptable amount of inconvenience. I’m not sure there’s a definite answer to that, but we should strive to find the middle ground between too much security and too much convenience.

Posted in Security

Is ASUS Sexist?

June 21, 2012

I randomly stumbled across this post the other day. ASUS was at Computex 2012 this year in Taiwan. There the company tweeted about it’s new transformer AIO. But they didn’t just tweet about their new gadget, they also tweeted about (what appears to me) a showgirl’s behind. This has apparently caused quite a stir, and ASUS has since deleted the tweet and issued an official apology.

I’m a little late on this but I’d like the opportunity to weigh in. First and foremost, I don’t think ASUS is sexist. I don’t think they intended to be malicious with their remark or to alienate any of their customers or anyone else for that matter.

What surprises me the most is the backlash. Some of the comments and reactions from people are ridiculous. You can peruse through them at your leisure from the link above, but some of the highlights include people making statements such as, “ASUS doesn’t think women want computers”, “what kind of company they are”, “ASUS just lost me as a customer”, and, “ASUS apparently doesn’t think women buy computers. Piggish in the extreme”. The last one is from Ed Bott, a writer over at ZDNET.

Are these people for real?

Do these people honestly believe what they’re saying and honestly believe that ASUS is a sexist company trying to… I don’t even know. Make it more difficult for women in tech? Lose customers? If people are so concerned about women in tech, then why are there showgirls at these events in the first place? These women clearly get all dolled up to be in front of thousands of people and cameras. Whether we want to admit it or not, their purpose there is to look good. I didn’t realize complimenting a woman’s posterior was now considered sexist.

ASUS’ comment wasn’t being malicious or harassing. It was just an off-comment with some light-hearted humour with no “endgame” or any other intent. These people need to get over themselves and stop making a mountain out of a mole hill. These people need to find something actually worthwhile to get upset over… like the massive human rights violations in China; I bet these same people will go home and browse the Internet on their iPads and other Apple products. And Apple isn’t alone in these violations; other tech companies have their gadgets manufactured in China.

But do we really think these people would be willing to “lose me as a customer” because of that? No. They rather race to Twitter and make an emotional comment about something they’ll literally forget about the next day.

Posted in Hardware, Miscellaneous

The Dvorak Keyboard – Part III

June 19, 2012

So it’s been nearly four full months using the Dvorak keyboard. Where do I stand? How do I feel about my progress? Well to be honest not as good is I envisioned, but that doesn’t mean I’m not making progress. I still make a lot of mistakes – way more than I am comfortable with, but I am also finding that I am able to type some words without even thinking where the keys are, which is really the endgame.

What I really need to do is do more exercises. I know where the keys are, I just don’t know them by muscle memory. I suppose that will come with time, but I notice myself “finding” the keys by hitting the wrong key and then correcting myself. Needless to say that’s not how it should be done.

For fun I jumped on Typeracer the other day just to see how I’d do it a competitive typing environment. I played two races before giving up. I actually won my first race (the only reason I stuck around for a second) with 34wpm. With a score like that, I know I didn’t win because I use a Dvorak keyboard; I won because obviously my competition wasn’t very good. I got owned in the second race and promptly closed the tab. It’s clear I simply don’t know the layout well enough yet. I just make way too many mistakes.

Let’s revisit some of the things I’ve talked about before: the M and W keys don’t give me as much trouble as they used to. I’m finding now actually to be having trouble with the W and V keys instead. I guess it’s fortunate that they’re uncommon letters. I have to make a conscious effort which one I am hitting but hopefully that goes away.

I still don’t like the placement of the O and A keys, but I have come to terms with them and have moved on. I still think the vowels (I hate typing that word, for reasons above) could be rearranged more intelligently, though.

I have recently written a script that analyzes text and does all sorts of calculations about the letters and letter frequency (this will be added to the projects section soon) among other cool stuff. I have found that M is far more common than previously thought and the letter F is far more scarce. So really I would swap these keys, though on second thought maybe I’d leave them as they are because I find it difficult to hit the F key.

The key that’s hardest for me is the X key by far. Thankfully I don’t have to use it very often. I find it’s much easier to hit the Z key than the X key, which is unfortunate because Z is the least common letter.

The Future

So where do I go from here? I’m not really sure. I guess practice makes perfect and I’ll learn all the muscle movements over time. There have been many frustrating moments but I do think I am getting better. Going back to the QWERTY keyboard is not an option at this point. My realistic outlook on when I’ll be just as good with the Dvorak layout as I was with the QWERTY layout is by end of this year. I think that’s a nice, conservative estimate. It gives me a lot of time to practice and to get the muscle movements down pat. I’ll probably check in with some more progress sometime in July.

Posted in Hardware

Project Management

June 18, 2012

It’s been a really long time since I’ve posted a blog entry – more than a calendar month. I have just been so busy I haven’t had an opportunity to write for the blog lately. That will hopefully change in the coming weeks as I am a little more free now.

This will be rather short but just wanted to mention a new project that has been launched recently. Speedway Photo was a website I worked on with a friend over the last few months. He does some fantastic racing photography and we wanted to redo his current website to bring it up to par with the quality of the photos.

The website isn’t 100% finished; there are still a few outstanding items we have to check off our list, but for the most part it’s done. It was just the two of us working on the website and managing it. It’s a little bit of a challenge working on a project and also being the project manager. It really helps if there’s an outside person overseeing everything.

Maybe next time there will be an external presence for our project management.

Posted in Miscellaneous