OK, I admit to engaging in some attention-grabbing in that heading,
and the title. It might even be a little bit unfair, although not
much. How about this instead?

SSH Keys Don't Really Improve Security

Ah, maybe even that is a bit much. How about ...

SSH Keys Are About Equal To SSH Passwords In Terms Of Security If You're At All Careful

Better? That's at least not insane sounding, yeah? By "at all
careful" there, I mean that you're running
denyhosts or
fail2ban or checking your shadow file
with john or otherwise mitigating
against the constant flood of external password-based brute force
attempts that's part of having ssh open on the modern internet.

So, on to backing up that claim.

The first thing to understand is that security analysis always
starts with attack vectors; if you can't talk about who might attack
you, how, and in some cases even why, you can't talk about how to
best defend against attack.

SSH-Based Attack Vectors In The Real World

(There's a much shorter version of this below.)

Most sysadmins, at one time or another, have experienced intrusions,
successful or otherwise. I am in the unfortunate position of having
experience two successful SSH intrusions, which are largely the
basis for this essay, because the things that sysadmins are telling
each other about how to prevent break-ins simply would not, and could
not, have prevented these. In one case, I actually deliberately
decided to not do anything to solve the "problem" at all, because
the system was working as I had designed it. But I digress.

If you're a sysadmin with any experience, think back to the last
time a system of yours was attacked, or even compromised. Or the
time before that. Or the time before that. And, as a favour to me,
ask yourself this question: when was the last time that a successful
(relevantly successful; actual damage and stuff) or near-successful
break-in actually had anything to do with anyone's password?

In my case, never, but it does happen of course; there are giant
networks of ssh-brute-forcing bots (which, by the way, is what
fail2ban is for);
presumably it works or they wouldn't bother. It's sill the case
that your passwords need to be moronically simple to be vulnerable
to this, though, and I don't think that anyone with such passwords
on their system would be reading this essay in the first place.
Running john against your shadow
file every once in a while can't hurt, though.

In my case, I've had a number of serious break-ins or attempts based
on web vectors; all easily controllable. I've had two SSH-based
attacks that worked: one was a password sniffing sshd installed on
another machine, and the other was ssh key based.
I get hundreds (literally) of ssh attacks every day; piles of syslog
lines about people trying to log in as root or whatever. None of
those attacks has ever succeeded at getting in.

The reason for this is very simple: at this time in computer
security, for normal people that aren't active targets (try working
for Symantec if you want some practice being an active target, BTW),
there are exactly two kinds of attacks:

1. Disgruntled employees, who are on site and have legitimate access, designed to trash things or copy them to sell to other companies (most successful attacks are disgruntled employees, by the way; never forget that)
2. Remote attacks over the Internet where you are simply one vulnerable host out of millions, you get added to a botnet and no human ever directly interacts with your machine; if your system doesn't crack on the first few tries, such attackers will just try another host. There's no shortage of vulnerable hosts on the 'net.

The first item there is totally outside the scope of this document, because it is impossible (by definition) to create a purely technical solution to prevent someone from mis-using access that they legitimately possess. If a basic part of a person's job is to delete files under /foo/bar, and they decide to delete everything under /foo/bar because they're mad, no software will stop them; that's an AI complete problem. Now, you can make it harder for them to also delete things under /foo/baz, which they shouldn't be deleting things under, but that's an SE Linux problem or something; it's certainly not an SSH problem, in most cases, and for those cases where it is, everything in this essay applies.

What you do not see on that list of attacks:

1. Individual, targeted attacks that involve a human being selecting you personally, interacting with your systems personally, and being willing to devote anything more than totally insignificant levels computing resources to get in to your machine

Unless you're a top-secret military project, no-one is going to
rifle through your garbage looking for your passwords. No-one is
going to go wandering around your office looking for passwords.
Basic things like 2-second-per-try timeouts on SSH and such
make external brute-force attacks a giant waste of computing
resources: normal attackers, in real-life, on today's Internet,
simply won't bother. They'll move on to easier prey.

If your important passwords are vulnerable to one or two or 50
hits by an external brute-force attack, you fail. If your important
passwords are vulnerable to someone getting a copy of shadow and
running jack the ripper on it, you also fail, but I've
never actually seen this happen, ever, and I've been a
sysadmin for almost 15 years now (as of January 2010). If one of
your users has such a password, that shouldn't be a big deal either:
a user with such a password didn't have any special privileges
(RIGHT!?), and non-privileged single-user intrusions are trivial to
deal with, in general.

I've never seen this happen either, by the way, via SSH. The 2
second (or whatever) delay on repeat attacks means that basically
anything except "password" and password == username is impossible to
brute force in a reasonable length of time, and attackers will move
on to easier prey.

Attack Vectors: The Summary

The 99.999% (five nines) attack vector on a normal (not government,
not a computer security firm, not a super-giant corporation, not a
bank) Internet-accessible host on today's modern Internet is an
attack of opportunity, delivered in such a way that whether or not
your SSH passwords are secure is very close to totally irrelevant.

A Digression: Sticky Notes Are Better Than Simple Passwords

Even when passwords matter, again, unless you're a government or a
megacorp, no-one is trolling your office or your garbage for
passwords.

There's a lot of back and forth between sysadmins on the questions
of: "Which is better? fido21 as a password, memorized, or
hncroNTHS31cgrl as a password written down?". Consistently, though,
over the last 5 years or so, I've seen sysadmins coming down on the
side of the latter. Given the nature of modern computer attacks,
I'm going to be public about it:

Unless your situation is very exceptional, a decently secure
password written down is better than a weak one memorized. The
former is essentially invulnerable to remote brute-force attacks,
and that makes all the difference.

Having said that, pass phrases are better than any of that; see
http://www.codinghorror.com/blog/archives/000342.html and
http://www1.umn.edu/oit/security/OIT74291_REGION1.html#passwords_and_passphrases
and other places google knows about.

Another Digression: Remote Fun With sudo

Having password requirements on sudo does not protect you from
ssh-key-only attacks. sudo generally has a 5 minute window where
users do not need to re-enter the password. An attacker with access
to a user's SSH key only needs to run a quiet loop somewhere trying
out sudo access and wait until the user legitimately uses sudo. The
one time I've been seriously rooted, a totally unrelated machine was
compromised, a user with sudo on my box had their SSH key stolen,
the SSH key was used to get into my box, and then sudo was run. We
don't know if the sudo timing trick I just mentioned was used, or if
the user's actual password was sniffed on the compromised machine,
but the fact remains: it is possible for an attacker to gain
access to a user's sudo privileges ''without ever knowing their
password'' if you allow SSH key access.

You can mostly fix this problem this with tty_tickets, though. It's
on by default in some distros.

Thanks For Security 101 There; How Do SSH Keys Fit In?

That's simple: in the modern attack environment, SSH keys are a
fuzzy, comfy, pretty doormat that says "Please Rape Me".

To understand where I'm coming from here, think about this: when you
trust a user's SSH key, what, exactly, are you trusting?

Please actually think about that for a minute.

The answer is: you are trusting the security of every other host
that the user with that key ever uses. You are not trusting
the user in question, which is the big mistake people make when
they think about SSH keys. You're not trusting the user, you're
trusting the system where the key is stored. Or, more likely,
the systems. Perhaps many of them.

But SSH Keys Have Passwords!

No, they don't. Not from the server's perspective.

Thinking About Security From A Server-Based Perspective

This is the big problem with discussions about ssh keys: people talk
about setting up ssh keys from the perspective of the user. As
an SSH user trying to ensure that only you get to log in as you, SSH
keys with passwords set on them are pretty good, actually. You, of
course, engage only in impeccable security practices. You would
never have a passwordless key for scripts or backups (I certainly
do, by the way, and it's a key that gives root access, but it's
associated with a forced command so that's not really a serious
issue), and since it's for backups it has to be a root key), you
only keep keys on one system (I copy my keys around) that you have
personal physical control over (hah!) and you never run ssh-agent
(see below). That's great; good for you. Access to your account is
now pretty excellently secure.

From the perspective of the server admin, though, none of this is in
any way relevant. SSH keys are sent to the server after the
password has been used to unlock them. An SSH server has no way to
tell if an SSH key has a password set on it.

People have told me, many times, the exact opposite, so this bears
repeating:

An SSH server cannot tell if an SSH key used to log in to it has a password set on it or not.

People get this sort of thing really, profoundly, staggeringly
wrong. I apologize to the author for picking on a particular
article, but this article on "two-factor security" in SSH
really upset me, because from a server perspective it's basically a
flat-out pack of lies. The author lists a staggeringly complicated
set of steps designed to make it so that the user can't log in
without using a physical USB key. It's really pretty complicated
for a security measure. Here's the fun part, though: the remote SSH
server can't even tell that these steps have been done. From
the server's perspective, nothing has changed. How can you call it
two-factor authentication if the server can't refuse entry unless
both factors are present? How can you call it two-factor
authentication when the server cannot possibly even notice that
one of the factors exists? I don't think you can, and I don't think
you should use that term for a setup like this; I find it dishonest,
although I'm sure that wasn't the author's intent. This sort of
user-centric thinking is endemic to discussions about SSH.

No, Really: SSH Keys Don't Have Passwords (From The Server's Perspective)

If you don't believe me, run sshd with -dddddd or something, and see
if you can tell any difference between an ssh key with a password,
and the same one without (use ssh-keygen -p to change the password).
You can't see a difference because there isn't one: the password is
used to decrypt the key on the client end, and only then is the
key used to talk to the server.

So, as the admin of a server with lots of different users, you have
no way to tell if users have passwords on their ssh keys, and no way
to enforce it one way or the other. This means that prudence
dictates that you must assume that every ssh key with access to
your box has no password.

Even if you trust all your users to have passwords on their ssh
keys, though, you still have to assume that, because of
ssh-agent.

ssh-agent And Remote Compromises

Just about everyone who has to do ssh more than once a day runs
ssh-agent. Why? To save having to type in passwords all the time.
From a server security perspective, that's a big red flag: how can
ssh-agent do that? Does it store the password in RAM? No, it
simply uses the password to decrypt the key and stores the key
in RAM.

A machine with ssh-agent running that has been root compromised has
no passwords on any key loaded into any ssh agent: anyone with root
can just manually attach to a running ssh-agent. There's nothing to
prevent this, at all.

In a way, that's irrelevant, though: someone with root on a box
your users log in to your machine from is going to have access to
your box in short order no matter what, if they want it. There are
all sorts of ways to get keys from a compromised box. ssh-agent
makes it slightly faster and less risky for the attacker, but it's a
fairly minor point. SSH keys and passwords are about equal here,
which is my point: SSH keys are not a security panacea.

I'm not talking about the machine a user runs ssh to, I'm
talking about the machine they're coming from, where ssh-agent is
actually running.

Note that, again, there is no way to tell if a user is using
ssh-agent: from the server perspective, a key is a key is a key.
Password, no password, ssh-agent, doesn't matter: it all looks the
same to the server.

So, ssh-agent means that if the remote machine is compromised, all
the keys are too. So we're back to this:

In Summary: Do You Trust Every Machine Every User Logs In To Your Machine From?

Every SSH key your system accepts as solely sufficient for
authentication means trusting every system that user might ever
decide to log in to your system from. You trust the physical
security of that system, and the administrators of that system. You
trust the administrators of that system not so much to not be
malicious as to be perfectly competent: to stay on top of every
security patch forever.

This is also true for passwords via SSH. Pretty much everything
I've said is equally true for passwords and keys; that's really the
point.

Since you have no way of knowing whether a key used to access your
system is password protected (and since people use ssh-agent), you
must assume none of them are password protected.

If you really do trust all of those systems, great; you're far
luckier than I am.

OK, So What Do We Do?

Well, for starters, you could stop telling me how insecure passwords
are with SSH. As far as I can tell, in the real world, SSH keys
are far more insecure than SSH passwords. With SSH passwords, I
only need to trust the user, and the physical environment (sticky
notes the password is on in the user's wallet, or whatever; I'm
assuming that no user with actual privileged access is stupid enough
to write the password down in plain text on some remote computer).
With SSH keys, I have to trust every computer system the user might
ever decide to log in from, and in some cases the systems they might
log in to those systems from too. That's a lot more things to
trust, and given the nature of modern attacks, I am far more
worried about compromises to random systems on the 'net than I am
about what's in a user's wallet.

I'm of the opinion that ssh keys and passwords are about equal in
terms of security; they both have their places. Neither is a
panacea, and you are not committing security suicide by leaving
either enabled.

The right solution for secure systems, IMO, is the
RequiredAuthentications keyword that the actual SSH Communications
Security company's SSH implementation (but not openssh) supports.
Require both a key and a password; that'll clear the problem
right up. Casual ssh-key-based attacks coming from some other
random machine that got compromised and had one of your user's keys
on it become a thing of the past.

What's that, you say? That's horribly inconvenient, because now the
user has to type in a password?

Welcome to discussions of security. There is almost always a
trade off between convenience and security.

It doesn't matter anyway, because the sshd on all the systems I use
doesn't have that capability anyways. I would like to see that
fixed some day, though; for systems where I really care about
security, ssh key attacks scare me quite a bit, having already been
bitten by them once now.

Exceptions

There are a couple of cases where it really matters whether you're
using keys or passwords

Keyboard Logging

As far as ssh keys being obviously more insecure than passwords,
trojans that try to capture the user typing passwords can level the
playing field there. I consider this something of a red herring,
for several reasons:

1. In all the attacks and near-attacks I've seen, I've never actually seen a keyboard logger, or anything like it. But then, this was all on Linux machines (I use Windows for my desktop, but it's not accessible from the general Internet).
2. While there are cases where it's possible for a trojan to know "Hey, this string right here is a password" (such as a password dialog), I'm not aware of any usage of SSH software on Windows (which is the attack vector we're talking about: most other OSes don't really have key logging trojans) where such a trick works (although if you store your passwords in the SSH program's config, that's probably vulnerable). This means a human has to sift through the key log, which is a non-casual attacker and not what I'm worried about as I've said.
3. Most importantly of all, though: even if keylogging trojans were a pervasive problem that didn't require human intervention, this wouldn't make ssh keys more secure than passwords, it would just make the two exactly the same for my purposes: I can't trust any machine a user logs in from, in either case.

Scripts

Scripts that need remote access should always use keys, and the
keys should be linked to a forced command and a particular incoming
IP on the remote end. Having a password in a script is inexcusable.
See "man authorized_keys", in particular the command= and from=
options.