Is md5 really more secure than crypt?

'password' authentication is insecure because plaintext passwords are stored
on disk and plaintext passwords are sent over the wire. I could steal the
password by watching the network or by looking in the pg_pwd file.

'crypt' authentication is insecure because plaintext passwords are stored on
disk, but encrypted passwords are sent over the wire. I can't steal the
password by watching the network, but I can still look in pg_pwd.

I thought that 'md5' authentication was supposed to be better than
'password' and 'crypt' because encrypted passwords are stored on disk and
encrypted passwords are sent over the wire.

md5 works by storing an md5 checksum for a user/password combination on disk
(this looks like an encrypted password). When a client connects, the server
sends a small salt value to the client. The client computes an md5 checksum
over the user/password combination, then a second checksum over the first
checksum plus the salt value. Then the send checksum is sent to the server.
The server combines the stored user/password checksum with the salt value
and then computes its own checksum. If the client's (second) checksum
matches the server's (second) checksum - the passwords match.

But, if can peek at the server's user/password checksum (in the pg_pwd
file), I can connect to a server, get the server's salt, and combine it with
the stolen checksum, arriving at the checksum expected by the server.

This is exactly how I would impersonate a user authenticated by 'crypt'.

So, to me, it doesn't seem that 'md5' is much more secure than 'crypt'. The
user/password hash stored in pg_pwd is essentially a plaintext password.
What am I missing here?

-- Murhpy

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

On Fri, Jun 14, 2002 at 10:54:35 -0400,
murphy pope <pope_murphy(at)hotmail(dot)com> wrote:
>
> But, if can peek at the server's user/password checksum (in the pg_pwd
> file), I can connect to a server, get the server's salt, and combine it
> with the stolen checksum, arriving at the checksum expected by the server.
>
> This is exactly how I would impersonate a user authenticated by 'crypt'.
>
> So, to me, it doesn't seem that 'md5' is much more secure than 'crypt'.
> The user/password hash stored in pg_pwd is essentially a plaintext
> password. What am I missing here?

I think MD5 is preferred because it provides better protection against
reversing a hash and you can use longer passwords. This helps against
some kinds of attacks.

On Fri, Jun 14, 2002 at 10:54:35AM -0400, murphy pope wrote:
> So, to me, it doesn't seem that 'md5' is much more secure than 'crypt'. The
> user/password hash stored in pg_pwd is essentially a plaintext password.
> What am I missing here?

I think the main reason is that if someone can log into the machine, access
the password file directly (probably via root), then you have more serious
problems than someone impersonating someone else on a connection. They could
simply suck your entire database out and read it elsewhere.

As for making it more secure, I would say to use a one-way hash on disk
(similar to /etc/passwd) and then connection via SSL to stop the password
being sniffed. But isn't this what happens already?
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> There are 10 kinds of people in the world, those that can do binary
> arithmetic and those that can't.

"murphy pope" <pope_murphy(at)hotmail(dot)com> writes:
> But, if can peek at the server's user/password checksum (in the pg_pwd
> file), I can connect to a server, get the server's salt, and combine it with
> the stolen checksum, arriving at the checksum expected by the server.

Hmm. The double hashing scheme was supposed to prevent that attack,
but looking at the code I think maybe it got implemented incorrectly.
We should go back and look at the design discussions to see if this
wasn't foreseen.

regards, tom lane

Tom Lane wrote:
> "murphy pope" <pope_murphy(at)hotmail(dot)com> writes:
> > But, if can peek at the server's user/password checksum (in the pg_pwd
> > file), I can connect to a server, get the server's salt, and combine it with
> > the stolen checksum, arriving at the checksum expected by the server.
>
> Hmm. The double hashing scheme was supposed to prevent that attack,
> but looking at the code I think maybe it got implemented incorrectly.
> We should go back and look at the design discussions to see if this
> wasn't foreseen.

OK, let me explain how md5 works right now, and why it may not be
possible to fix the problem outlined above.

First, let's assume GUC 'password_encryption' is 'true'. (I just set
this to default to true for 7.3, as agreed to months ago on the list.)

OK, user is created and stored in pg_shadow. The user-supplied password
is MD5 encrypted and stored in pg_shadow.

OK, user logs in. The backend sends two salts to the client --- one is
the salt used to store that user's password in pg_shadow, the second is
a random salt (random for every connection request). The client
encrypts the user's password with the pg_shadow salt, then the random
salt, and sends it back to the server. The server takes the
md5-encrypted pg_shadow password, encrypts with the same random salt
used by the client, and compares that to what the clients sends.

Now, the complaint is that if someone sees the pg_shadow password, they
can replay after using the random salt in the client to log in as that
user. That is true. I think the only value to encrypting passwords
stored in pg_shadow is that they can't see the actual password, which
may be used in other places by that person. People have complained about
this case, so it is good we can prevent it.

However, why can't we prevent the playback? With /etc/passwd, if
someone sees the password entry in that file, it doesn't mean they can
log in as that user. Why can't we do that?

Think of /bin/login. It has a secure connection with the terminal
(kernel character device driver), so it sends the password in the clear.
We don't have a secure connection to the client, so we have to randomize
what they send us. This prevents playback by wire sniffers. With
/bin/login, they clearly don't prevent the playback vulnerability. If
you see the characters coming from the client to /bin/login, you can
replay it to log in yourself.

What would be great would be if we could store an MD5 of what the client
should send us, like /bin/login does, but because we use random salt
every time, I don't see any way to do this. What they send us is
randomized, so we can't store some encrypted version of it.

Am I missing something? I know there are smart people on this list who
can tell me if my hunch is correct.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

<snip>
> OK, user logs in. The backend sends two salts to the client
> --- one is
> the salt used to store that user's password in pg_shadow, the
> second is
> a random salt (random for every connection request). The client
> encrypts the user's password with the pg_shadow salt, then the random
> salt, and sends it back to the server. The server takes the
> md5-encrypted pg_shadow password, encrypts with the same random salt
> used by the client, and compares that to what the clients sends.
>
> Now, the complaint is that if someone sees the pg_shadow
> password, they
> can replay after using the random salt in the client to log in as that
> user. That is true. I think the only value to encrypting passwords
> stored in pg_shadow is that they can't see the actual password, which
> may be used in other places by that person. People have
> complained about
> this case, so it is good we can prevent it.

This means that the only way a password can be compromized is to see the
password hash in pg_shadow, and the only thing it can be used for is
logging into postgres throught playback. No plain-text password can be
obtained. The only way to see the hash would be as the postgres user
which normally has no password (out of the box anyway), and therefore as
root first, in which case none of this much matters anyway.

The random salt prevents a packet sniffer from doing any good unless the
random salt is re-produced later and a hacker is lucky enough to have
gotten it on the connection he wants to crack. Possible, but... very
unlikely, unless a pattern can be established in the pseudo random salts
sent to the client. I doubt that's very likely either.

Short of using SSL to encrypt the entire session (passwords and all), I
can't think of a way to make it any stronger. It sounds pretty safe to
me.

-- Glen Parker
glenebob(at)nwlink(dot)com

> However, why can't we prevent the playback? With /etc/passwd, if
> someone sees the password entry in that file, it doesn't mean they can
> log in as that user. Why can't we do that?
>
> Think of /bin/login. It has a secure connection with the terminal
> (kernel character device driver), so it sends the password in
> the clear.
> We don't have a secure connection to the client, so we have
> to randomize
> what they send us. This prevents playback by wire sniffers. With
> /bin/login, they clearly don't prevent the playback vulnerability. If
> you see the characters coming from the client to /bin/login, you can
> replay it to log in yourself.
>
> What would be great would be if we could store an MD5 of what
> the client
> should send us, like /bin/login does, but because we use random salt
> every time, I don't see any way to do this. What they send us is
> randomized, so we can't store some encrypted version of it.
>
> Am I missing something? I know there are smart people on
> this list who
> can tell me if my hunch is correct.

"Glen Parker" <glenebob(at)nwlink(dot)com> writes:
> This means that the only way a password can be compromized is to see the
> password hash in pg_shadow, and the only thing it can be used for is
> logging into postgres throught playback. No plain-text password can be
> obtained.

Good point. If you can read the pg_shadow password entries, then you
are already superuser w.r.t. Postgres, so what need have you to break
into any other database user identities?

I think the concern that was originally advanced about all this was
that the database admin should not be able to read the clear-text
passwords of his users, because far too many people use the same
password for different purposes, and so a DBA might learn how to
get into accounts he shouldn't have access to. The MD5 scheme does
fix that problem (unless the other account chances to use the very
same MD5 hash scheme we do, which seems unlikely).

Still, we had a *long* discussion about the design of the MD5 password
scheme about a year ago, and I thought we'd come up with a design that
was proof against this sort of attack. Need to go back and consult
the list archives...

regards, tom lane

Tom Lane wrote:
> "Glen Parker" <glenebob(at)nwlink(dot)com> writes:
> > This means that the only way a password can be compromized is to see the
> > password hash in pg_shadow, and the only thing it can be used for is
> > logging into postgres throught playback. No plain-text password can be
> > obtained.
>
> Good point. If you can read the pg_shadow password entries, then you
> are already superuser w.r.t. Postgres, so what need have you to break
> into any other database user identities?
>
> I think the concern that was originally advanced about all this was
> that the database admin should not be able to read the clear-text
> passwords of his users, because far too many people use the same
> password for different purposes, and so a DBA might learn how to
> get into accounts he shouldn't have access to. The MD5 scheme does
> fix that problem (unless the other account chances to use the very
> same MD5 hash scheme we do, which seems unlikely).

Yes, I was unclear on the pg_shadow salt. The salt used in that case is
the username, so it doesn't need to be passed to the client. The client
already knows the username.

If other apps use the same encoding scheme, someone could match and know
that the user was using the same password for both applications, but it
doesn't help them know the password or break into the other app.

> Still, we had a *long* discussion about the design of the MD5 password
> scheme about a year ago, and I thought we'd come up with a design that
> was proof against this sort of attack. Need to go back and consult
> the list archives...

Yes, I had thought that seeing pg_shadow was as secure as seeing
/etc/passwd, but the user correctly points out that this is false.
Seeing pg_shadow is a much bigger problem. However, our pg_shadow is
not as visible as /etc/passwd (at least before there was a /etc/shadow).

Anyway, I wish we could improve it, but am an resigned to the fact we
can't.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Anyway, I wish we could improve it, but am an resigned to the fact we
> can't.

On first glance it seems obvious that *no* scheme could be proof against
the proposed attack. By hypothesis, the attacker has gotten a look at
the contents of pg_shadow --- therefore, he knows everything the
postmaster does about the user's authentication secret(s). How can the
postmaster pose a challenge that the attacker cannot answer, if the
attacker knows just as much as the postmaster?

It could be done if the postmaster's challenge were of the form "send me
something that *hashes to* the secret I have on disk", rather than the
current implementation's "here's a salt, hash the secret with it and
send it back". But as far as I can see, that means sending a cleartext
password across the wire, which is a cure considerably worse than this
disease. I don't see any way to combine that idea with a one-time
hash for wire-security purposes.

The equivalent attack against /etc/passwd would work just as well,
if the attacker could use a version of /bin/login that accepted the
already-crypted password instead of a cleartext password. /bin/login
avoids this difficulty by insisting on a cleartext password; but instead
it opens itself to wire-sniffing.

Interesting thought: perhaps the most secure combination would be MD5
passwords on disk, SSL connection encryption to guard against
wire-sniffing, and a *cleartext* password challenge. Then the attacker
actually has to prove he knows the password, and not just what's on
disk.

The DBA can easily set up pg_hba.conf to require SSL connections and
cleartext password auth. Do we have a setting that allows him to
enforce that all stored passwords must be MD5-crypted? I forget.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Anyway, I wish we could improve it, but am an resigned to the fact we
> > can't.
>
> On first glance it seems obvious that *no* scheme could be proof against
> the proposed attack. By hypothesis, the attacker has gotten a look at
> the contents of pg_shadow --- therefore, he knows everything the
> postmaster does about the user's authentication secret(s). How can the
> postmaster pose a challenge that the attacker cannot answer, if the
> attacker knows just as much as the postmaster?
>
> It could be done if the postmaster's challenge were of the form "send me
> something that *hashes to* the secret I have on disk", rather than the

Yes, that would be the trick. Have the client send them something that
when MD5 hashed matches pg_shadow. But when we randomize the salt, we
lose the ability.

> current implementation's "here's a salt, hash the secret with it and
> send it back". But as far as I can see, that means sending a cleartext
> password across the wire, which is a cure considerably worse than this
> disease. I don't see any way to combine that idea with a one-time
> hash for wire-security purposes.

It wouldn't have to be cleartext. You could do some MD5 hashing as we
do now, except send a fixed salt to the client every time. (Fixed for
that user.) Then, they can send a single-MD5 hashed version of the
password, the backend md5 hashes it again with a fixed salt and compares
to pg_shadow. That way, if they see pg_shadow, they can't just play it
back from the client. They would have to see what the client sent, and
then we are back to the need for random salt to prevent playback by
seeing the client communications.

Once you randomize, the backend can't store an encrypted version of the
secret. The way we do it now, we encrypt with a fixed salt in
pg_shadow, which works fine. I can't see any way to improve this in a
way that the client couldn't foil by knowing pg_shadow _and_ keeping
that random step in there.

> The equivalent attack against /etc/passwd would work just as well,
> if the attacker could use a version of /bin/login that accepted the
> already-crypted password instead of a cleartext password. /bin/login
> avoids this difficulty by insisting on a cleartext password; but instead
> it opens itself to wire-sniffing.

> Interesting thought: perhaps the most secure combination would be MD5
> passwords on disk, SSL connection encryption to guard against
> wire-sniffing, and a *cleartext* password challenge. Then the attacker
> actually has to prove he knows the password, and not just what's on
> disk.

> The DBA can easily set up pg_hba.conf to require SSL connections and
> cleartext password auth. Do we have a setting that allows him to
> enforce that all stored passwords must be MD5-crypted? I forget.

Wow, interesting thought. Actually, we do already support this. When
you specify 'md5' in pg_hba.conf, you expect the client to double-md5
encrypt the user-supplied password, and this is where knowing pg_shadow
helps an attacker. All the other authentication methods, including
plaintext password and ssl, expect the password in the clear, and the
backend encrypts and compares to pg_shadow, so this prevents an attacker
who knows pg_shadow from getting in. (Again, this assumes GUC
encrypted_passwords, which it will be in 7.3)

(Wow, I hope I don't get lots of these "it takes 10 minutes for me to
think of an answer" type of questions at my administration talk at
O'Reilly. I will have to give a "see me after the talk" answers.)

I wonder if people using SSL should be encouraged to use 'password'
rather than 'md5' in 7.3? I am sure some admins use SSL and md5
thinking it is more secure, when it is less.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I wonder if people using SSL should be encouraged to use 'password'
> rather than 'md5' in 7.3? I am sure some admins use SSL and md5
> thinking it is more secure, when it is less.

Depends on what your notion of security is, I guess. After sending my
earlier message, I had another thought: from the point of view of an
honest user, the cleartext password scheme is less secure, because a
dishonest DBA could easily tweak the postmaster to log submitted
passwords --- and thereby capture a user password that he shouldn't
have. In the MD5 scheme, the user need only trust his client-side
software to be sure that his original password is never exposed to
anyone, including the DBA.

Of course a paranoid user won't be using the same password for two
different purposes anyway ;-).

But anyway, this line of thinking suggests that we shouldn't be in a
hurry to rip out the cleartext-password auth method; it does have
some virtues.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I wonder if people using SSL should be encouraged to use 'password'
> > rather than 'md5' in 7.3? I am sure some admins use SSL and md5
> > thinking it is more secure, when it is less.
>
> Depends on what your notion of security is, I guess. After sending my
> earlier message, I had another thought: from the point of view of an
> honest user, the cleartext password scheme is less secure, because a
> dishonest DBA could easily tweak the postmaster to log submitted
> passwords --- and thereby capture a user password that he shouldn't
> have. In the MD5 scheme, the user need only trust his client-side
> software to be sure that his original password is never exposed to
> anyone, including the DBA.

I see. Good point.

> Of course a paranoid user won't be using the same password for two
> different purposes anyway ;-).
>
> But anyway, this line of thinking suggests that we shouldn't be in a
> hurry to rip out the cleartext-password auth method; it does have
> some virtues.

Yes. Not sure how to even document it. Seems pretty complicated.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

It occurs to me that we could make this work if we had a hash algorithm
that was commutative, in the sense that

hash(hash(a, b), c) = hash(hash(a, c), b)

for all possible passwords a and salts b, c. Then the idea
would be:

1. The value stored in pg_shadow is secret = hash(password, username)
same as now (or we could use some random salt, but we'd have to store
that salt too, so the username is probably as good as anything).

2. During connection start, pick a random salt and send it to the
client. The client computes response = hash(password, salt) and
sends it to the postmaster. Then the postmaster computes
hash(response, username) and hash(secret, salt) and compares these.
Commutativity would ensure that the values come out equal if the correct
password is supplied.

An attacker could figure out the value hash(secret, salt) if he'd seen
pg_shadow --- but if the hash function is strong then this does him no
good, because he won't be able to compute a response that will hash to
that target value.

MD5 is not commutative in this sense, and it might be that any hash
algorithm that is could not be cryptographically strong. But we could
look around and see what's out there...

regards, tom lane

Yes, I assumed secure hash commutativity was not possible. If it were,
we clearly could reorder things.

In fact, such commutativity seems impossible because a hash is an
abbreviated version of the original value.

---------------------------------------------------------------------------

Tom Lane wrote:
> It occurs to me that we could make this work if we had a hash algorithm
> that was commutative, in the sense that
>
> hash(hash(a, b), c) = hash(hash(a, c), b)
>
> for all possible passwords a and salts b, c. Then the idea
> would be:
>
> 1. The value stored in pg_shadow is secret = hash(password, username)
> same as now (or we could use some random salt, but we'd have to store
> that salt too, so the username is probably as good as anything).
>
> 2. During connection start, pick a random salt and send it to the
> client. The client computes response = hash(password, salt) and
> sends it to the postmaster. Then the postmaster computes
> hash(response, username) and hash(secret, salt) and compares these.
> Commutativity would ensure that the values come out equal if the correct
> password is supplied.
>
> An attacker could figure out the value hash(secret, salt) if he'd seen
> pg_shadow --- but if the hash function is strong then this does him no
> good, because he won't be able to compute a response that will hash to
> that target value.
>
> MD5 is not commutative in this sense, and it might be that any hash
> algorithm that is could not be cryptographically strong. But we could
> look around and see what's out there...
>
> regards, tom lane
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Tom Lane dijo:

> It occurs to me that we could make this work if we had a hash algorithm
> that was commutative, in the sense that

[...]

> MD5 is not commutative in this sense, and it might be that any hash
> algorithm that is could not be cryptographically strong. But we could
> look around and see what's out there...

Here http://www.research.att.com/~smb/papers/aeke.pdf the authors
describe something like the scheme you are looking for. They even talk
about "commutative hash functions" and how they help to protect against
"dictionary attacks and password file compromise". However, they
mention that "at present, we do not know of any family of commutative
one-way functions that satisfy the protocol requirements, while hiding
sufficient information".

They also talk about asymmetric encryption and describe a protocol for
key exchange using hashed passwords and some kind of public/private key
pair.

Maybe the paper sheds some light on the discussion.

--
Alvaro Herrera (<alvherre[a]atentus.com>)
"La conclusion que podemos sacar de esos estudios es que
no podemos sacar ninguna conclusion de ellos" (Tanenbaum)

Tom Lane wrote:
>
> MD5 is not commutative in this sense, and it might be that any hash
> algorithm that is could not be cryptographically strong. But we could
> look around and see what's out there...
>

I've never come across a commutative hash, but what you describe sounds
similar to the way asymmetric (i.e. public key) algorithms work. But
even with SSL, etc, the server certificate is normally encrypted on disk
to prevent compromise in the event someone gains local access. The
problem then becomes one of bootstrapping the system -- SSL web servers
with encrypted certificates need a warm body around when they boot to
type in the password used to encrypt the certificate.

My take is that if we're really concerned with the scenario described,
we'd need something similar. Have the dba type a password when the
postmaster is started, use that password to decrypt the stored encrypted
hashed users' passwords. Everything else stays pretty much as is. The
question remains as to whether it is really worth all the trouble,
especially since the server needs to be compromised in the first place
before the current behavior is a problem.

(Except, as I've mentioned before, security gurus would prefer we use
HMAC instead of a simple hash for authentication -- see:
http://www-cse.ucsd.edu/users/mihir/papers/kmd5.pdf
and
http://www.faqs.org/rfcs/rfc2104.html
if you're curious. It basically involves using a hash like md5 or sha1
in a specific way. I've implemented this before and would be happy to
help if there is interest. I think there is even an HMAC function in
contrib/pgcrypto now.)

Joe