Proposal for encrypting pg_shadow passwords

Here is my proposal to fix the security problem of storing cleartext
passwords in pg_shadow. The solution is somewhat complex because we
have to allow 7.2 servers to communicate with 7.1 clients, at least for
a short while.

Here is a summary of what we currently do and proposed solutions.

PG_HBA.CONF
-----------
pg_hba.conf has three authentication options of interest to this
discussion:

trust: no authentication required

password: plaintext password is sent over network from client
to server

crypt: random salt is sent to client; client encrypts using that salt
and returns encrypted password to server. Server encrypts pg_shadow
password with same random salt and compares. This is why current
pg_shadow password is cleartext. (Call this "crypt authentication".)

DOUBLE ENCRYPTION
-----------------
The solution for encrypting pg_shadow passwords is to encrypt using a
salt when stored in pg_shadow, and to generate a random salt for each
authentication request. Send _both_ salts to the client, let the client
double encrypt using the pg_shadow salt first, then the random salt, and
send it back. The server encrypt using only the random salt and
compares.

As soon as we encrypt pg_shadow passwords, we can't communicate with
pre-7.2 clients using crypt-authentication. Actually, we could, but we
would have to send the same pg_shadow salt every time, which is insecure
because someone snooping the wire could just play back the same reply so
it is better to just fail such authentications.

USER INTERFACE
--------------
So, my idea is to add an option to CREATE/ALTER USER:

CREATE USER WITH ENCRYPTED PASSWORD 'fred';
CREATE USER WITH UNENCRYPTED PASSWORD 'fred';
ALTER USER WITH ENCRYPTED PASSWORD 'fred';
ALTER USER WITH UNENCRYPTED PASSWORD 'fred';

Keep in mind ENCRYPTED/UNENCRYPTED controls how it is stored in
pg_shadow, not wither "fred" is a cleartext or preencrypted password.
We plan to prefix md5 passwords with "md5" to handle this issue. (Md5
passwords are also 35-characters in length.)

Also add a new GUC config option:

SET password_encrypted_default TO 'OFF';

It would ship as OFF in 7.2 and can be removed in a later release. Once
all clients are upgraded to 7.2, you can change the default to ON and do
ALTER USER WITH PASSWORD 'fred' to encrypt the pg_shadow passwords. The
passwords are in cleartext in pg_shadow so it is easy to do.

MD5
---
I assume we will use MD5 for encryption of pg_shadow passwords. The
letters "md5" will appear at the start of the password string and it
will be exactly 35 characters. Vince sent me the code. We will need to
add MD5 capability to libpq, ODBC, and JDBC. (I hope JDBC will not be a
problem.) When using CREATE/ALTER user, the system will automatically
consider a 35-character string that starts with "md5" to be a
pre-md5-encrypted password, while anything else will be md5 encrypted.

SECONDARY PASSWORD FILES
------------------------
To add complexity to this, we also support secondary password files.
(See pg_hba.conf.sample and pg_password manual in CVS for updated
descriptions.) These password files allow encrypted passwords in the
same format as they appear in traditional /etc/passwd. (Call this
crypt-style passwords.) I realize most BSD's use MD5 in /etc/shadow
now.

Right now we can use passwords from the file only if we use
password-authentication. We can't use crypt-authentication because the
passwords already have a salt and we don't want to sent the same salt
every time. One nice feature of secondary passwords is you can copy
/etc/passwd or /etc/shadow and use that as your secondary password file
for PostgreSQL. I don't know how many people use that but it is nice
feature. Remember the secondary password files sit in /data which is
readable only by the PostgreSQL install user.

DOUBLE-CRYPT ENCRYPTION
-----------------------
So, we are going to add a new double-MD5 encryption protocol to allow
pg_shadow passwords to be encrypted. Do we also add a
double-crypt-style-password protocol to allow crypt-authentication with
secondary password files that use crypt-style passwords or just require
the secondary password files to use MD5?

Comments?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Mon, Jun 25, 2001 at 11:04:15PM -0400, Bruce Momjian wrote:
> password: plaintext password is sent over network from client
> to server
>
> crypt: random salt is sent to client; client encrypts using that salt
> and returns encrypted password to server. Server encrypts pg_shadow
> password with same random salt and compares. This is why current
> pg_shadow password is cleartext. (Call this "crypt authentication".)

did you see my post of a week or so ago?

host dbname ipaddr netmask password /some/file
- uses second field of /some/file, as per /etc/passwd
- compares second field of /some/file with crypt(clear-text)

host dbname ipaddr netmask crypt (no file specified)
- as above

host dbname ipaddr netmask password (no file specified)
- same as if the line was s/password/crypt/g

i have mods that allow (in a completely backward compatible fashion)

host dbname ipaddr netmask password pg_shadow
- uses password from pg_shadow
- compares pg_shadow->password with crypt(clear-text)

while i applaud the dual-crypt enhancements for the newer versions,
i think these patches allow storage of encrypted passwords in pg_shadow
without any substantial changes (or possible damage to existing code).

i am using these mods in conjuction with php scripts, and as such i need
not give "webuser" or "nobody" any privs on my tables.

--
[ Jim Mercer jim(at)reptiles(dot)org +1 416 410-5633 ]
[ Now with more and longer words for your reading enjoyment. ]

On Mon, Jun 25, 2001 at 11:18:20PM -0400, Jim Mercer wrote:
> host dbname ipaddr netmask crypt (no file specified)
> - as above
(meaning bruce's description)

--
[ Jim Mercer jim(at)reptiles(dot)org +1 416 410-5633 ]
[ Now with more and longer words for your reading enjoyment. ]

On Monday, June 25, 2001, at 08:04 PM, Bruce Momjian wrote:

> I assume we will use MD5 for encryption of pg_shadow passwords. The
> letters "md5" will appear at the start of the password string and it
> will be exactly 35 characters. Vince sent me the code. We
> will need to
> add MD5 capability to libpq, ODBC, and JDBC. (I hope JDBC will
> not be a
> problem.)

JDK 1.1 and later has MD5 available; this shouldn't be a problem.

-- Bruce

--------------------------------------------------------------------------
Bruce Toback Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc. (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142 | But ah, my foes, and oh, my
friends -
Phoenix AZ 85028 | It gives a lovely light.
btoback(at)optc(dot)com | -- Edna St. Vincent Millay

> DOUBLE ENCRYPTION
> -----------------
> The solution for encrypting pg_shadow passwords is to encrypt using a
> salt when stored in pg_shadow, and to generate a random salt for each
> authentication request. Send _both_ salts to the client, let the client
> double encrypt using the pg_shadow salt first, then the random salt, and
> send it back. The server encrypt using only the random salt and
> compares.
>

I posted something on this a few weeks ago. See
http://fts.postgresql.org/db/mw/msg.html?mid=1021155 for details, but the
summary is that it would be better (IMHO) to use HMAC for authentication.
HMAC has
been mathematically proven to be as secure as the underlying hash algorithm
used.
Here's the reference for HMAC --
http://www-cse.ucsd.edu/users/mihir/papers/kmd5.pdf.

It would actually work almost identically to what you've described. Store
the password as a hash using MD5 and some salt. Send the password salt and a
random salt to the client. The client uses the password salt with MD5 (and
local knowledge of the plaintext password) to reproduce the stored password,
then calculates an HMAC of the random salt and sends it back. The server
also calculates the HMAC of the random salt using the stored hashed
password, and compares.

Just my 2 cents . . .

-- Joe

> > DOUBLE ENCRYPTION
> > -----------------
> > The solution for encrypting pg_shadow passwords is to encrypt using a
> > salt when stored in pg_shadow, and to generate a random salt for each
> > authentication request. Send _both_ salts to the client, let the client
> > double encrypt using the pg_shadow salt first, then the random salt, and
> > send it back. The server encrypt using only the random salt and
> > compares.
> >
>
> I posted something on this a few weeks ago. See
> http://fts.postgresql.org/db/mw/msg.html?mid=1021155 for details, but the
> summary is that it would be better (IMHO) to use HMAC for authentication.
> HMAC has
> been mathematically proven to be as secure as the underlying hash algorithm
> used.
> Here's the reference for HMAC --
> http://www-cse.ucsd.edu/users/mihir/papers/kmd5.pdf.
>
> It would actually work almost identically to what you've described. Store
> the password as a hash using MD5 and some salt. Send the password salt and a
> random salt to the client. The client uses the password salt with MD5 (and
> local knowledge of the plaintext password) to reproduce the stored password,
> then calculates an HMAC of the random salt and sends it back. The server
> also calculates the HMAC of the random salt using the stored hashed
> password, and compares.

Yes, I remember that. I figured MD5 was standard and secure enough for
our purposes. Newer stuff sometimes has problems because it has not
been tested long enough and I would hate to change this if a problem is
found.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026