Re: [HACKERS] Krb5 & multiple DB connections

Greetings,

I've just run smack-dab into the bug described here:

http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00083.php

and it's somewhat frustrating since the end of that thread is a
reasonably small patch which fixes the problem:

http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00083.php

Or did for that user anyway but it sure looks like it's solve my
problem as well (which is also a case of using mod_auth_krb5, etc,
though it breaks even for the same user since apache now, correctly,
deletes and recreates the cache with a different filename for each
connection).

I realize it's not entirely fair (given that it was years ago) to ask
this, but, anyone happen to know why the patch wasn't accepted? It
almost patched cleanly against current HEAD even. I went ahead and
made the few changes by hand that didn't apply cleanly and it compiled
just fine; attached patch is against current HEAD and should apply
cleanly.

I'd really love to get this fixed.

Thanks (off to go implement Tom's suggestion for pg_restore :)!

Stephen

* Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> and it's somewhat frustrating since the end of that thread is a
> reasonably small patch which fixes the problem:
>
> http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00083.php

Erp, not quite sure how I managed that, the end of the thread is *here*:

http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00090.php

Sorry about that.

Stephen

Greetings,

I am writing some plugin for pq. After running ./configure and make, shall I
just go to /src/bin/psql to execute my executable psql -U (usrname) dbname?
It seems that my updates in the file parse_expr.c is not reflect in this
executable? For example, I changed one line from

if (NULL == tup)
ereport(ERROR, (errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("key value not found")));

-->

if (NULL == tup)
ereport(ERROR, (errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("key value not found while building relation")));

But the program still reports key value not found when it runs.

Best wishes,
John

John wrote:
> I am writing some plugin for pq. After running ./configure and make,
> shall I just go to /src/bin/psql to execute my executable psql -U
> (usrname) dbname? It seems that my updates in the file parse_expr.c
> is not reflect in this executable?

Evidently you're not (only) writing a plugin but changing the source
code. In that case, follow the normal installation instructions.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Greetings,

* Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> I realize it's not entirely fair (given that it was years ago) to ask
> this, but, anyone happen to know why the patch wasn't accepted? It
> almost patched cleanly against current HEAD even. I went ahead and
> made the few changes by hand that didn't apply cleanly and it compiled
> just fine; attached patch is against current HEAD and should apply
> cleanly.

I've now tested this patch at home w/ 8.2HEAD and it seems to fix the
bug. I plan on testing it under 8.1.2 at work tommorow with
mod_auth_krb5, etc, and expect it'll work there. Assuming all goes
well and unless someone objects I'll forward the patch to -patches.
It'd be great to have this fixed as it'll allow us to use Kerberos to
authenticate to phppgadmin and other web-based tools which use
Postgres.

Thanks!

Stephen

Greetings,

* Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> I've now tested this patch at home w/ 8.2HEAD and it seems to fix the
> bug. I plan on testing it under 8.1.2 at work tommorow with
> mod_auth_krb5, etc, and expect it'll work there. Assuming all goes
> well and unless someone objects I'll forward the patch to -patches.
> It'd be great to have this fixed as it'll allow us to use Kerberos to
> authenticate to phppgadmin and other web-based tools which use
> Postgres.

While playing with this patch under 8.1.2 at home I discovered a
mistake in how I manually applied one of the hunks to fe-auth.c.
Basically, the base code had changed and so the patch needed to be
modified slightly. This is because the code no longer either has a
freeable pointer under 'name' or has 'name' as NULL.

The attached patch correctly frees the string from pg_krb5_authname
(where it had been strdup'd) if and only if pg_krb5_authname returned
a string (as opposed to falling through and having name be set using
name = pw->name;). Also added a comment to this effect.
Please review.

Thanks,

Stephen

Greetings,

The attached patch fixes a bug which was originally brought up in May
of 2002 in this thread:

http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00083.php

The original bug reporter also supplied a patch to fix the problem:

http://archives.postgresql.org/pgsql-interfaces/2002-05/msg00090.php

I ran into exactly the same issue using 8.1.2. I've updated the
original patch to work for HEAD and 8.1.2. I've tested the patch on
both HEAD and 8.1.2, both at home and at work, and it works quite
nicely. In fact, I hope to have a patch to phppgadmin which will make
it properly handle Kerberized logins.

Thanks,

Stephen

Stephen Frost <sfrost(at)snowman(dot)net> writes:
> The attached patch fixes a bug which was originally brought up in May
> of 2002 in this thread:

Now that I've looked at it, I find this patch seems fairly wrongheaded.
AFAICS the entire point of the original coding is to allow the setup
work needed to create the krb5_context etc to be amortized across
multiple connections. The patch destroys that advantage, but yet keeps
as much as it can of the notational cruft induced by the original
design -- for instance, there seems little point in the
pg_krb5_initialised flag if we aren't ever going to have any
pre-initialized state.

I have little idea of how expensive the operations called by
pg_krb5_init really are. If they are expensive then it'd probably
make sense to keep the current static variables but treat 'em as a
one-element cache, ie, recompute if a new user name is being demanded.
If not, we ought to be able to simplify some things.

Another point here is how all this interacts with thread safety.
If we get rid of the static variables, do we still need the
pglock_thread() operations?

regards, tom lane

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > The attached patch fixes a bug which was originally brought up in May
> > of 2002 in this thread:
>
> Now that I've looked at it, I find this patch seems fairly wrongheaded.
> AFAICS the entire point of the original coding is to allow the setup
> work needed to create the krb5_context etc to be amortized across
> multiple connections. The patch destroys that advantage, but yet keeps
> as much as it can of the notational cruft induced by the original
> design -- for instance, there seems little point in the
> pg_krb5_initialised flag if we aren't ever going to have any
> pre-initialized state.

I'm honestly not entirely sure I agree about that being the point of the
original coding but regardless the crux of the problem here is that
there's no way to get libpq to use a cache other than the one it's
initialized with for a given session. That part of the Kerberos API
which supports that isn't exposed in any way beyond the KRB5CCNAME
environment variable and the call to ask for the 'default' ccache is
called with the static variable the second time and it ignores the
request when there's an already valid (it thinks) ccache.

> I have little idea of how expensive the operations called by
> pg_krb5_init really are. If they are expensive then it'd probably
> make sense to keep the current static variables but treat 'em as a
> one-element cache, ie, recompute if a new user name is being demanded.
> If not, we ought to be able to simplify some things.

We'd have to recompute based on the KRB5CCNAME environment variable
changing, which is certainly an option. It's not necessairly the case
that the username is changing, possibly just the cache. Additionally,
the calls themselves are not very expensive when being called on an
existing cache, the most expensive thing is reaching out to the KDC to
get a new service ticket which will either need to be done, or won't,
depending on if a valid service ticket already exists in the cache or
not.

> Another point here is how all this interacts with thread safety.
> If we get rid of the static variables, do we still need the
> pglock_thread() operations?

Good question, I'm afraid probably not. I'd have to look through it
again but last I checked MIT Kerberos prior to 1.4 (and I'm not 100%
sure it's resolved in 1.4) wasn't threadsafe itself.

I'd certainly be happy to rework the patch based on these comments, of
course. Honestly, I'm pretty sure the original patch was intended to be
minimal (and is for the most part). These changes would introduce more
logic but if that's alright I'd be happy to do it.

Thanks,

Stephen

Stephen Frost <sfrost(at)snowman(dot)net> writes:
>> I have little idea of how expensive the operations called by
>> pg_krb5_init really are. If they are expensive then it'd probably
>> make sense to keep the current static variables but treat 'em as a
>> one-element cache, ie, recompute if a new user name is being demanded.
>> If not, we ought to be able to simplify some things.

> We'd have to recompute based on the KRB5CCNAME environment variable
> changing, which is certainly an option. It's not necessairly the case
> that the username is changing, possibly just the cache.

Hm, apparently I completely misunderstand the problem here. What I
thought the bug was was that the cache wasn't recomputed given an
attempt to connect as a different Postgres username than the first
time. If that's not the issue, then what is?

regards, tom lane

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> >> I have little idea of how expensive the operations called by
> >> pg_krb5_init really are. If they are expensive then it'd probably
> >> make sense to keep the current static variables but treat 'em as a
> >> one-element cache, ie, recompute if a new user name is being demanded.
> >> If not, we ought to be able to simplify some things.
>
> > We'd have to recompute based on the KRB5CCNAME environment variable
> > changing, which is certainly an option. It's not necessairly the case
> > that the username is changing, possibly just the cache.
>
> Hm, apparently I completely misunderstand the problem here. What I
> thought the bug was was that the cache wasn't recomputed given an
> attempt to connect as a different Postgres username than the first
> time. If that's not the issue, then what is?

The specific problem which I and the original reporter ran into is this:

KRB5CCACHE=/tmp/krb5cc_apache_aev0kF
pg_connect() -- works fine
pg_close() -- works fine
rm /tmp/krb5cc_apache_aev0kF
KRB5CCACHE=/tmp/krb5cc_apache_cVMRtA
pg_connect() -- Doesn't work, Kerberos error is "no credentials cache"

What's happening here is that for every connection to apache by the
client a new credentials cache is created, and then destroyed when the
connection closes. When using PHP (ie: phppgadmin) and mod_php (as is
common) the Apache process is the one actually making the connection to
the database and a given Apache process usually serves multiple requests
in its lifetime, sometimes to the same user, sometimes to different
users.

The static variables being used are for: krb5_init_context,
krb5_cc_default, krb5_cc_get_principal, and krb5_unparse_name.
Technically, between one connection and the next, krb5_cc_default,
krb5_cc_get_principal and krb5_unparse_name could reasonably return
different values. krb5_init_context is pretty unlikely to change as
that would mean /etc/krb5.conf changed. Looking through the krb5 source
code it appears that the only one which checks for something existing
is krb5_cc_default_name (called by krb5_cc_default), which will just
return the current ccache name if one has been set.
(src/lib/krb5/os/ccdefname.c:278, krb5-1.4.3)

We initially brought up this issue with the Kerberos folks actually:
http://pch.mit.edu/pipermail/kerberos/2006-February/009225.html

They pretty clearly felt that the application was responsible for
handling the cache in the event it changes. Unfortunately, it's not the
application which is talking to Kerberos but another library in this
case which doesn't expose the Kerberos API to the application in such a
way to allow the application to notify Kerberos of the cache change.

Looking through the Kerberos API again it looks like it might be
possible to use krb5_cc_set_default_name(context, NULL) to force
krb5_cc_default_name() (from krb5_cc_default()) to re-find the cache.
Finding the cache again is reasonably inexpensive. I've tried a couple
of things along these lines now but I havn't found a workable solution
which doesn't reinitialize the main Kerberos context, unfortunately.
I'll keep working on it as time allows though honestly I don't believe
it's generally terribly expensive to reinitialize the context...

Sorry it took so long to reply, running down the paths through the
various libraries takes a bit of time and I was really hoping to be able
to suggest an alternative solution using krb5_cc_set_default_name.

Thanks,

Stephen

Stephen Frost <sfrost(at)snowman(dot)net> writes:
> The specific problem which I and the original reporter ran into is this:

> KRB5CCACHE=/tmp/krb5cc_apache_aev0kF
> pg_connect() -- works fine
> pg_close() -- works fine
> rm /tmp/krb5cc_apache_aev0kF
> KRB5CCACHE=/tmp/krb5cc_apache_cVMRtA
> pg_connect() -- Doesn't work, Kerberos error is "no credentials cache"

And why exactly is the application trying to munge the cache like that?
(I assume there is some state change you're not bothering to mention,
or there would be no need for a new cache, no?)

It sure seems like somebody is solving the wrong problem, and doing it
with a sledgehammer...

regards, tom lane

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > The specific problem which I and the original reporter ran into is this:
>
> > KRB5CCACHE=/tmp/krb5cc_apache_aev0kF
> > pg_connect() -- works fine
> > pg_close() -- works fine
> > rm /tmp/krb5cc_apache_aev0kF
> > KRB5CCACHE=/tmp/krb5cc_apache_cVMRtA
> > pg_connect() -- Doesn't work, Kerberos error is "no credentials cache"
>
> And why exactly is the application trying to munge the cache like that?
> (I assume there is some state change you're not bothering to mention,
> or there would be no need for a new cache, no?)

Thought I mentioned it further down- each new Apache TCP connection gets
a new ccache file because it goes through the Negotiate protocol again
and gets a new TGT on the web server.

It works the exact same way as SSH, really. When you connect to a
remote server using SSH it will store the TGT (assuming you have
delegate_credentials enabled) in a random file in /tmp. It's not an
issue with SSH though because you don't tend to have applications which
continue running across multiple SSH sessions and which use the TGT to
reconnect to other services multiple times.

Actually, thinking about this a minute longer, it can happen w/
SSH/screen and psql. Here's an example:

----------------------
sfrost(at)snowman:/data/sfrost/postgres/testinstall.krb5> kinit
Password for sfrost(at)SNOWMAN(dot)NET:
sfrost(at)snowman:/data/sfrost/postgres/testinstall.krb5> ssh snowman
sfrost(at)snowman:/home/sfrost> screen
sfrost(at)snowman:/home/sfrost> psql -d template1 -h snowman
Welcome to psql 8.1.2, the PostgreSQL interactive terminal.
[...]
Disconnect from screen, log out, reconnect, reconnect to screen
[...]
template1=> \connect template1
pg_krb5_init: krb5_cc_get_principal: No credentials cache found
----------------------

There's not really a solution to this issue (besides exit psql, reset
the environment variable, and restart psql) unless we allow the ccache
to be set in psql and then be propogated down to libpq. I actually
hacked up some .profile/.logout scripts to avoid this problem by forcing
the cache to a fixed known-spot under my home directory. I don't think
that really works for this though because everything is running as one
uid under Apache and having a fixed long-term filename would present
some additional security issues.

> It sure seems like somebody is solving the wrong problem, and doing it
> with a sledgehammer...

The sledgehammer approach would be to require people to run everything
as CGIs. This does work, of course, but is rather painful on a number
of levels...

Thanks,

Stephen

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------

Stephen Frost wrote:
> Greetings,
>
> * Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> > I've now tested this patch at home w/ 8.2HEAD and it seems to fix the
> > bug. I plan on testing it under 8.1.2 at work tommorow with
> > mod_auth_krb5, etc, and expect it'll work there. Assuming all goes
> > well and unless someone objects I'll forward the patch to -patches.
> > It'd be great to have this fixed as it'll allow us to use Kerberos to
> > authenticate to phppgadmin and other web-based tools which use
> > Postgres.
>
> While playing with this patch under 8.1.2 at home I discovered a
> mistake in how I manually applied one of the hunks to fe-auth.c.
> Basically, the base code had changed and so the patch needed to be
> modified slightly. This is because the code no longer either has a
> freeable pointer under 'name' or has 'name' as NULL.
>
> The attached patch correctly frees the string from pg_krb5_authname
> (where it had been strdup'd) if and only if pg_krb5_authname returned
> a string (as opposed to falling through and having name be set using
> name = pw->name;). Also added a comment to this effect.
> Please review.
>
> Thanks,
>
> Stephen

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Is this something we need for 8.1.X?

---------------------------------------------------------------------------

Bruce Momjian wrote:
>
> Your patch has been added to the PostgreSQL unapplied patches list at:
>
> http://momjian.postgresql.org/cgi-bin/pgpatches
>
> It will be applied as soon as one of the PostgreSQL committers reviews
> and approves it.
>
> ---------------------------------------------------------------------------
>
>
> Stephen Frost wrote:
> > Greetings,
> >
> > * Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> > > I've now tested this patch at home w/ 8.2HEAD and it seems to fix the
> > > bug. I plan on testing it under 8.1.2 at work tommorow with
> > > mod_auth_krb5, etc, and expect it'll work there. Assuming all goes
> > > well and unless someone objects I'll forward the patch to -patches.
> > > It'd be great to have this fixed as it'll allow us to use Kerberos to
> > > authenticate to phppgadmin and other web-based tools which use
> > > Postgres.
> >
> > While playing with this patch under 8.1.2 at home I discovered a
> > mistake in how I manually applied one of the hunks to fe-auth.c.
> > Basically, the base code had changed and so the patch needed to be
> > modified slightly. This is because the code no longer either has a
> > freeable pointer under 'name' or has 'name' as NULL.
> >
> > The attached patch correctly frees the string from pg_krb5_authname
> > (where it had been strdup'd) if and only if pg_krb5_authname returned
> > a string (as opposed to falling through and having name be set using
> > name = pw->name;). Also added a comment to this effect.
> > Please review.
> >
> > Thanks,
> >
> > Stephen
>
> [ Attachment, skipping... ]
>
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 6: explain analyze is your friend
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

* Bruce Momjian (pgman(at)candle(dot)pha(dot)pa(dot)us) wrote:
> Is this something we need for 8.1.X?

Personally, I think it's a bug which should be fixed. I don't think
everyone agrees on that though and there are some parts which could be a
bit controversial. The main issue is that now the entire Kerberos
context is recreated on each connection. Now, it's really not all that
expensive generally but I can understand the preference to just reopen
the ccache when it changes. I tried to do that by just free'ing and
redo'ing the parts having to do w/ the ccache but it didn't quite pan
out right and I havn't figured out what I did wrong yet. The other
approach would be to actually track the cache name and check if it
changes. That's more difficult to do correctly across platforms though
I think.

Thanks,

Stephen

Then the patch safest for just 8.2 then.

---------------------------------------------------------------------------

Stephen Frost wrote:
-- Start of PGP signed section.
> * Bruce Momjian (pgman(at)candle(dot)pha(dot)pa(dot)us) wrote:
> > Is this something we need for 8.1.X?
>
> Personally, I think it's a bug which should be fixed. I don't think
> everyone agrees on that though and there are some parts which could be a
> bit controversial. The main issue is that now the entire Kerberos
> context is recreated on each connection. Now, it's really not all that
> expensive generally but I can understand the preference to just reopen
> the ccache when it changes. I tried to do that by just free'ing and
> redo'ing the parts having to do w/ the ccache but it didn't quite pan
> out right and I havn't figured out what I did wrong yet. The other
> approach would be to actually track the cache name and check if it
> changes. That's more difficult to do correctly across platforms though
> I think.
>
> Thanks,
>
> Stephen
-- End of PGP section, PGP failed!

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

* Bruce Momjian (pgman(at)candle(dot)pha(dot)pa(dot)us) wrote:
> Then the patch safest for just 8.2 then.

My hope is to come up with a better patch which will be acceptable for
both 8.1.x and 8.2.. I'll try and come up with something this week. I
don't think it's a huge issue if it's not in 8.1.3 tho.

Thanks,

Stephen

Patch applied. Thanks.

Backpatched to 8.1.X.

---------------------------------------------------------------------------

Stephen Frost wrote:
> Greetings,
>
> * Stephen Frost (sfrost(at)snowman(dot)net) wrote:
> > I've now tested this patch at home w/ 8.2HEAD and it seems to fix the
> > bug. I plan on testing it under 8.1.2 at work tommorow with
> > mod_auth_krb5, etc, and expect it'll work there. Assuming all goes
> > well and unless someone objects I'll forward the patch to -patches.
> > It'd be great to have this fixed as it'll allow us to use Kerberos to
> > authenticate to phppgadmin and other web-based tools which use
> > Postgres.
>
> While playing with this patch under 8.1.2 at home I discovered a
> mistake in how I manually applied one of the hunks to fe-auth.c.
> Basically, the base code had changed and so the patch needed to be
> modified slightly. This is because the code no longer either has a
> freeable pointer under 'name' or has 'name' as NULL.
>
> The attached patch correctly frees the string from pg_krb5_authname
> (where it had been strdup'd) if and only if pg_krb5_authname returned
> a string (as opposed to falling through and having name be set using
> name = pw->name;). Also added a comment to this effect.
> Please review.
>
> Thanks,
>
> Stephen

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +