Re: Proposal: BSD Authentication support

Hi,

I would like to add support for BSD authentication, used on OpenBSD. BSD authentication works similarly to PAM and allows authentication using local accounts.

We use OpenBSD on our Postgres servers at work.

I have attached an initial implementation of the proposal.

Cheers,
Marisa Emerson

This sounds like a sensible thing to me. I'm actually surprised, it
sounds like something we would have already seen. Do some people just
use PAM on OpenBSD? Are both supported?

You should add the patch to https://commitfest.postgresql.org to
ensure it doesn't slip through the cracks. It's too late for January
though there's nothing stopping people from commenting on or even
committing patches outside the commitfest.

There's a port for PAM, but we would prefer to use BSD Auth as its quite
a lot cleaner and is standard on OpenBSD.

I've attached an updated patch that includes documentation. It has been
tested against OpenBSD 5.8. I'll add this thread to the commitfest.

On 07/01/16 23:26, Greg Stark wrote:
> This sounds like a sensible thing to me. I'm actually surprised, it
> sounds like something we would have already seen. Do some people just
> use PAM on OpenBSD? Are both supported?
>
> You should add the patch to https://commitfest.postgresql.org to
> ensure it doesn't slip through the cracks. It's too late for January
> though there's nothing stopping people from commenting on or even
> committing patches outside the commitfest.
>

I've attached the latest version of this patch. I've fixed up an issue
with the configuration scripts that I missed.

On 08/01/16 12:40, Marisa Emerson wrote:
> There's a port for PAM, but we would prefer to use BSD Auth as its quite
> a lot cleaner and is standard on OpenBSD.
>
> I've attached an updated patch that includes documentation. It has been
> tested against OpenBSD 5.8. I'll add this thread to the commitfest.
>
> On 07/01/16 23:26, Greg Stark wrote:
>> This sounds like a sensible thing to me. I'm actually surprised, it
>> sounds like something we would have already seen. Do some people just
>> use PAM on OpenBSD? Are both supported?
>>
>> You should add the patch to https://commitfest.postgresql.org to
>> ensure it doesn't slip through the cracks. It's too late for January
>> though there's nothing stopping people from commenting on or even
>> committing patches outside the commitfest.
>>

On Tue, Jan 12, 2016 at 2:27 AM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
> I've attached the latest version of this patch. I've fixed up an issue with
> the configuration scripts that I missed.

Looks reasonable on a quick read-through. Can anyone with access to a
BSD system review and test?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Forgive my late comment ... I haven't used the PAM support in postgresql
either, or I'd know. PAM (I know for sure), and I suppose similarly BSD
Authentication, models a generalized auth interaction where a given
authentication module can send a number of arbitrary prompts back to the
client (via callbacks so different protocols and UIs can be used), and
demand a number of arbitrary responses, so that a variety of authentication
schemes can easily be supported.

Is the PostgreSQL support (for either PAM or BSD Authentication) able to
handle that in its designed generality, or only for the case (number of
requested items = 1, item 1 = a password)?

Could the general form be handled with the existing fe/be protocol,
or would the protocol have to grow?

-Chap

On Thu, Jan 14, 2016 at 11:59 PM, Chapman Flack <chap(at)anastigmatix(dot)net> wrote:
> Forgive my late comment ... I haven't used the PAM support in postgresql
> either, or I'd know. PAM (I know for sure), and I suppose similarly BSD
> Authentication, models a generalized auth interaction where a given
> authentication module can send a number of arbitrary prompts back to the
> client (via callbacks so different protocols and UIs can be used), and
> demand a number of arbitrary responses, so that a variety of authentication
> schemes can easily be supported.
>
> Is the PostgreSQL support (for either PAM or BSD Authentication) able to
> handle that in its designed generality, or only for the case (number of
> requested items = 1, item 1 = a password)?
>
> Could the general form be handled with the existing fe/be protocol,
> or would the protocol have to grow?

We support something like this for GSS, but not for other
authentication methods. See:

http://www.postgresql.org/docs/current/static/protocol-flow.html

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 1/14/16 11:22 PM, Robert Haas wrote:
> On Tue, Jan 12, 2016 at 2:27 AM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>> I've attached the latest version of this patch. I've fixed up an issue with
>> the configuration scripts that I missed.
> Looks reasonable on a quick read-through. Can anyone with access to a
> BSD system review and test?

Is anyone with access to/experience with BSD able to review and test
this patch? Seems like it would make a great addition to 9.6.

Thanks,

--
-David
david(at)pgmasters(dot)net

On Sat, Mar 12, 2016 at 5:14 AM, David Steele <david(at)pgmasters(dot)net> wrote:
> On 1/14/16 11:22 PM, Robert Haas wrote:
>> On Tue, Jan 12, 2016 at 2:27 AM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>>> I've attached the latest version of this patch. I've fixed up an issue with
>>> the configuration scripts that I missed.
>> Looks reasonable on a quick read-through. Can anyone with access to a
>> BSD system review and test?
>
> Is anyone with access to/experience with BSD able to review and test
> this patch? Seems like it would make a great addition to 9.6.

It looks like this needs review from an OpenBSD user specifically.
FreeBSD and NetBSD use PAM instead of BSD auth.

--
Thomas Munro
http://www.enterprisedb.com

On 1/7/16 9:40 PM, Marisa Emerson wrote:
> There's a port for PAM, but we would prefer to use BSD Auth as its quite
> a lot cleaner and is standard on OpenBSD.
>
> I've attached an updated patch that includes documentation. It has been
> tested against OpenBSD 5.8. I'll add this thread to the commitfest.

(Not a BSD user, just reviewing the code.)

configure.in has "build with BSD support", which should be "build with
BSD Authentication support".

There should be some documentation of the new configure option in
installation.sgml.

The documentation in client-auth.sgml speaks of a postgresql user and an
auth group. Maybe that's clear to users of BSD, but I don't know
whether these are OS entities or groups that I need to create or what.

The auth_userokay() call hardcodes a "type" of "pg-auth". That seems
important and should probably be documented. Extrapolating from PAM, I
think that should perhaps be an option in pg_hba.conf.

On 3/11/16 4:38 PM, Thomas Munro wrote:
> It looks like this needs review from an OpenBSD user specifically.
> FreeBSD and NetBSD use PAM instead of BSD auth.

FreeBSD has man pages for this stuff, so maybe they also have it now.

On Sat, Mar 12, 2016 at 5:14 AM, David Steele <david(at)pgmasters(dot)net> wrote:
> On 1/14/16 11:22 PM, Robert Haas wrote:
>> On Tue, Jan 12, 2016 at 2:27 AM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>>> I've attached the latest version of this patch. I've fixed up an issue with
>>> the configuration scripts that I missed.
>> Looks reasonable on a quick read-through. Can anyone with access to a
>> BSD system review and test?
>
> Is anyone with access to/experience with BSD able to review and test
> this patch? Seems like it would make a great addition to 9.6.

(Disclaimer: I am not a regular OpenBSD user or a security expert.)
I tried this out on OpenBSD 5.8 and it works as described, using
default "passwd" style authentication.

+ <note>
+ <para>
+ To use BSD Authentication, the postgresql user must first be added
+ to the <literal>auth</literal> group.
+ </para>
+ </note>

Our usual wording is "the PostgreSQL user account". Perhaps we should
be more explicit about the fact that membership of this Unix group is
needed on *OpenBSD*, since other current or future BSD forks could
vary. I see that the specific reason this is needed on this OpenBSD
5.8 box is so that it can fork/exec the setuid login_XXX binaries that
live under /usr/libexec/auth.

auth_userokay is called with a type of "pg-auth". I noticed from
looking at man page and source of some other applications that the
convention is usually a hardcoded string like "auth-myserver",
"auth-sockd", "auth-ssh", "auth-doas", "auth-popa3d" etc. So perhaps
we should have "auth-postgresql" (or "auth-postgres" or "auth-pgsql")
here? And as Peter E already said, that string should probably be
documented: it looks a bit like it is useful for allowing the
available authentication styles to be restricted or defaulted
specifically for PostgreSQL in login.conf based on that string.
(Though when I tried to set that up, it seemed to ignore my
possibly-incorrectly-specified rule asking it to use "reject" so I may
have misunderstood.)

The style argument is hard coded as NULL, as I see is the case in some
other applications. From the man page: "If style is not NULL, it
specifies the desired style of authentication to be used. If it is
NULL then the default style for the user is used. In this case, name
may include the desired style by appending it to the user's name with
a single colon (‘:’) as a separator." I wonder if such
user-controllable styles are OK (though I guess would require username
mapping to strip them off if we do want that as a feature). I wonder
if it should be possible to provide the style argument that we pass to
auth_userokay explicitly in pg_hba.conf, so that the DBA could
explicitly say BSD auth with style=radius.

I also tested on a system with no BSD auth support and configure
behaved sensibly with and without the new option ('error: header file
<bsd_auth.h> is required for BSD Authentication support'). I tried
configuring BSD auth in pg_hba.conf on a system built without the new
feature and it behaved sensibly ('invalid authentication method "bsd":
not supported by this build').

--
Thomas Munro
http://www.enterprisedb.com

>Our usual wording is "the PostgreSQL user account". Perhaps we should
>be more explicit about the fact that membership of this Unix group is
>needed on *OpenBSD*, since other current or future BSD forks could
>vary. I see that the specific reason this is needed on this OpenBSD
>5.8 box is so that it can fork/exec the setuid login_XXX binaries that
>live under /usr/libexec/auth.

The BSD Authentication framework currently only exists on OpenBSD. I've added some explicit documentation that this mechanism is currently only supported on OpenBSD and I've tried to be a bit more explicit about the auth group as suggested by Peter.

>auth_userokay is called with a type of "pg-auth". I noticed from
>looking at man page and source of some other applications that the
>convention is usually a hardcoded string like "auth-myserver",
>"auth-sockd", "auth-ssh", "auth-doas", "auth-popa3d" etc. So perhaps
>we should have "auth-postgresql" (or "auth-postgres" or "auth-pgsql")
>here? And as Peter E already said, that string should probably be
>documented: it looks a bit like it is useful for allowing the
>available authentication styles to be restricted or defaulted
>specifically for PostgreSQL in login.conf based on that string.
>(Though when I tried to set that up, it seemed to ignore my
>possibly-incorrectly-specified rule asking it to use "reject" so I may
>have misunderstood.)

This is correct, although so far I've only tested using the default login class. The attached patch includes some more explicit documentation about this string.

>The style argument is hard coded as NULL, as I see is the case in some
>other applications. From the man page: "If style is not NULL, it
>specifies the desired style of authentication to be used. If it is
>NULL then the default style for the user is used. In this case, name
>may include the desired style by appending it to the user's name with
>a single colon (‘:’) as a separator." I wonder if such
>user-controllable styles are OK (though I guess would require username
>mapping to strip them off if we do want that as a feature). I wonder
>if it should be possible to provide the style argument that we pass to
>auth_userokay explicitly in pg_hba.conf, so that the DBA could
>explicitly say BSD auth with style=radius.

I've so far only tested passwd authentication. I'd be interested to test some of the other authentication styles, I think this would be a useful feature.

On Fri, Mar 18, 2016 at 2:58 AM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>
>>Our usual wording is "the PostgreSQL user account". Perhaps we should
>>be more explicit about the fact that membership of this Unix group is
>>needed on *OpenBSD*, since other current or future BSD forks could
>>vary. I see that the specific reason this is needed on this OpenBSD
>>5.8 box is so that it can fork/exec the setuid login_XXX binaries that
>>live under /usr/libexec/auth.
>
> The BSD Authentication framework currently only exists on OpenBSD. I've added some explicit documentation that this mechanism is currently only supported on OpenBSD and I've tried to be a bit more explicit about the auth group as suggested by Peter.
>
>>auth_userokay is called with a type of "pg-auth". I noticed from
>>looking at man page and source of some other applications that the
>>convention is usually a hardcoded string like "auth-myserver",
>>"auth-sockd", "auth-ssh", "auth-doas", "auth-popa3d" etc. So perhaps
>>we should have "auth-postgresql" (or "auth-postgres" or "auth-pgsql")
>>here? And as Peter E already said, that string should probably be
>>documented: it looks a bit like it is useful for allowing the
>>available authentication styles to be restricted or defaulted
>>specifically for PostgreSQL in login.conf based on that string.
>>(Though when I tried to set that up, it seemed to ignore my
>>possibly-incorrectly-specified rule asking it to use "reject" so I may
>>have misunderstood.)
>
> This is correct, although so far I've only tested using the default login class. The attached patch includes some more explicit documentation about this string.

You used one name in the docs and another in the code:

+ BSD Authentication on PostgreSQL uses the <literal>auth-postgres</literal>
+ login type and authenticates with the <literal>postgres</literal> login

+ retval = auth_userokay(user, NULL, "auth-postgresql", passwd);

>>The style argument is hard coded as NULL, as I see is the case in some
>>other applications. From the man page: "If style is not NULL, it
>>specifies the desired style of authentication to be used. If it is
>>NULL then the default style for the user is used. In this case, name
>>may include the desired style by appending it to the user's name with
>>a single colon (‘:’) as a separator." I wonder if such
>>user-controllable styles are OK (though I guess would require username
>>mapping to strip them off if we do want that as a feature). I wonder
>>if it should be possible to provide the style argument that we pass to
>>auth_userokay explicitly in pg_hba.conf, so that the DBA could
>>explicitly say BSD auth with style=radius.
>
> I've so far only tested passwd authentication. I'd be interested to test some of the other authentication styles, I think this would be a useful feature.

Agreed.

It looks like this is still very useful with the default, and maybe
adding support for specifying the auth style in pg_hba.conf could be a
follow-up patch if anyone is interested in writing that.

Except for the string mismatch above I would personally say this is
ready for a committer to look at. Since Stas Kelvich also signed up
as a reviewer I'll give it a few days to see if he has feedback before
updating the commitfest status.

--
Thomas Munro
http://www.enterprisedb.com

On 18/03/16 03:57, Thomas Munro wrote:
> You used one name in the docs and another in the code:
>
> + BSD Authentication on PostgreSQL uses the <literal>auth-postgres</literal>
> + login type and authenticates with the <literal>postgres</literal> login
>
> + retval = auth_userokay(user, NULL, "auth-postgresql", passwd);

Woops, fix attached.

On Fri, Mar 18, 2016 at 12:49 PM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
> On 18/03/16 03:57, Thomas Munro wrote:
>>
>> You used one name in the docs and another in the code:
>>
>> + BSD Authentication on PostgreSQL uses the
>> <literal>auth-postgres</literal>
>> + login type and authenticates with the <literal>postgres</literal>
>> login
>>
>> + retval = auth_userokay(user, NULL, "auth-postgresql", passwd);
>
>
> Woops, fix attached.

Thanks!

I'm CCng Pierre-Emmanuel André, maintainer of the OpenBSD postgresql
port/package, just in case he has any feedback.

Pierre-Emmanuel, here's the full thread in case you missed it:
http://www.postgresql.org/message-id/1521c2f9465.e357a543197915.6912077634413325318@insec.sh

--
Thomas Munro
http://www.enterprisedb.com

On Fri, Mar 18, 2016 at 06:30:35PM +1300, Thomas Munro wrote:
> On Fri, Mar 18, 2016 at 12:49 PM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
> > On 18/03/16 03:57, Thomas Munro wrote:
> >>
> >> You used one name in the docs and another in the code:
> >>
> >> + BSD Authentication on PostgreSQL uses the
> >> <literal>auth-postgres</literal>
> >> + login type and authenticates with the <literal>postgres</literal>
> >> login
> >>
> >> + retval = auth_userokay(user, NULL, "auth-postgresql", passwd);
> >
> >
> > Woops, fix attached.
>
> Thanks!
>
> I'm CCng Pierre-Emmanuel André, maintainer of the OpenBSD postgresql
> port/package, just in case he has any feedback.
>
> Pierre-Emmanuel, here's the full thread in case you missed it:
> http://www.postgresql.org/message-id/1521c2f9465.e357a543197915.6912077634413325318@insec.sh
>

Hi,

Sorry for the late answer.
I've tested the patch on @amd64 with the latest PostgreSQL 9.6devel. I can't judge the diff
itself (it looks good but i'm not a PostgreSQL developper) but everything works fine.
It would be a nice feature to have on OpenBSD.

Regards,

On Fri, Apr 8, 2016 at 6:48 AM, Pierre-Emmanuel André <pea(at)openbsd(dot)org> wrote:
> On Fri, Mar 18, 2016 at 06:30:35PM +1300, Thomas Munro wrote:
>> On Fri, Mar 18, 2016 at 12:49 PM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>> > On 18/03/16 03:57, Thomas Munro wrote:
>> >>
>> >> You used one name in the docs and another in the code:
>> >>
>> >> + BSD Authentication on PostgreSQL uses the
>> >> <literal>auth-postgres</literal>
>> >> + login type and authenticates with the <literal>postgres</literal>
>> >> login
>> >>
>> >> + retval = auth_userokay(user, NULL, "auth-postgresql", passwd);
>> >
>> >
>> > Woops, fix attached.
>>
>> Thanks!
>>
>> I'm CCng Pierre-Emmanuel André, maintainer of the OpenBSD postgresql
>> port/package, just in case he has any feedback.
>>
>> Pierre-Emmanuel, here's the full thread in case you missed it:
>> http://www.postgresql.org/message-id/1521c2f9465.e357a543197915.6912077634413325318@insec.sh
>
> Sorry for the late answer.
> I've tested the patch on @amd64 with the latest PostgreSQL 9.6devel. I can't judge the diff
> itself (it looks good but i'm not a PostgreSQL developper) but everything works fine.
> It would be a nice feature to have on OpenBSD.

Opinion poll:

Should we commit this patch?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 4/8/16 11:20 AM, Robert Haas wrote:
> On Fri, Apr 8, 2016 at 6:48 AM, Pierre-Emmanuel André <pea(at)openbsd(dot)org> wrote:
>> On Fri, Mar 18, 2016 at 06:30:35PM +1300, Thomas Munro wrote:
>>> On Fri, Mar 18, 2016 at 12:49 PM, Marisa Emerson <mje(at)insec(dot)sh> wrote:
>>>> On 18/03/16 03:57, Thomas Munro wrote:
>>>>>
>>>>> You used one name in the docs and another in the code:
>>>>>
>>>>> + BSD Authentication on PostgreSQL uses the
>>>>> <literal>auth-postgres</literal>
>>>>> + login type and authenticates with the <literal>postgres</literal>
>>>>> login
>>>>>
>>>>> + retval = auth_userokay(user, NULL, "auth-postgresql", passwd);
>>>>
>>>>
>>>> Woops, fix attached.
>>>
>>> Thanks!
>>>
>>> I'm CCng Pierre-Emmanuel André, maintainer of the OpenBSD postgresql
>>> port/package, just in case he has any feedback.
>>>
>>> Pierre-Emmanuel, here's the full thread in case you missed it:
>>> http://www.postgresql.org/message-id/1521c2f9465.e357a543197915.6912077634413325318@insec.sh
>>
>> Sorry for the late answer.
>> I've tested the patch on @amd64 with the latest PostgreSQL 9.6devel. I can't judge the diff
>> itself (it looks good but i'm not a PostgreSQL developper) but everything works fine.
>> It would be a nice feature to have on OpenBSD.
>
> Opinion poll:
>
> Should we commit this patch?

To summarize:

Robert Haas and Peter Eisentraut have done code-only reviews. Thomas
Munro has reviewed and tested with a caveat that he is no BSD expert.
Pierre-Emmanuel tested but did not do a code review review due to his
unfamiliarity with the Postgres code-base.

The patch does not seem invasive or likely to cause problems on non-BSD
systems. From my perspective this has gotten about as much review as it
can, so +1 from me at least.

--
-David
david(at)pgmasters(dot)net

David Steele <david(at)pgmasters(dot)net> writes:
> On 4/8/16 11:20 AM, Robert Haas wrote:
>> Should we commit this patch?

> To summarize:

> Robert Haas and Peter Eisentraut have done code-only reviews. Thomas
> Munro has reviewed and tested with a caveat that he is no BSD expert.
> Pierre-Emmanuel tested but did not do a code review review due to his
> unfamiliarity with the Postgres code-base.

> The patch does not seem invasive or likely to cause problems on non-BSD
> systems. From my perspective this has gotten about as much review as it
> can, so +1 from me at least.

Given our lack of BSD developers, there's probably not much more we can
do, but one more set of eyeballs on the patch wouldn't hurt. I'll take
it, unless somebody else wants to.

regards, tom lane

Marisa Emerson <mje(at)insec(dot)sh> writes:
> Woops, fix attached.

Pushed with minor adjustments.

regards, tom lane