Re: pg_basebackup: could not get transaction log end position from server: FATAL: could not open file "./pg_hba.conf~": Permission denied

Hello,

Can we get that fixed please? It seems rather bad behavior for
pg_basebackup to fatal out because of the permissions on a backup file
of all things. Instead, we should do WARNING and say skipped.

JD

Hi,

On 2014-05-16 07:28:42 -0700, Joshua D. Drake wrote:
> Can we get that fixed please? It seems rather bad behavior for pg_basebackup
> to fatal out because of the permissions on a backup file of all things.
> Instead, we should do WARNING and say skipped.

Doesn't sound like a good idea to me. We'd need to have a catalog of
common unimportant fileendings and such. We surely *do* want to error
out when we fail to copy an important file.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 05/16/2014 07:30 AM, Andres Freund wrote:
>
> Hi,
>
> On 2014-05-16 07:28:42 -0700, Joshua D. Drake wrote:
>> Can we get that fixed please? It seems rather bad behavior for pg_basebackup
>> to fatal out because of the permissions on a backup file of all things.
>> Instead, we should do WARNING and say skipped.
>
> Doesn't sound like a good idea to me. We'd need to have a catalog of
> common unimportant fileendings and such. We surely *do* want to error
> out when we fail to copy an important file.they
>

pg_hba.conf~ is not an important file.

We know what files are important, especially in $PGDATA, they aren't
variable, so why is pg_basebackup failing on a file it should know or
care nothing about?

JD

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> On 05/16/2014 07:30 AM, Andres Freund wrote:
>> On 2014-05-16 07:28:42 -0700, Joshua D. Drake wrote:
>>> Can we get that fixed please? It seems rather bad behavior for pg_basebackup
>>> to fatal out because of the permissions on a backup file of all things.
>>> Instead, we should do WARNING and say skipped.

>> Doesn't sound like a good idea to me. We'd need to have a catalog of
>> common unimportant fileendings and such. We surely *do* want to error
>> out when we fail to copy an important file.they

> pg_hba.conf~ is not an important file.

Rather than blaming the messenger, you should be asking why there are
files in $PGDATA that the server can't read. That's a recipe for trouble
no matter what.

Or in words of one syllable: this is a bug in your editor, not in Postgres.

regards, tom lane

On 2014-05-16 08:13:04 -0700, Joshua D. Drake wrote:
> On 05/16/2014 07:30 AM, Andres Freund wrote:
> >
> >Hi,
> >
> >On 2014-05-16 07:28:42 -0700, Joshua D. Drake wrote:
> >>Can we get that fixed please? It seems rather bad behavior for pg_basebackup
> >>to fatal out because of the permissions on a backup file of all things.
> >>Instead, we should do WARNING and say skipped.
> >
> >Doesn't sound like a good idea to me. We'd need to have a catalog of
> >common unimportant fileendings and such. We surely *do* want to error
> >out when we fail to copy an important file.they
> >
>
> pg_hba.conf~ is not an important file.

Where do we know that from?

> We know what files are important, especially in $PGDATA, they aren't
> variable, so why is pg_basebackup failing on a file it should know or care
> nothing about?

No, we don't necessarily. It'd e.g. bad to succeed if postgresql.conf
includes another file and we fail when backing that up even though it's
in the data directory.
But even otherwise it'd be a non-neglegible amount of code to enumerate
possibly important files (which wouldn't fully reliable. We can't access
the catalogs). Code that's only there to work around a user doing
something bad that's trivially fixable. Nah.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 05/16/2014 08:19 AM, Tom Lane wrote:

>> pg_hba.conf~ is not an important file.
>
> Rather than blaming the messenger, you should be asking why there are
> files in $PGDATA that the server can't read. That's a recipe for trouble
> no matter what.
>
> Or in words of one syllable: this is a bug in your editor, not in Postgres.

Hardly and shows a distinct lack of user space experience. It also shows
how useless pg_basebackup "can" be. Basically you are saying, "Well
yeah, there is this rogue file that doesn't belong, fark it... we will
blow away a 2TB base backup and make you start over because.. meh,
pg_basebackup is lazy."

Software is supposed to make our lives easier, not harder. I should be
able to evaluate the errors for the conditions they create. This is why
rsync is and for the forseeable future will be king for creating base
backups.

JD

>
> regards, tom lane
>
>

At a minimum:

Check to see if there is going to be a permission error BEFORE the base
backup begins:

starting basebackup:
checking perms: ERROR no access to pg_hba.conf~ base backup will fail

JD

Hi,

On 2014-05-16 08:45:12 -0700, Joshua D. Drake wrote:
> Software is supposed to make our lives easier, not harder. I should be able
> to evaluate the errors for the conditions they create. This is why rsync is
> and for the forseeable future will be king for creating base backups.

It's dangerous to ignore errors rsync errors other than 'file
vanished'. This hardly is an argument for your position.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 05/16/2014 08:48 AM, Andres Freund wrote:
>
> Hi,
>
> On 2014-05-16 08:45:12 -0700, Joshua D. Drake wrote:
>> Software is supposed to make our lives easier, not harder. I should be able
>> to evaluate the errors for the conditions they create. This is why rsync is
>> and for the forseeable future will be king for creating base backups.
>
> It's dangerous to ignore errors rsync errors other than 'file
> vanished'. This hardly is an argument for your position.

Are you reading what I write?

I said. "I should be able to evaluate the errors for the conditions they
create."

I never suggested ignoring anything. The point is RSYNC gives me a
chance at success, pg_basebackup does not (in respect to this specific
condition).

JD

On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake <jd(at)commandprompt(dot)com>wrote:

> At a minimum:
>
> Check to see if there is going to be a permission error BEFORE the base
> backup begins:
>
> starting basebackup:
> checking perms: ERROR no access to pg_hba.conf~ base backup will fail

That's pretty much what it does if you enable progress meter. I realize you
don't necessarily want that one, but we could have a switch that still
tells the server to measure the size, but not actually print the output?
While it costs a bit of overhead to do that, that's certainly something
that's a lot more safe than ignoring errors.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 2014-05-16 18:20:35 +0200, Magnus Hagander wrote:
> On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake <jd(at)commandprompt(dot)com>wrote:
>
> > At a minimum:
> >
> > Check to see if there is going to be a permission error BEFORE the base
> > backup begins:
> >
> > starting basebackup:
> > checking perms: ERROR no access to pg_hba.conf~ base backup will fail
>
>
> That's pretty much what it does if you enable progress meter. I realize you
> don't necessarily want that one, but we could have a switch that still
> tells the server to measure the size, but not actually print the output?
> While it costs a bit of overhead to do that, that's certainly something
> that's a lot more safe than ignoring errors.

Don't think it'll show you that error - that mode only stats() files,
right? So you'd need to add access() or open()s.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Fri, May 16, 2014 at 6:25 PM, Andres Freund <andres(at)2ndquadrant(dot)com>wrote:

> On 2014-05-16 18:20:35 +0200, Magnus Hagander wrote:
> > On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake <jd(at)commandprompt(dot)com
> >wrote:
> >
> > > At a minimum:
> > >
> > > Check to see if there is going to be a permission error BEFORE the base
> > > backup begins:
> > >
> > > starting basebackup:
> > > checking perms: ERROR no access to pg_hba.conf~ base backup will fail
> >
> >
> > That's pretty much what it does if you enable progress meter. I realize
> you
> > don't necessarily want that one, but we could have a switch that still
> > tells the server to measure the size, but not actually print the output?
> > While it costs a bit of overhead to do that, that's certainly something
> > that's a lot more safe than ignoring errors.
>
> Don't think it'll show you that error - that mode only stats() files,
> right? So you'd need to add access() or open()s.
>
>
You're right, we don't. I thought we did, but was clearly remembering wrong.

I guess we could add an access() call to that codepath though. Not sure if
that's going to cause any real overhead compared to the rest of what we're
doing anyway?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 2014-05-16 18:29:25 +0200, Magnus Hagander wrote:
> On Fri, May 16, 2014 at 6:25 PM, Andres Freund <andres(at)2ndquadrant(dot)com>wrote:
>
> > On 2014-05-16 18:20:35 +0200, Magnus Hagander wrote:
> > > On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake <jd(at)commandprompt(dot)com
> > >wrote:
> > >
> > > > At a minimum:
> > > >
> > > > Check to see if there is going to be a permission error BEFORE the base
> > > > backup begins:
> > > >
> > > > starting basebackup:
> > > > checking perms: ERROR no access to pg_hba.conf~ base backup will fail
> > >
> > >
> > > That's pretty much what it does if you enable progress meter. I realize
> > you
> > > don't necessarily want that one, but we could have a switch that still
> > > tells the server to measure the size, but not actually print the output?
> > > While it costs a bit of overhead to do that, that's certainly something
> > > that's a lot more safe than ignoring errors.
> >
> > Don't think it'll show you that error - that mode only stats() files,
> > right? So you'd need to add access() or open()s.
> >
> >
> You're right, we don't. I thought we did, but was clearly remembering wrong.
>
> I guess we could add an access() call to that codepath though. Not sure if
> that's going to cause any real overhead compared to the rest of what we're
> doing anyway?

It's not free. But I don't think it'd seriously matter in comparison.

But it doesn't protect you if the file is created during the backup -
which as you know can take a long time. For example because somebody
felt the need to increase wal_keep_segments.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 05/16/2014 09:20 AM, Magnus Hagander wrote:
>
> On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake <jd(at)commandprompt(dot)com
> <mailto:jd(at)commandprompt(dot)com>> wrote:
>
> At a minimum:
>
> Check to see if there is going to be a permission error BEFORE the
> base backup begins:
>
> starting basebackup:
> checking perms: ERROR no access to pg_hba.conf~ base backup will fail
>
>
> That's pretty much what it does if you enable progress meter. I realize
> you don't necessarily want that one, but we could have a switch that
> still tells the server to measure the size, but not actually print the
> output? While it costs a bit of overhead to do that, that's certainly
> something that's a lot more safe than ignoring errors.

That seems reasonable.

JD

Andres Freund-3 wrote
> On 2014-05-16 18:29:25 +0200, Magnus Hagander wrote:
>> On Fri, May 16, 2014 at 6:25 PM, Andres Freund &lt;

> andres@

> &gt;wrote:
>>
>> > On 2014-05-16 18:20:35 +0200, Magnus Hagander wrote:
>> > > On Fri, May 16, 2014 at 5:46 PM, Joshua D. Drake &lt;

> jd@

> &gt; > >wrote:
>> > >
>> > > > At a minimum:
>> > > >
>> > > > Check to see if there is going to be a permission error BEFORE the
>> base
>> > > > backup begins:
>> > > >
>> > > > starting basebackup:
>> > > > checking perms: ERROR no access to pg_hba.conf~ base backup will
>> fail
>> > >
>> > >
>> > > That's pretty much what it does if you enable progress meter. I
>> realize
>> > you
>> > > don't necessarily want that one, but we could have a switch that
>> still
>> > > tells the server to measure the size, but not actually print the
>> output?
>> > > While it costs a bit of overhead to do that, that's certainly
>> something
>> > > that's a lot more safe than ignoring errors.
>> >
>> > Don't think it'll show you that error - that mode only stats() files,
>> > right? So you'd need to add access() or open()s.
>> >
>> >
>> You're right, we don't. I thought we did, but was clearly remembering
>> wrong.
>>
>> I guess we could add an access() call to that codepath though. Not sure
>> if
>> that's going to cause any real overhead compared to the rest of what
>> we're
>> doing anyway?
>
> It's not free. But I don't think it'd seriously matter in comparison.
>
> But it doesn't protect you if the file is created during the backup -
> which as you know can take a long time. For example because somebody
> felt the need to increase wal_keep_segments.
>
> Greetings,
>
> Andres Freund

Can we simply backup the non-data parts of $PGDATA first then move onto the
data-parts? For the files that we'd be dealing with it would be
sufficiently quick to just try and fail, immediately, then check for all
possible preconditions first. The main issue seems to be the case where the
2TB of data get backed-up and then a small 1k file blows away all that work.
Lets do those 1k files first.

David J.

--
View this message in context: http://postgresql.1045698.n5.nabble.com/pg-basebackup-could-not-get-transaction-log-end-position-from-server-FATAL-could-not-open-file-pg-hbd-tp5804225p5804257.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.

On 05/16/2014 08:11 PM, David G Johnston wrote:
> Can we simply backup the non-data parts of $PGDATA first then move onto the
> data-parts? For the files that we'd be dealing with it would be
> sufficiently quick to just try and fail, immediately, then check for all
> possible preconditions first. The main issue seems to be the case where the
> 2TB of data get backed-up and then a small 1k file blows away all that work.
> Lets do those 1k files first.

You'll still need to distinguish "data" and "non-data" parts somehow.
One idea would be to backup any files in the top directory first, before
recursing into the subdirectories. That would've caught the OP's case,
and probably many other typical cases where you drop something
unexpected into $PGDATA. You could still have something funny nested
deep in the data directory, but that's much less common.

- Heikki