Re: pg_basebackup failed to back up large file

On 2014-06-03 11:04:58 -0400, Tom Lane wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > On Tue, Jun 3, 2014 at 4:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> There's a far bigger problem there, which is if we're afraid that
> >> current_len_left might exceed 4GB then what is it exactly that guarantees
> >> it'll fit in an 11-digit field?
>
> > Well, we will only write 11 digits in there, that's when we read it. But
> > print_val() on the server side should probably have an overflow check
> > there, which it doesn't. It's going to write some strange values int here
> > if it overflows..
>
> My point is that having backups crash on an overflow doesn't really seem
> acceptable. IMO we need to reconsider the basebackup protocol and make
> sure we don't *need* to put values over 4GB into this field. Where's the
> requirement coming from anyway --- surely all files in PGDATA ought to be
> 1GB max?

Fujii's example was logfiles in pg_log. But we allow to change the
segment size via a configure flag, so we should support that or remove
the ability to change the segment size...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services