Re: pg_basebackup failed to back up large file

On Jun 3, 2014 6:17 PM, "Andres Freund" <andres(at)2ndquadrant(dot)com> wrote:
>
> On 2014-06-03 17:57:52 +0200, Magnus Hagander wrote:
> > On Tue, Jun 3, 2014 at 5:42 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > > What we had better do, IMO, is fix things so that we don't have a
filesize
> > > limit in the basebackup format. After a bit of googling, I found out
that
> > > recent POSIX specs for tar format include "extended headers" that
among
> > > other things support member files of unlimited size [1]. Rather than
> > > fooling with partial fixes, we should make the basebackup logic use an
> > > extended header when the file size is over INT_MAX.
>
> > Yeah, pax seems to be the way to go. It's at least supported by GNU tar
-
> > is it also supported on say BSD, or other popular platforms? (The size
> > extension in the general ustar format seems to be, so it would be a
shame
> > if this one is less portable)
>
> PG's tar.c already uses the ustar format and the referenced extension is
> an extension to ustar as far as I understand it. So at least tarballs
> with files < 8GB would still continue to be readable with all currently
> working implementations.

Yeah, that is a clear advantage of that method. Didn't read up on pax
format backwards compatibility, does it have some trick to achieve
something similar?

/Magnus