Re: pg_basebackup from cascading standby after timeline switch

On 18 December 2012 00:53, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>> On 17 December 2012 14:16, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com> wrote:
>>> I also wonder if pg_basebackup should
>>> include *all* timeline history files in the backup, not just the latest one
>>> strictly required to restore.
>
>> Why? the timeline history file includes the previous timelines already.
>
> The original intention was that the WAL archive might contain multiple
> timeline files corresponding to various experimental recovery attempts;
> furthermore, such files might be hand-annotated (that's why there's a
> comment provision). So they would be at least as valuable from an
> archival standpoint as the WAL files proper. I think we ought to just
> copy all of them, period. Anything less is penny-wise and
> pound-foolish.

What I'm saying is that the new history file is created from the old
one, so the latest file includes all the history as a direct copy. The
only thing new is one line of information.

Copying all files grows at O(N^2) with redundancy and will eventually
become a space problem and a performance issue for smaller systems.
There should be some limit to keep this sane, for example, the last 32
history files, or the last 1000 lines of history. Some sane limit.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services