Re: [RFC] Incremental backup v2: add backup profile to base backup

Il 06/10/14 16:51, Robert Haas ha scritto:
> On Mon, Oct 6, 2014 at 8:59 AM, Marco Nenciarini
> <marco(dot)nenciarini(at)2ndquadrant(dot)it> wrote:
>> Il 04/10/14 08:35, Michael Paquier ha scritto:
>>> On Sat, Oct 4, 2014 at 12:31 AM, Marco Nenciarini
>>> <marco(dot)nenciarini(at)2ndquadrant(dot)it> wrote:
>>>> Compared to first version, we switched from a timestamp+checksum based
>>>> approach to one based on LSN.
>>> Cool.
>>>
>>>> This patch adds an option to pg_basebackup and to replication protocol
>>>> BASE_BACKUP command to generate a backup_profile file. It is almost
>>>> useless by itself, but it is the foundation on which we will build the
>>>> file based incremental backup (and hopefully a block based incremental
>>>> backup after it).
>>> Hm. I am not convinced by the backup profile file. What's wrong with
>>> having a client send only an LSN position to get a set of files (or
>>> partial files filed with blocks) newer than the position given, and
>>> have the client do all the rebuild analysis?
>>>
>>
>> The main problem I see is the following: how a client can detect a
>> truncated or removed file?
>
> When you take a differential backup, the server needs to send some
> piece of information about every file so that the client can compare
> that list against what it already has. But a full backup does not
> need to include similar information.
>

I agree that a full backup does not need to include a profile.

I've added the option to require the profile even for a full backup, as
it can be useful for backup softwares. We could remove the option and
build the profile only during incremental backups, if required. However,
I would avoid the needing to scan the whole backup to know the size of
the recovered data directory, hence the backup profile.

Regards,
Marco

--
Marco Nenciarini - 2ndQuadrant Italy
PostgreSQL Training, Services and Support
marco(dot)nenciarini(at)2ndQuadrant(dot)it | www.2ndQuadrant.it