Re: Proposal: Incremental Backup

2014-08-01 18:20 GMT+02:00 Claudio Freire <klaussfreire(at)gmail(dot)com>:

> On Fri, Aug 1, 2014 at 12:35 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >> c) the map is not crash safe by design, because it needs only for
> >> incremental backup to track what blocks needs to be backuped, not for
> >> consistency or recovery of the whole cluster, so it's not an heavy cost
> for
> >> the whole cluster to maintain it. we could think an option (but it's
> heavy)
> >> to write it at every flush on file to have crash-safe map, but I not
> think
> >> it's so usefull . I think it's acceptable, and probably it's better to
> force
> >> that, to say: "if your db will crash, you need a fullbackup ",
> >
> > I am not sure if your this assumption is right/acceptable, how can
> > we say that in such a case users will be okay to have a fullbackup?
> > In general, taking fullbackup is very heavy operation and we should
> > try to avoid such a situation.
>
>
> Besides, the one taking the backup (ie: script) may not be aware of
> the need to take a full one.
>
> It's a bad design to allow broken backups at all, IMNSHO.
>

Hi Claudio,
thanks for your observation
First: the case it's after a crash of a database, and it's not something
happens every day or every week. It's something that happens in rare
conditions, or almost my experience is so. If it happens very often
probably there are other problems.
Second: to avoid the problem to know if the db needed to have a full backup
to rebuild the map we could think to write in the map header the backup
reference (with an id and LSN reference for example ) so if the
someone/something try to do an incremental backup after a crash, the map
header will not have noone full backup listed [because it will be empty] ,
and automaticcaly switch to a full one. I think after a crash it's a good
practice to do a full backup, to see if there are some problems on files or
on filesystems, but if I am wrong I am happy to know :) .

Remember that I propose a map in ram to reduce the impact on performances,
but we could create an option to leave the choose to the user, if you want
a crash safe map, at every flush will be updated also a map file , if not,
the map will be in ram.

Kind Regards

Mat