Reduce the amount of data loss on the standby

Lists: pgsql-hackers
From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Reduce the amount of data loss on the standby
Date: 2011-01-14 22:31:35
Message-ID: AANLkTim-9eg59eD=+JbqyoKhPzVbqFL270HTOGgWFc32@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi,

I propose the patch which reduces the amount of data loss
on the standby.

If you create the trigger file after replication is terminated,
the standby tries to replay all the WAL available in pg_xlog
and the archive. This behavior is OK, and it's helpful to reduce
the amount of data loss.

But, if you create the trigger file while walreceiver is in
progress, the standby replays only the WAL streamed from
the master. Even if there are many outstanding WAL files in
the archive or pg_xlog, they are not replayed. This might
cause much data loss, so I think we should fix it.

The attached patch changes the startup process so that
it tries to replay all the WAL available in the archive and pg_xlog
even if the trigger file is found while walreceiver is running.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment Content-Type Size
reduce_data_loss_v1.patch application/octet-stream 1.0 KB

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Reduce the amount of data loss on the standby
Date: 2011-02-08 15:34:12
Message-ID: 1297179252.1770.8061.camel@ebony
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sat, 2011-01-15 at 07:31 +0900, Fujii Masao wrote:

> I propose the patch which reduces the amount of data loss
> on the standby.

Committed. Well spotted.

--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services