Re: pg_receivexlog --status-interval add fsync feedback

From: <furuyao(at)pm(dot)nttdata(dot)co(dot)jp>
To: <masao(dot)fujii(at)gmail(dot)com>
Cc: <robertmhaas(at)gmail(dot)com>, <hlinnakangas(at)vmware(dot)com>, <simon(at)2ndquadrant(dot)com>, <sawada(dot)mshk(at)gmail(dot)com>, <pgsql-hackers(at)postgresql(dot)org>, <teranishih(at)nttdata(dot)co(dot)jp>
Subject: Re: pg_receivexlog --status-interval add fsync feedback
Date: 2014-11-10 10:19:30
Message-ID: A9C510524E235E44AE909CD4027AE196BF7D6FB26E@MBX-MSG-SV03.msg.nttdata.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Fri, Oct 31, 2014 at 5:46 PM, <furuyao(at)pm(dot)nttdata(dot)co(dot)jp> wrote:
> >> > We seem to be going in circles. You suggested having two options,
> >> > --feedback, and --fsync, which is almost exactly what Furuya posted
> >> > originally. I objected to that, because I think that user interface
> >> is
> >> > too complicated. Instead, I suggested having just a single option
> >> > called --synchronous, or even better, have no option at all and
> >> > have the server tell the client if it's participating in
> >> > synchronous replication, and have pg_receivexlog automatically
> >> > fsync when it is, and not otherwise [1]. That way you don't need
> to
> >> > expose any new options to the user. What did you think of that idea?
> >>
> >> I think it's pretty weird to make the fsync behavior of the client
> is
> >> controlled by the server.
> >>
> >> I also don't think that fsync() on the client side is useless in
> >> asynchronous replication. Yeah, it's true that there are no
> >> *guarantees* with asynchronous replication, but the bound on how long
> >> the data can take to get out to disk is a heck of a lot shorter if
> >> you fsync frequently than if you don't. And on the flip side, that
> >> has a performance impact.
> >>
> >> So I actually think the design you proposed is not as good as what
> >> was proposed by Furuya and Simon. But I don't feel incredibly
> >> strongly about it.
> >
> > Thanks for lots of comments!!
> >
> > I fixed the patch.
> > As a default, it behave like a walreceiver.
> > Same as walreceiver, it fsync and send a feedback after fsync.
>
> On second thought, flipping the default behavior seems not worthwhile
> here.
> Which might surprise the existing users and cause some troubles to them.
> I'd like to back to the Heikki's original suggestion like just adding
> --synchronous option. That is, only when --synchronous is specified, WAL
> is flushed and feedback is sent back as soon as WAL is received.
>
> I changed the patch heavily in that way. Please find the attached patch.
> By default, pg_receivexlog flushes WAL data only when WAL file is closed.
> If --synchronous is specified, like the standby's WAL receiver, sync
> commands are issued as soon as there is WAL data which has not been flushed
> yet. Also status packets are sent back to the server just after WAL data
> is flushed whatever --status-interval is set to. I added the description
> of this behavior to the doc.
>
> Thought?

I think it's as you pointed out.
Thank you for the patch.
I did a review of the patch.
There was no problem.
I confirmed the following.

1. applied cleanly and compilation was without warnings and errors
2. all regress tests was passed ok
3. when --synchronous is not specified, do not fsync except wal has switched
4. it get normal backup when pg_basebackup has executed with '-X s'
5. when --synchronous is specified, it fsync every time when it receive WAL data
6. when --synchronous is specified, in spite of -s option, feedback are sent back when it fsync
7. when --slog is not specified, it wouldn't feedback fsync location
8. no problem in document

Regards,

--
Furuya Osamu

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2014-11-10 10:23:33 Re: alter user/role CURRENT_USER
Previous Message Michael Paquier 2014-11-10 08:26:51 Re: [REVIEW] Re: Compression of full-page-writes