Re: pg_receivexlog --status-interval add fsync feedback

On Thu, Oct 9, 2014 at 6:42 PM, <furuyao(at)pm(dot)nttdata(dot)co(dot)jp> wrote:
>> >>> If we remove --fsync-interval, resoponse time to user will not be
>> delay.
>> >>> Although, fsync will be executed multiple times in a short period.
>> >>> And there is no way to solve the problem without --fsync-interval,
>> >>> what
>> >> should we do about it?
>> >>
>> >> I'm sorry, I didn't understand that.
>> >
>> > Here is an example.
>> > When WAL is sent at 100ms intervals, fsync() is executed 10 times per
>> second.
>> > If --fsync-interval is set by 1 second, we have to wait SQL
>> responce(kind of making WAL record) for 1 second, though, fsync() won't
>> be executed several times in 1 second.
>> > I think --fsync-interval meets the demands of people who wants to
>> restrain fsync() happens for several time in short period, but what do
>> you think?
>> > Is it ok to delete --fsync-interval ?
>>
>> I still don't see the problem.
>>
>> In synchronous mode, pg_receivexlog should have similar logic as
>> walreceiver does. It should read as much WAL from the socket as it can
>> without blocking, and fsync() and send reply after that. And also fsync
>> whenever switching to new segment. If the master sends WAL every 100ms,
>> then pg_recevexlog will indeed fsync() 10 times per second. There's
>> nothing wrong with that.
>>
>> In asynchronous mode, only fsync whenever switching to new segment.
>>
>> >> Yeah. Or rather, add a new message type, to indicate the
>> >> synchronous/asynchronous status.
>> >
>> > What kind of 'message type' we have to add ?
>> >
>> > Do we need to separate 'w' into two types ? synchronous and
>> asynchronous ?
>> >
>> > OR
>> >
>> > Add a new message type, kind of 'notify synchronous', and inform
>> > pg_receivexlog of synchronous status when it connect to the server.
>>
>> Better to add a new "notify" message type. And pg_recevexlog should be
>> prepared to receive it at any time. The status might change on the fly,
>> if the server's configuration is reloaded.
>
> Thanks for the reply.
>
>> In synchronous mode, pg_receivexlog should have similar logic as walreceiver does.
>
> OK. I understand that removing --fsync-interval has no problem.

+1 for adding something like --synchronous option. To me,
it sounds walreceiver-compatible mode rather than synchronous.

>> Better to add a new "notify" message type. And pg_recevexlog should be prepared to receive it at any time. The status might change on the fly, if the server's configuration is reloaded.
>
> OK. I'll consider it.

I don't think that's good idea because it prevents us from using
pg_receivexlog as async walreceiver (i.e., received WAL data is
fsynced and feedback is sent back to the server soon,
but transaction commit in the server doesn't wait for the feedback).

Regards,

--
Fujii Masao