Re: checkpointer continuous flushing

On 2015-06-02 18:59:05 +0200, Fabien COELHO wrote:
>
> >>>IMO this feature, if done correctly, should result in better performance
> >>>in 95+% of the workloads
> >>
> >>To demonstrate that would require time...
> >
> >Well, that's part of the contribution process. Obviously you can't test
> >100% of the problems, but you can work hard with coming up with very
> >adversarial scenarios and evaluate performance for those.
>
> I did spent time (well, a machine spent time, really) to collect some
> convincing data for the simple version without sorting to demonstrate that
> it brings a clear value, which seems not to be enough...

"which seems not to be enough" - man. It's trivial to make things
faster/better/whatever if you don't care about regressions in other
parts. And if we'd add a guc for each of these cases we'd end up with
thousands of them.

> My opinion is that throughput is given too much attention in general, but if
> both can be kept/improved, this would be easier to sell, obviously.

Your priorities are not everyone's. That's life.

> >That might be the case in a database with a single small table;
> >i.e. where all the writes go to a single file. But as soon as you have
> >large tables (i.e. many segments) or multiple tables, a significant part
> >of the writes issued independently from checkpointing will be outside
> >the processing of the individual segment.
>
> Statistically, I think that it would reduce the number of unrelated writes
> taken in a fsync by about half: the last table to be written on a
> tablespace, at the end of the checkpoint, will have accumulated
> checkpoint-unrelated writes (bgwriter, whatever) from the whole checkpoint
> time, while the first table will have avoided most of them.

That's disregarding that a buffer written out by a backend starts to get
written out by the kernel after ~5-30s, even without a fsync triggering
it.