Re: Load distributed checkpoint

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Ron Mayer" <rm_pg(at)cheapcomplexdevices(dot)com>, "ITAGAKI Takahiro" <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>, <pgsql-hackers(at)postgresql(dot)org>, "Zeugswetter Andreas ADI SD" <ZeugswetterA(at)spardat(dot)at>
Subject: Re: Load distributed checkpoint
Date: 2006-12-12 15:03:23
Message-ID: 457E705A.EE98.0025.0@wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

>>> On Tue, Dec 12, 2006 at 3:22 AM, in message
<E1539E0ED7043848906A8FF995BDA5790198ECC8(at)m0143(dot)s-mxs(dot)net>,
"Zeugswetter
Andreas ADI SD" <ZeugswetterA(at)spardat(dot)at> wrote:

>> > One thing I do worry about is if both postgresql and the OS
>> > are both delaying write()s in the hopes of collapsing them
>> > at the same time. If so, this would cause both to be experience
>> > bigger delays than expected, and make checkpoints worse.
>>
>> That is my concern. Letting 30 seconds worth of dirty pages
> accumulate
>> between checkpoints and then trying to smooth the writes within
>> checkpoint code seems like a doomed effort.
>
> 30 seconds ??? You are supposed to avoid excessive checkpoints.
> If you are reducing checkpoint_timeout to avoid the spike, you are
imho
> definitely tuning the wrong knob.

Sorry for the fuzzy language -- I was intending to describe a situation
where dirty pages accumulate by checkpoint time which would take 30
seconds to write to disk. We were into this situation (and worse) with
the default bgwriter settings.

> you have to make bgwriter more aggressive.

This is what I've been saying.

I've also been saying that if the PostgreSQL way is to let the file
system handle the caching and I/O scheduling, we should trust it to know
what to do with dirty pages, and not try to second-guess it. (Of course
there are knobs to tune the file system if needed.) Our checkpoint
performance issues went away when we went to settings which basically
never leave a dirty page hidden from the file system for more than two
seconds.

-Kevin

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Patrick Welche 2006-12-12 15:22:19 coalesce and aggregate functions
Previous Message DANTE Alexandra 2006-12-12 11:47:33 Re: PostgreSQL 8.2 : regression failures on IA-64

Browse pgsql-patches by date

  From Date Subject
Next Message Belinda M. Giardine 2006-12-12 16:32:47 date comparisons
Previous Message Zeugswetter Andreas ADI SD 2006-12-12 09:22:05 Re: Load distributed checkpoint