Re: WAL Rate Limiting

On Wed, Jan 15, 2014 at 7:50 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> We've discussed previously the negative impact of large bulk
> operations, especially wrt WAL writes. Patch here allows maintenance
> operations to have their WAL generation slowed down as a replication
> lag prevention feature.
>
> I believe there was originally intended to be some work on I/O rate
> limiting, but that hasn't happened and is in some ways orthogonal to
> this patch and we will likely eventually want both.
>
> Single new parameter works very similarly to vacuum_cost_delay
>
> wal_rate_limit_delay = Xms
>
> slows down CLUSTER, VACUUM FULL, ALTER TABLE (rewrite & set
> tablespace), CREATE INDEX
> so basically same things we optimise WAL for and the same places where
> we honour maintenance_work_mem
> (discuss: should we add COPY, CTAS etc also?)
> (discuss: do we need another parameter to specify "cost"? Currently
> patch uses "sleep every 64kB of WAL")
>
> VACUUM is not included, since we already have controls for that -
> honouring two controls would be complex and weird.
>
>
I wonder if should replace vacuum_cost/delay with say
maintenance_cost/delay and use it in all maintenance activities including
what you just listed out above. While the exact semantics of vacuum and
other maintenance activities may differ, ISTM the final goal is just the
same i.e. reduce load on the master.

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee