Re: Backup throttling

On Tue, Aug 27, 2013 at 12:58 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Tue, Aug 20, 2013 at 2:37 AM, Heikki Linnakangas
> <hlinnakangas(at)vmware(dot)com> wrote:
> > Throttling in the client seems much better to me. TCP is designed to
> handle
> > a slow client.
>
> Other people have already offered some good points in this area, but
> let me just add one thought that I don't think has been mentioned yet.
> We have a *general* need to be able to throttle server-side resource
> utilization, particularly I/O. This is a problem not only for
> pg_basebackup, but for COPY, CLUSTER, VACUUM, and even things like
> UPDATE. Of all of those, the only one for which we currently have any
> kind of a solution is VACUUM. Now, maybe pg_basebackup also needs its
> own special-purpose solution, but I think we'd do well to consider a
> general I/O rate-limiting strategy and then consider particular needs
> in the light of that framework. In that context, server-side seems
> better to me, because something like CLUSTER isn't going to produce
> anything that the client can effectively limit.
>

+1 it is very easy at the moment to for example run a manual vacuum
full/cluster against a big table and generate WAL so quickly that the hot
standby disconnects because it gets "too far behind".