Re: autovacuum next steps, take 3

Hi, Alvaro

Alvaro Herrera wrote:
> keep such a task list in shared memory, because we aren't able to grow
> that memory after postmaster start.

We can use the fix-size share memory to maintain such a queue. The
maximum task size is the number of all tables. So the size of the queue
can be the same with max_fsm_relations which is usually larger than the
numbers of tables and indexes in the cluster. This is sufficient to
contain most of the vacuum tasks.

Even though the queue is over flow, for task-gatherer is scanning the
whole cluster every autovacuum_naptime, it is quickly enough to pick
those tasks up again. We don’t need to write any thing to external file.
So there is no problem to use a fix-size share memory to maintain a
global queue.

> Other idea I had was having a third kind of autovacuum process, namely a
> "schedule builder"

If we have such a global queue, task-gathering worker can connect to
every database every naptime to gather tasks in time.

The task-gathering worker won’t build the schedule, LAUNCHER or external
program responses for such activity. How to dispatch tasks to worker is
just a scheduling problem, a good dispatching algorithm needs to ensure
each worker can finish their tasks on time, this might resolve the
headache HOT table problem. But this is a further issue to be discussed
after 8.3.

Best Regards

Galy Lee
lee.galy _at_ oss.ntt.co.jp
NTT Open Source Software Center