bg worker: overview

[Markus Wanner <markus(at)bluegap(dot)ch>]

Hi,

I've been working on modularizing Postgres-R to ease review and maybe 
allow code reuse. As threatened at the Cluster Meeting in Tokyo and 
again at CHAR(10), I'm now presenting more results of that effort: the 
background workers infrastructure module.

Postgres-R so far used custom backends to apply transactions from remote 
nodes. These were controlled by an additional coordinator process, which 
acted as a job dispatcher and obviously didn't have a client connection. 
There were obvious similarities between that and the existing autovacuum 
component, with its launcher that controls multiple worker processes.

I've combined these two components into a single, general purpose 
background worker infrastructure component, which is now capable to 
serve autovacuum as well as Postgres-R. And it might be of use for other 
purposes as well, most prominently parallel query processing. Basically 
anything that needs a backend connected to a database to do any kind of 
background processing, possibly parallelized.

Overall, this module represents quite a large portion of the Postgres-R 
patch. 15% by lines inserted (2912 vs 19332) and as much as 95% by lines 
deleted (1422 vs 1482).

With this further modularization, I hope to increase understandability 
and wish to encourage more hackers to have a look at (parts of) the 
Postgres-R source code. Of course, I highly appreciate reviews and 
discussions. And it would be very nice to see this module reused. Please 
don't hesitate to ask questions, if you need help.

(I don't dare to add these patches to the commit fest, as this 
refactoring doesn't have any immediate benefit for Postgres itself, at 
the moment.)

Regards

Markus Wanner

P.S.: git adicts, everything's up here:
http://git.postgres-r.org/?p=bgworker