Re: pg_dump directory archive format / parallel pg_dump

On Wed, Feb 2, 2011 at 13:32, Joachim Wieland <joe(at)mcknight(dot)de> wrote:
> Here is a rebased version with some minor changes as well.

I read the patch works as below. Am I understanding correctly?
1. Open all connections in a parent process.
2. Start transactions for each connection in the parent.
3. Spawn child processes with fork().
4. Each child process uses one of the inherited connections.

I think we have 2 important technical issues here:
* The consistency is not perfect. Each transaction is started
with small delays in step 1, but we cannot guarantee no other
transaction between them.
* Can we inherit connections to child processes with fork() ?
Moreover, we also need to pass running transactions to children.
I wonder libpq is designed for such usage.

To solve both issues, we might want a way to control visibility
in a database server instead of client programs. Don't we need
server-side support like [1] before developing parallel dump?
[1] http://wiki.postgresql.org/wiki/ClusterFeatures#Export_snapshots_to_other_sessions

> I haven't
> tested it on Windows now but will do so as soon as the Unix part has
> been reviewed.

It might be better to remove Windows-specific codes from the first try.
I doubt Windows message queue is the best API in such console-based
application. I hope we could use the same implementation for all
platforms for inter-process/thread communication.

--
Itagaki Takahiro