Re: Further pg_upgrade analysis for many tables

On 11/14/2012 10:08 AM, Bruce Momjian wrote:
> On Wed, Nov 14, 2012 at 06:11:27AM +0200, Ants Aasma wrote:
>>
>> I agree that parallel restore for schemas is a hard problem. But I
>> didn't mean parallelism within the restore, I meant that we could
>> start both postmasters and pipe the output from dump directly to
>> restore. This way the times for dumping and restoring can overlap.
> Wow, that is a very creative idea. The current code doesn't do that,
> but this has the potential of doubling pg_upgrade's speed, without
> adding a lot of complexity. Here are the challenges of this approach:
>
> * I would need to log the output of pg_dumpall as it is passed to psql
> so users can debug problems

Instead of piping it directly, have pg_upgrade work as a tee, pumping
bytes both to psql and a file. This doesn't seem terribly hard.

>
> * pg_upgrade never runs the old and new clusters at the same time for
> fear that it will run out of resources, e.g. shared memory, or if they
> are using the same port number. We can make this optional and force
> different port numbers.

Right.

cheers

andrew