Re: Testing of parallel restore with current snapshot

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Tom Lane wrote:
>> I don't want to mess with it right now either, but perhaps we should
>> have a TODO item to improve the intelligence of parallel restore so that
>> it really does try to do things this way.

> Other things being equal it schedules things in TOC order, which often
> works as we want anyway. I think there's a good case for altering the
> name sort order of pg_dump to group sub-objects of a table (indexes,
> constraints etc.) together, ie. instead of sorting by <objectname>, we'd
> sort by <tablename, objectname>. This would possibly improve the effect
> seen in parallel restore without requiring any extra intelligence there.

I'm not at all excited about substituting one arbitrary ordering rule
for another one ...

What is probably happening that accounts for Josh's positive experience
is that all the indexes of a particular table "come free" from the
dependency restrictions at the same instant, namely when the data load
for that table ends. So if there's nothing else to do they'll get
scheduled together. However, if the data load for table B finishes
before all the indexes of table A have been scheduled, then B's indexes
will start competing with A's for scheduling slots. The performance
considerations suggest that we'd be best advised to finish out all of
A's indexes before scheduling any of B's, but I'm not sure that that's
what it will actually do.

Based on this thought, what seems to make sense as a quick-and-dirty
answer is to make sure that items get scheduled in the same order they
came free from dependency restrictions. I don't recall whether that
is true at the moment, or how hard it might be to make it true if it
isn't already.

regards, tom lane