Re: WIP patch for parallel pg_dump

On Mon, Dec 6, 2010 at 10:40 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Mon, Dec 6, 2010 at 9:45 AM, Heikki Linnakangas
>> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>>> Well, then you need some sort of cross-backend communication, which is
>>> always a bit clumsy.
>
>> A temp file seems quite sufficient, and not at all difficult.
>
> "Not at all difficult" is nonsense.  To do that, you need to invent some
> mechanism for sender and receivers to identify which temp file they want
> to use,

Why is this even remotely hard? That's the whole point of having the
"publish" operation return a token. The token either is, or uniquely
identifies, the file name.

> and you need to think of some way to clean up the files when the
> client forgets to tell you to do so. That's going to be at least as
> ugly as anything else.

Backends don't forget to call their end-of-transaction hooks, do they?
They might crash, but we already have code to remove temp files on
server restart. At most it would need minor adjustment.

>  And I think it's unproven that this approach
> would be security-hole-free either.  For instance, what about some other
> session overwriting pg_dump's snapshot temp file?

Why would this be any different from any other temp file? We surely
must have a mechanism in place to ensure that the temporary files used
by sorts or hash joins don't get overwritten by some other session, or
the system would be totally unstable.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company