From: | Andres Freund <andres(at)2ndquadrant(dot)com> |
---|---|
To: | Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> |
Cc: | MauMau <maumau307(at)gmail(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: [patch] pg_copy - a command for reliable WAL archiving |
Date: | 2014-08-20 14:14:09 |
Message-ID: | 20140820141409.GJ10041@alap3.anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 2014-08-20 09:50:56 -0400, Alvaro Herrera wrote:
> Andres Freund wrote:
> > On 2014-08-19 17:42:06 -0400, Alvaro Herrera wrote:
> > > MauMau wrote:
> > >
> > > > With that said, copying to a temporary file like <dest>.tmp and
> > > > renaming it to <dest> sounds worthwhile even as a basic copy
> > > > utility. I want to avoid copying to a temporary file with a fixed
> > > > name like _copy.tmp, because some advanced utility may want to run
> > > > multiple instances of pg_copy to copy several files into the same
> > > > directory simultaneously. However, I'm afraid multiple <dest>.tmp
> > > > files might continue to occupy disk space after canceling copy or
> > > > power failure in some use cases, where the copy of the same file
> > > > won't be retried. That's also the reason why I chose to not use a
> > > > temporary file like cp/copy.
> > >
> > > Is there a way to create a link to a file which only exists as an open
> > > file descriptor? If there was, you could create a temp file, open an
> > > fd, then delete the file. That would remove the issue with files being
> > > leaked due to failures of various kinds.
> >
> > Isn't this a solution looking for a problem? We're using tempfiles in
> > dozens of other places and I really don't see why this is the place to
> > stop doing so. Just copy to <dest>.tmp and move it into place. If things
> > crash during that, the command will be repeated shortly afterwards again
> > *anyway*. Let's not get into platform specific games here.
>
> The issue is what happens if there's a crash while the temp file is in
> the middle of being filled.
The archive command will be be run again a couple seconds and remove the
half-filled temp file.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Alvaro Herrera | 2014-08-20 14:18:28 | Re: [patch] pg_copy - a command for reliable WAL archiving |
Previous Message | Greg Stark | 2014-08-20 14:08:25 | Re: [patch] pg_copy - a command for reliable WAL archiving |