Re: possible bug: orphaned files left after immediate shutdown during DDL

Case:

BEGIN;
CREATE TABLE foo AS SELECT generate_series(1,1000);
CHECKPOINT;
SELECT relfilenode FROM pg_class WHERE relname='foo';

Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
shutdown), and then restart. The file 23456 is still in the filesystem,
but there's no record in pg_class for it. I don't see any obvious path
where it will be removed, so it looks like it will just stay there
forever.

My question is: is this a conscious decision to be paranoid during
recovery, or is this a bug? Or is there some reason that properly
determining which files should be removed at recovery time is
challenging?

Regards,
Jeff Davis

Jeff Davis <pgsql(at)j-davis(dot)com> writes:
> Case:

> BEGIN;
> CREATE TABLE foo AS SELECT generate_series(1,1000);
> CHECKPOINT;
> SELECT relfilenode FROM pg_class WHERE relname='foo';

> Let's say that returns 23456. Send the postmaster a SIGQUIT (immediate
> shutdown), and then restart. The file 23456 is still in the filesystem,
> but there's no record in pg_class for it. I don't see any obvious path
> where it will be removed, so it looks like it will just stay there
> forever.

> My question is: is this a conscious decision to be paranoid during
> recovery, or is this a bug?

It's intentional ... not that other people haven't complained about it
before. Remember that what you have done is forced a crash, and
recovery from it is crash recovery. If we proactively removed such
files we would very possibly be destroying evidence of forensic value.

IMO, immediate shutdown is not a tool to be used at random, and this
isn't something we need to fix.

regards, tom lane

On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
> It's intentional ... not that other people haven't complained about it
> before. Remember that what you have done is forced a crash, and
> recovery from it is crash recovery. If we proactively removed such
> files we would very possibly be destroying evidence of forensic value.

I thought that might be the case, but I wasn't able to find any previous
discussions.

It might be a good idea to issue a warning during recovery, however,
like "possible orphaned file ...". I'm not sure if it's worth the
bookkeeping effort though.

Regards,
Jeff Davis

Jeff Davis wrote:
> On Wed, 2011-02-09 at 22:58 -0500, Tom Lane wrote:
> > It's intentional ... not that other people haven't complained about it
> > before. Remember that what you have done is forced a crash, and
> > recovery from it is crash recovery. If we proactively removed such
> > files we would very possibly be destroying evidence of forensic value.
>
> I thought that might be the case, but I wasn't able to find any previous
> discussions.
>
> It might be a good idea to issue a warning during recovery, however,
> like "possible orphaned file ...". I'm not sure if it's worth the
> bookkeeping effort though.

I thought we had a TODO item about removing orphaned files, but I don't
see it now, perhaps because I thought we had fixed that.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +