bgwriter holds onto file handles of deleted files

On an overnight test run, I ran into an unexpected "out of space" condition.

The writer process was holding on to dozens of file descriptors for
long-ago deleted files from dropped tables and indexes, preventing
their disk space from being freed for reuse.

I think the checkpoint writer process learns about deleted files from
the "absorb fsync" mechanism and promptly closes those handles. But
since the bgwriter process was split from the checkpoint writer, the
bgwriter no longer benefits from this mechanism.

It looks like it hold the file handles until either is it is killed
and restarted, or until they get flushed out of vfd cache (which holds
~1000 files on my machine, so that can be a long time and lot of disk
space).

I don't know if this is a bug exactly, but it seems pretty unfortunate.

Cheers,

Jeff

On 1/8/12, Alex Shulgin <ash(at)commandprompt(dot)com> wrote:
>
> Jeff Janes <jeff(dot)janes(at)gmail(dot)com> writes:
>
>> It looks like it hold the file handles until either is it is killed
>> and restarted, or until they get flushed out of vfd cache (which holds
>> ~1000 files on my machine, so that can be a long time and lot of disk
>> space).
>>
>> I don't know if this is a bug exactly, but it seems pretty unfortunate.
>
> I wonder if this is similar to what we've experienced here (follow the
> thread for the actual patches):
> http://archives.postgresql.org/pgsql-hackers/2011-06/msg00295.php
>
> Maybe you're running a version still affected by that bug?

Hi Alex,

I'm running git HEAD. Sorry, I should have stated that.

Cheers,

Jeff

On Sat, Jan 7, 2012 at 7:19 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:

> I don't know if this is a bug exactly, but it seems pretty unfortunate.

I'll call it a bug and I think it's mine. I've added it to the list of
open items for 9.2

Thanks for testing.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services