Re: slow commits with heavy temp table usage in 8.4.0

From: "Todd A(dot) Cook" <tcook(at)blackducksoftware(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: slow commits with heavy temp table usage in 8.4.0
Date: 2009-08-06 17:32:44
Message-ID: 4A7B13BC.6020606@blackducksoftware.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
>
> I took a look through the CVS history and verified that there were
> no post-8.4 commits that looked like they'd affect performance in
> this area. So I think it's got to be a platform difference not a
> PG version difference. In particular I think we are probably looking
> at a filesystem issue: how fast can you delete [...] 30000 files.

I'm still on Fedora 7, so maybe this will be motivation to upgrade.

FYI, on my 8.2.13 system, the test created 30001 files which were all
deleted during the commit. On my 8.4.0 system, the test created 60001
files, of which 30000 were deleted at commit and 30001 disappeared
later (presumably during a checkpoint?).

> But I'm not sure
> if the use-case is popular enough to deserve such a hack.

FWIW, the full app was looping over a set of datasets. On each iteration,
it computed some intermediate results into a temp table, generated several
reports from those intermediate results, and finally truncated the table
for the next iteration.

-- todd

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Todd A. Cook 2009-08-06 17:34:04 Re: slow commits with heavy temp table usage in 8.4.0
Previous Message Kevin Grittner 2009-08-06 17:16:46 Re: the case for machine-readable error fields