Re: PG_DUMP very slow because of STDOUT ??

From: Andras Fabian <Fabian(at)atrada(dot)net>
To: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Re: PG_DUMP very slow because of STDOUT ??
Date: 2010-07-13 07:53:07
Message-ID: B1A1AD14D5F9D647BD2A00988C53B8220ACA307B@atradaex03.nbg.atrada.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi Scott,

No, we didn't have a kernel update (it is still the stock Ubuntu 10.04 Server kernel ... 2.6.32.2). And in the meantime - this morning - I have discovered, that the rebooted server is again slowing down! It is not at the level of the not-rebooted-server (about 45 mins for the 3 Gig file)... it "only" needs 22 minutes, but it is already quite a bit away from the optimum of 3 minutes (or less). So, definitely, something is "deteriorating" in the system ...

And I also did dome readings with iostat -xd 5 ... And the target drive to which the output of the STDOUT is directed is below 1% utilization (mostly around 0.2 - 0.4%) with rare "peaks around 2-3% when it does a little bit more. And this is maybe one of the interesting observations. It seems to periodically "flush" a bit more out, just to fall asleep again (with minimum write activity). The drive, from which the reads come (the one, where PG-s data files are ... it is the 8-disk RAID 10), has a little bit more activity (utilization 6-8%) but this data is also - concurrently - in use by some apps reading from the DB (just, normal traffic on the DB).

Andras Fabian

-----Ursprüngliche Nachricht-----
Von: Scott Marlowe [mailto:scott(dot)marlowe(at)gmail(dot)com]
Gesendet: Dienstag, 13. Juli 2010 09:45
An: Andras Fabian
Cc: Tom Lane; pgsql-general(at)postgresql(dot)org
Betreff: Re: [GENERAL] PG_DUMP very slow because of STDOUT ??

On Tue, Jul 13, 2010 at 12:31 AM, Andras Fabian <Fabian(at)atrada(dot)net> wrote:
> Hi Scott,
>
> Although I can't guarantee for 100% that there was no RAID rebuild at some point, I am almost sure that it wasn't the case. Two machines - the ones which were already in production - exhibited this problem. Both of them were already up for some weeks. Now, the reboot rather "fixed" one of them instead of making it worse (as your theory goes this way) the problem "disappeared" (but I don't know for how long). Now, only one of the production machines has the issue ... the one which wasn't rebooted. Strange, strange. Nevertheless thank you for your idea ... this is exactly the way I try to approach the problem, by making some theories and trying to prove or disapprove them :-)
> Now I will try to further investigate along the tips from Craig and Greg.

Was there maybe a kernel update that hadn't been applied by reboot?

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Josip Rodin 2010-07-13 07:55:53 Re: simple functions, huge overhead, no cache
Previous Message Scott Marlowe 2010-07-13 07:45:12 Re: PG_DUMP very slow because of STDOUT ??