From: | Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc> |
---|---|
To: | Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: Review: Revise parallel pg_restore's scheduling heuristic |
Date: | 2009-07-19 13:07:42 |
Message-ID: | 4A631A9E.5090601@kaltenbrunner.cc |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Kevin Grittner wrote:
> "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:
>
>> Performance tests to follow in a day or two.
>
> I'm looking to beg another week or so on this to run more tests. What
> I can have by the end of today is pretty limited, mostly because I
> decided it made the most sense to test this with big complex
> databases, and it just takes a fair amount of time to throw around
> that much data. (This patch didn't seem likely to make a significant
> difference on smaller databases.)
>
> My current plan is to test this on a web server class machine and a
> distributed application class machine. Both database types have over
> 300 tables with tables with widely ranging row counts, widths, and
> index counts.
>
> It would be hard to schedule the requisite time on our biggest web
> machines, but I assume an 8 core 64GB machine would give meaningful
> results. Any sense what numbers of parallel jobs I should use for
> tests? I would be tempted to try 1 (with the -1 switch), 8, 12, and
> 16 -- maybe keep going if 16 beats 12. My plan here would be to have
> the dump on one machine, and run pg_restore there, and push it to a
> database on another machine through the LAN on a 1Gb connection.
> (This seems most likely to be what we'd be doing in real life.) I
> would run each test with the CVS trunk tip with and without the patch
> applied. The database is currently 1.1TB.
you need to be careful here - in my latest round of benchmarking I had
actually test with the workload generator on the same box because on
fast boxes we can easily achive >100MB/s total load rate these days.
At these load rates you are very close or over the pratical limits of
GigE...
Stefan
From | Date | Subject | |
---|---|---|---|
Next Message | Stefan Kaltenbrunner | 2009-07-19 13:13:15 | Re: [PATCH v3] Avoid manual shift-and-test logic in AllocSetFreeIndex |
Previous Message | Dimitri Fontaine | 2009-07-19 12:53:38 | Re: Sampling profiler updated |