Re: Some help on buffers and other performance tricks

0= Optimize your schema to be a tight as possible. Your goal is to give yourself the maximum chance that everything you want to work on is in RAM when you need it.
1= Upgrade your RAM to as much as you can possibly strain to afford. 4GB at least. It's that important.
2= If the _entire_ DB does not fit in RAM after upgrading your RAM, the next step is making sure your HD IO subsystem is adequate to your needs.
3= Read the various pg tuning docs that are available and Do The Right Thing.
4= If performance is still not acceptable, then it's time to drill down into what specific actions/queries are problems.
If you get to here and the entire DBMS is still not close to acceptable, your fundamental assumptions have to be re-examined.

Ron

-----Original Message-----
From: Yves Vindevogel <yves(dot)vindevogel(at)implements(dot)be>
Sent: Nov 9, 2005 3:11 PM
To: pgsql-performance(at)postgresql(dot)org
Subject: [PERFORM] Some help on buffers and other performance tricks

Hi all,

I've got PG 8.0 on Debian sarge set up ...
I want to speed up performance on the system.

The system will run PG, Apache front-end on port 80 and Tomcat / Cocoon
for the webapp.
The webapp is not so heavily used, so we can give the max performance
to the database.
The database has a lot of work to do, we upload files every day.
The current server has 8 databases of around 1 million records. This
will be more in the future.
There's only one main table, with some smaller tables. 95% of the
records are in that one table.
A lot of updates are done on that table, affecting 10-20% of the
records.

The system has 1 gig of ram. I could give 512Mb to PG.
Filesystem is ext2, with the -noatime parameter in fstab

Could I get some suggestions in how to configure my buffers, wals, ....
?

Met vriendelijke groeten,
Bien � vous,
Kind regards,

Yves Vindevogel
Implements

Ron Peacetree wrote:
> 0= Optimize your schema to be a tight as possible. Your goal is to give yourself the maximum chance that everything you want to work on is in RAM when you need it.
> 1= Upgrade your RAM to as much as you can possibly strain to afford. 4GB at least. It's that important.
> 2= If the _entire_ DB does not fit in RAM after upgrading your RAM, the next step is making sure your HD IO subsystem is adequate to your needs.
> 3= Read the various pg tuning docs that are available and Do The Right Thing.
> 4= If performance is still not acceptable, then it's time to drill down into what specific actions/queries are problems.
> If you get to here and the entire DBMS is still not close to acceptable, your fundamental assumptions have to be re-examined.

IMHO you should really be examining your queries _before_ you do any
investment in hardware, because later those may prove unnecessary.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Wed, 9 Nov 2005 20:07:52 -0300
Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:

> IMHO you should really be examining your queries _before_ you do any
> investment in hardware, because later those may prove unnecessary.

It all really depends on what you're doing. For some of the systems
I run, 4 GBs of RAM is *WAY* overkill, RAID 1+0 is overkill, etc.

In general I would slightly change the "order of operations" from:

1) Buy tons of RAM
2) Buy lots of disk I/O
3) Tune your conf
4) Examine your queries

to

1) Tune your conf
2) Spend a few minutes examining your queries
3) Buy as much RAM as you can afford
4) Buy as much disk I/O as you can
5) Do in depth tuning of your queries/conf

Personally I avoid planning my schema around my performance at
the start. I just try to represent the data in a sensible,
normalized way. While I'm sure I sub-consciously make decisions
based on performance considerations early on, I don't do any major
schema overhauls until I find I can't get the performance I need
via tuning.

Obviously there are systems/datasets/quantities where this won't
always work out best, but for the majority of systems out there
complicating your schema, maxing your hardware, and THEN tuning
is IMHO the wrong approach.

---------------------------------
Frank Wiles <frank(at)wiles(dot)org>
http://www.wiles.org
---------------------------------

Frank Wiles wrote:

> Obviously there are systems/datasets/quantities where this won't
> always work out best, but for the majority of systems out there
> complicating your schema, maxing your hardware, and THEN tuning
> is IMHO the wrong approach.

I wasn't suggesting to complicate the schema -- I was actually thinking
in systems where some queries are not using indexes, some queries are
plain wrong, etc. Buying a very expensive RAID and then noticing that
you just needed to create an index, is going to make somebody feel at
least somewhat stupid.

--
Alvaro Herrera Valdivia, Chile ICBM: S 39º 49' 17.7", W 73º 14' 26.8"
Y una voz del caos me habló y me dijo
"Sonríe y sé feliz, podría ser peor".
Y sonreí. Y fui feliz.
Y fue peor.

On Wed, 9 Nov 2005 21:43:33 -0300
Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:

> Frank Wiles wrote:
>
> > Obviously there are systems/datasets/quantities where this won't
> > always work out best, but for the majority of systems out there
> > complicating your schema, maxing your hardware, and THEN tuning
> > is IMHO the wrong approach.
>
> I wasn't suggesting to complicate the schema -- I was actually
> thinking in systems where some queries are not using indexes, some
> queries are plain wrong, etc. Buying a very expensive RAID and then
> noticing that you just needed to create an index, is going to make
> somebody feel at least somewhat stupid.

Sorry I was referring to Ron statement that the first step should
be to "Optimize your schema to be as tight as possible."

But I agree, finding out you need an index after spending $$$ on
extra hardware would be bad. Especially if you have to explain it
to the person forking over the $$$! :)

---------------------------------
Frank Wiles <frank(at)wiles(dot)org>
http://www.wiles.org
---------------------------------