Re: linux server configuration

We are about to build a new database server, our plan is to use Debian.

Is there documentation of recommended server configurations for Linux,
such as kernel parameters, preferred file system, etc that work best
with postgresql?

I'm not talking about the pg configuration, which I have seen a lot of
documentation about, more on getting the OS ready.

Thanks

Sim

I don't think that it makes sense to look at PG tuning and server
tuning as two separate tasks. XFS was recently benchmarked using
bonnie++ by Greg Smith, with interesting results:

http://blog.2ndquadrant.com/en/2010/04/the-return-of-xfs-on-linux.html

That said, my guess is that the majority of large Linux installations
are using ext3 or ext4. It isn't the case that there's a favoured
filesystem for Postgres.

The kernel parameters that you'll want to change are sysV IPC parameters:

http://www.postgresql.org/docs/current/static/kernel-resources.html

If you want a guide to choosing hardware for Postgres, I can highly
recommend Greg Smith's new book, "Postgresql 9 High performance".

--
Regards,
Peter Geoghegan

On Thu, Jan 6, 2011 at 4:20 AM, Sim Zacks <sim(at)compulab(dot)co(dot)il> wrote:
> We are about to build a new database server, our plan is to use Debian.
>
> Is there documentation of recommended server configurations for Linux, such
> as kernel parameters, preferred file system, etc that work best with
> postgresql?

This really depends on your hardware. If you're running a battery
backed caching RAID controller it's a good idea to lobotamize the io
scheduler since it just gets in the way at that point:

echo noop > /sys/block/sda/queue/scheduler

On our machines with 128G of ram, we have occasionally seen swap
storms for no apparent reason, and have turned off swap.

/sbin/swapoff -a

On a machine with 16 or more cores, the linux kernel may decide to use
the zone reclaim mode, which is supposed to keep memory pinned in the
bank accessible by a certain cpu / core for things like
virtualization. This leads to disasterous performance on file servers
and pg servers. Turn it off in /etc/sysctl.conf:

vm.zone_reclaim_mode = 0

For most uses, linux doesn't need a lot of tuning really. Turning the
scheduler to noop gained us a bit of performance especially as load
increases it's more noticeable that the OS is no longer trying to do
the RAID controller's job. If you want to hunt for more tips, search
through Greg Smith's site, or buy his excellent pg 9.0 performance
book. It'll keep you occupied for days and days with all the cool
stuff you can do to make fast pg machines.

As a followup, I'd like to point out that you can probably get more
performance wise from hardware upgrades than from tuning your OS.
Something as simple as an $800 caching RAID controller can make a
workstation class machine into a monster performer, going from 250 tps
to 3000 tps with one simple change. Tuning might get you from 250 to
275. Hardware upgrades can take you much much further. Memory
upgrades are cheap (I just ordered 8 Gigs for my laptop for $89). 4G
ECC RAM is pretty cheap for servers too. Any newly minted database
server with less than 16 gigs of ram is inexcusable today. 32 is
really the minimum I'd throw at a new server now. Also, very fast 8
or 12 core cpus, both from intel and amd, are cheap too. An 8 core
2.0GHz Magny Cours on newegg is only $275. Put a pair of those in a
db server with 32 Gigs ram and a decent caching RAID controller and a
handful of decent hard drives and you don't really need to do a lot of
tuning. Even a 4 drive machine can give a good account of itself with
that hardware.

What I'm trying to say is don't spend tons of time tuning slow
hardware, you'll get at most a few % gain.

On Thu, Jan 6, 2011 at 9:38 AM, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:
> As a followup, I'd like to point out that you can probably get more
> performance wise from hardware upgrades than from tuning your OS.
> Something as simple as an $800 caching RAID controller can make a
>

Totally agree here. Throwing hardware at the problem is a very cost
effective solution to performance issues! :)

On Thu, Jan 6, 2011 at 6:20 AM, Sim Zacks <sim(at)compulab(dot)co(dot)il> wrote:
> We are about to build a new database server, our plan is to use Debian.
>
> Is there documentation of recommended server configurations for Linux, such
> as kernel parameters, preferred file system, etc that work best with
> postgresql?
>
> I'm not talking about the pg configuration, which I have seen a lot of
> documentation about, more on getting the OS ready.

My company has two identical PostgreSQL servers running on Debian
(Squeeze) & we didn't tune the kernel and left it as it was installed
by Debian. I partitioned the drives as 'ext4' & and the data is stored
on a iSCSI NAS (RAID 5) configuration. The servers have been rock
solid. Just be super careful that you don't blindly upgrade the server
(using apt-get upgrade) and accidentally swap from 8.4 > 9.0. I ran
into this problem and it was very messy. This is a PG issue however,
not a Debian issue. On a side note I've ran PostgreSQL 8.4 in a
production environment on the following:

- CentOS 5 64-bit
- RHEL 5 64-bit
- Arch Linux 64-bit
- Ubuntu 10.04 Server
- Slackware Linux 13 64-bit
- Gentoo Linux (Uggh) 64-bit

Out of all those distributions, I can honestly say that between Arch
Linux and Debian, no other distribution comes close.