Re: High context switches occurring

I ran a bit exhaustive pgbench on 2 test machines I have (quad dual core
Intel and Opteron). Ofcourse the Opteron was much faster, but
interestingly, it was experiencing 3x more context switches than the
Intel box (upto 100k, versus ~30k avg on Dell). Both are RH4.0
64bit/PG8.1 64bit.

Sun (v40z):
-bash-3.00$ time pgbench -c 1000 -t 30 pgbench
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 1000
number of transactions per client: 30
number of transactions actually processed: 30000/30000
tps = 45.871234 (including connections establishing)
tps = 46.092629 (excluding connections establishing)

real 10m54.240s
user 0m34.894s
sys 3m9.470s

Dell (6850):
-bash-3.00$ time pgbench -c 1000 -t 30 pgbench
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 1
number of clients: 1000
number of transactions per client: 30
number of transactions actually processed: 30000/30000
tps = 22.088214 (including connections establishing)
tps = 22.162454 (excluding connections establishing)

real 22m38.301s
user 0m43.520s
sys 5m42.108s

Thanks,
Anjan

-----Original Message-----
From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Tuesday, November 22, 2005 2:42 PM
To: Anjan Dave
Cc: Vivek Khera; Postgresql Performance
Subject: Re: [PERFORM] High context switches occurring

"Anjan Dave" <adave(at)vantage(dot)com> writes:
> Would this problem change it's nature in any way on the recent
Dual-Core
> Intel XEON MP machines?

Probably not much.

There's some evidence that Opterons have less of a problem than Xeons
in multi-chip configurations, but we've seen CS thrashing on Opterons
too. I think the issue is probably there to some extent in any modern
SMP architecture.

regards, tom lane

On Dec 6, 2005, at 2:04 PM, Anjan Dave wrote:

> interestingly, it was experiencing 3x more context switches than the
> Intel box (upto 100k, versus ~30k avg on Dell). Both are RH4.0

I'll assume that's context switches per second... so for the opteron
that's 65400000 cs's and for the Dell that's 40740000 switches during
the duration of the test. Not so much a difference...

You see, the opteron was context switching more because it was doing
more work :-)

"Anjan Dave" <adave(at)vantage(dot)com> writes:
> -bash-3.00$ time pgbench -c 1000 -t 30 pgbench
> starting vacuum...end.
> transaction type: TPC-B (sort of)
> scaling factor: 1
> number of clients: 1000
> number of transactions per client: 30
> number of transactions actually processed: 30000/30000
> tps = 45.871234 (including connections establishing)
> tps = 46.092629 (excluding connections establishing)

I can hardly think of a worse way to run pgbench :-(. These numbers are
about meaningless, for two reasons:

1. You don't want number of clients (-c) much higher than scaling factor
(-s in the initialization step). The number of rows in the "branches"
table will equal -s, and since every transaction updates one
randomly-chosen "branches" row, you will be measuring mostly row-update
contention overhead if there's more concurrent transactions than there
are rows. In the case -s 1, which is what you've got here, there is no
actual concurrency at all --- all the transactions stack up on the
single branches row.

2. Running a small number of transactions per client means that
startup/shutdown transients overwhelm the steady-state data. You should
probably run at least a thousand transactions per client if you want
repeatable numbers.

Try something like "-s 10 -c 10 -t 3000" to get numbers reflecting test
conditions more like what the TPC council had in mind when they designed
this benchmark. I tend to repeat such a test 3 times to see if the
numbers are repeatable, and quote the middle TPS number as long as
they're not too far apart.

regards, tom lane

Tom Lane wrote:
> "Anjan Dave" <adave(at)vantage(dot)com> writes:
> > -bash-3.00$ time pgbench -c 1000 -t 30 pgbench
> > starting vacuum...end.
> > transaction type: TPC-B (sort of)
> > scaling factor: 1
> > number of clients: 1000
> > number of transactions per client: 30
> > number of transactions actually processed: 30000/30000
> > tps = 45.871234 (including connections establishing)
> > tps = 46.092629 (excluding connections establishing)
>
> I can hardly think of a worse way to run pgbench :-(. These numbers are
> about meaningless, for two reasons:
>
> 1. You don't want number of clients (-c) much higher than scaling factor
> (-s in the initialization step). The number of rows in the "branches"
> table will equal -s, and since every transaction updates one

Should we throw a warning when someone runs the test this way?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> 1. You don't want number of clients (-c) much higher than scaling factor
>> (-s in the initialization step).

> Should we throw a warning when someone runs the test this way?

Not a bad idea (though of course only for the "standard" scripts).
Tatsuo, what do you think?

regards, tom lane

> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> 1. You don't want number of clients (-c) much higher than scaling factor
> >> (-s in the initialization step).
>
> > Should we throw a warning when someone runs the test this way?
>
> Not a bad idea (though of course only for the "standard" scripts).
> Tatsuo, what do you think?

That would be annoying since almost every users will get the kind of
warnings. What about improving the README?
--
Tatsuo Ishii
SRA OSS, Inc. Japan

On Tue, 2005-12-06 at 22:49, Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> 1. You don't want number of clients (-c) much higher than scaling factor
> >> (-s in the initialization step).
>
> > Should we throw a warning when someone runs the test this way?
>
> Not a bad idea (though of course only for the "standard" scripts).
> Tatsuo, what do you think?

Just to clarify, I think the pgbench program should throw the warning,
not postgresql itself. Not sure if that's what you were meaning or
not. Maybe even have it require a switch to run in such a mode, like a
--yes-i-want-to-run-a-meaningless-test switch or something.