pgbench progress report improvements

From: Fabien <coelho(at)cri(dot)ensmp(dot)fr>
To: PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org>
Subject: pgbench progress report improvements
Date: 2013-08-06 08:47:14
Message-ID: alpine.DEB.2.02.1308061042030.29999@localhost6.localdomain6
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here is a patch submission for reference to the next commitfest.

Improve pgbench measurements & progress report

- Use progress option both under init & bench.

Activate progress report by default, every 5 seconds.
When initializing, --quiet reverts to the old every 100,000 insertions
behavior...

- Measure transaction latency instead of computing it.

The previous computed figure does not make sense under throttling,
as sleep throttling time was included in the figures.
The latency and its standard deviation are printed out under progress
and in the final report.

- Take thread start time at the beginning of the thread.

Otherwise it includes pretty slow thread/fork system start times in
the measurements. May help with bug #8326.

- Reduce and compensate throttling underestimation bias.

This is a consequence of relying on an integer random generator.
It was about 0.5% with 1000 distinct values. Multiplier added to
compensate the 0.05% bias with 10000 distinct integer values.

- Updated documentation & help message.

Sample output under initialization with --progress=1

creating tables...
223000 of 1000000 tuples (22%) done (elapsed 1.01 s, remaining 3.52 s).
430200 of 1000000 tuples (43%) done (elapsed 2.03 s, remaining 2.68 s).
507300 of 1000000 tuples (50%) done (elapsed 3.44 s, remaining 3.34 s).
571400 of 1000000 tuples (57%) done (elapsed 4.05 s, remaining 3.03 s).
786800 of 1000000 tuples (78%) done (elapsed 5.03 s, remaining 1.36 s).
893200 of 1000000 tuples (89%) done (elapsed 6.02 s, remaining 0.72 s).
1000000 of 1000000 tuples (100%) done (elapsed 6.93 s, remaining 0.00 s).
...

Sample output under benchmarking with --progress=1

progress: 1.0 s, 116.7 tps, 8.528 +- 1.217 ms lat
progress: 2.0 s, 117.7 tps, 8.493 +- 1.165 ms lat
progress: 3.0 s, 115.5 tps, 8.654 +- 1.650 ms lat
progress: 4.0 s, 116.8 tps, 8.559 +- 1.365 ms lat
progress: 5.0 s, 111.7 tps, 8.947 +- 3.475 ms lat
progress: 6.0 s, 116.7 tps, 8.563 +- 1.387 ms lat
progress: 7.0 s, 116.6 tps, 8.572 +- 1.474 ms lat
...
# oops, things happen:
progress: 36.0 s, 10.9 tps, 91.864 +- 124.874 ms lat
progress: 37.0 s, 115.2 tps, 8.678 +- 1.792 ms lat
...

--
Fabien.

Attachment Content-Type Size
pgbench-measurements-v1.patch text/x-diff 20.9 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dimitri Fontaine 2013-08-06 09:14:33 Re: Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])
Previous Message Sergey Konoplev 2013-08-06 07:44:16 Re: System catalog vacuum issues