Re: pgbench throttling latency limit

On 09/11/2014 05:16 PM, Fabien COELHO wrote:
>
>> How should skipped transactions should be taken into account in the log file
>> output, with and without aggregation? I assume we'll want to have some trace
>> of skipped transactions in the logs.
>
> The problem with this point is that how to report something "not done" is
> unclear, especially as the logic of the log is one line per performed
> transaction.
>
> Obviously we can log something, but as the transaction are not performed
> the format would be different, which break the expectation of a simple and
> homogeneous log file format that people like to process directly.
>
> So bar any great idea, I would suggest not to log skipped transactions and
> to wait for someone to need to have access to this detailed information
> and for whom the final report is not enough.

We have to come up with something. The point of the log file is that it
contains all the information you need to build your own graphs, when
pgbench's built-in reporting features are not enough. If it doesn't
contain detailed information about skipped transactions, a report based
on the log file would be inaccurate, or at least misleading.

How about printing a line in the log for every skipped transaction, with
the string "skipped" in place of the latency. The "completion time" can
show the time when the transaction was skipped, and the lag can show the
difference between the scheduled time and the time it was skipped. Or
put another way, print a line as if the transaction completed
immediately, but with the "skipped" string in the latency field.

The "skipped" string will trip a program that doesn't expect that, but
since this is a new feature that you have to enable manually, that's OK.

The output would look something like this (modified from the manual's
example by hand, so the numbers don't add up):

0 199 2241 0 1175850568 995598 1020
0 200 2465 0 1175850568 998079 1010
0 201 skipped 1175850569 608 3011
0 202 skipped 1175850569 608 2400
0 203 skipped 1175850569 608 1000
0 204 2513 0 1175850569 608 500
0 205 2038 0 1175850569 2663 500

- Heikki