Re: [PATCH] pgbench --throttle (submission 7 - with lag measurement)

Hello Tatsuo,

> Now I have question regarding the function.
>
> ./pgbench -p 5433 -S -T 10 -R 10000 test
> tps = 7133.745911 (including connections establishing)
>
> What does "average rate limit lag" mean? From the manual:
> [...]
> So in my understanding the number shows the delay time before *each*
> transaction starts.

... with respect to the schedule time assigned by the rate-limiting
stochastic process. This is to detect that rate limiting does not work
properly.

> If my understanding is correct, why 71339 (total transactions) * 862.534
> ms = 61532 sec could exceed 10 seconds, which is the total run time?

It is possible, because each transaction is far behind schedule, and you
cumulate the lateness.

Say you have transactions schedules every 0.1 second, but they take 2
second to complete:

1. scheduled at 0.0, start at 0.0
2. scheduled at 0.1, start at 2.0, 1.9 second lag
3. scheduled at 0.2, start at 4.0, 3.8 second lag, cumulative lag 5.7 s
4. scheduled at 0.3, start at 6.0, 5.7 second lag, cumulative lag 11.4 s
5. scheduled at 0.4, start at 8.0, 7.6 second lag, cumulative lag 19.0 s
6. scheduled at 0.5, never starts

If we stop at 10.0 seconds, 5 transaction have been processed, the average
lag is about 3.8 seconds, the cumulative lag is 19.0 seconds. The lag of a
given transaction can cover lag from previous ones.

Basically, if the lag is anything but small, it means that the database
cannot handle the load and that something is amiss. In your example you
required 10000 tps, but the database can only handle 7000 tps.

Note that the database could catchup at some point, say it usually can
handle more that 10000 tps, but while the database dump is running it
falls far behing schedule, and then one the dump is done it goes back to
nominal and "late" transactions are finally processed. The max lag would
show that something was amiss during the bench, even if the average lag
is quite low.

> Also I noticed small bug.
>
> ./pgbench -R 0 test
> invalid rate limit: 0
>
> Shouldn't this be treated as if -R is not specified? Actually in the program:
>
> /*
> * When threads are throttled to a given rate limit, this is the target delay
> * to reach that rate in usec. 0 is the default and means no throttling.
> */
> int64 throttle_delay = 0;
>
> So it seems treating "-R 0" means "no throttling" makes more sense to me.

Note that the rate is expressed in tps which make sense to users, but the
internal variable is in usec which is more useful for scheduling, and is
the inverse of the other.

So -R 0 would mean zero tps, that is an infinite delay, but a 0 delay
would require an infinite tps. As requiring 0 tps does not make sense, I
decided to disable that. If you really fill that "-R 0" should mean
disable the feature, I'm fine with that, but this is not exactly logical
wrt tps.

--
Fabien.