Re: Add min and max execute statement time in pg_stat_statement

(2014/01/23 12:00), Andrew Dunstan wrote:
>
> On 01/22/2014 08:28 PM, KONDO Mitsumasa wrote:
>> (2014/01/22 22:26), Robert Haas wrote:
>>> On Wed, Jan 22, 2014 at 3:32 AM, KONDO Mitsumasa
>>> <kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>>>>> OK, Kondo, please demonstrate benchmarks that show we have <1% impact
>>>>> from this change. Otherwise we may need a config parameter to allow
>>>>> the calculation.
>>>>
>>>> OK, testing DBT-2 now. However, error range of benchmark might be 1% higher.
>>>> So I show you detail HTML results.
>>>
>>> To see any impact from spinlock contention, I think you're pretty much
>>> going to need a machine with >32 cores, I think, and lots of
>>> concurrency. pgbench -S is probably a better test than DBT-2, because
>>> it leaves out all the writing, so percentage-wise more time will be
>>> spent doing things like updating the pgss hash table.
>> Oh, thanks to inform me. I think essential problem of my patch has bottle neck
>> in sqrt() function and other division caluculation. I will replcace sqrt()
>> function in math.h to more faster algorithm. And moving unneccessary part of
>> caluculation in LWlocks or other locks. It might take time to improvement, so
>> please wait for a while.
>>
>
> Umm, I have not read the patch, but are you not using Welford's method? Its
> per-statement overhead should be absolutely tiny (and should not compute a square
> root at all per statement - the square root should only be computed when the
> standard deviation is actually wanted, e.g. when a user examines
> pg_stat_statements) See for example
> <http://www.johndcook.com/standard_deviation.html>
Thanks for your advice. I read your example roughly, however, I think calculating
variance is not so heavy in my patch. Double based sqrt caluculation is most
heavily in my mind. And I find fast square root algorithm that is used in 3D games.
http://en.wikipedia.org/wiki/Fast_inverse_square_root

This page shows inverse square root algorithm, but it can caluculate normal
square root, and it is faster algorithm at the price of precision than general
algorithm. I think we want to fast algorithm, so it will be suitable.

Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center