pg_stat_statements: calls under-estimation propagation

From: Daniel Farina <drfarina(at)acm(dot)org>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_stat_statements: calls under-estimation propagation
Date: 2012-12-28 11:43:02
Message-ID: CACN56+NLMTwHg8eQQqNYzqe2Q0nEGJoKmGFiUSK_aoHw627Q8Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

After long delay (sorry) here's a patch implementing what was
hand-waved at in
http://archives.postgresql.org/pgsql-hackers/2012-10/msg00176.php

I am still something at a loss at how to test it besides prodding it
by hand; it seems like it's going to involve infrastructure or
introducing hooks into pg_stat_statements for the express purpose.

The patch can also be sourced from:

https://github.com/fdr/postgres.git error-prop-pg_stat_statements

Without further ado, the cover letter taken from the top of the patch:

This tries to establish a maximum under-estimate of the number of
calls for a given pg_stat_statements entry. That means the number of
calls to the canonical form of the query is between 'calls' and 'calls
+ calls_underest'.

This is useful to determine when accumulating statistics if a given
record is bouncing in and out of the pg_stat_statements table, having
its ncalls reset all the time, but also having calls_underest grow
very rapidly.

Records that always stay in pg_stat_statements will have a
calls_underest that do not change at all.

An interesting case is when a query that usually is called is not
called for a while, and falls out of pg_stat_statements. The result
can be that the query with the most 'calls' can also have more
uncertainty than the query with the second most calls, which is also
exactly the truth in reality.

Unceremoniously bundled into this patch is a reduction in the minimum
table size for pg_stat_statements, from 100 to 1. Using tiny values
is not likely to be seen in production, but makes testing the patch a
lot easier in some situations.

I will add this to the commitfest.

--
fdr

Attachment Content-Type Size
add-pg_stat_statements-calls-underestimation-v1.patch application/octet-stream 9.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2012-12-28 11:44:42 Re: enhanced error fields
Previous Message Amit Kapila 2012-12-28 11:28:41 Re: Performance Improvement by reducing WAL for Update Operation