Exposing currentTransactionWALVolume

Short patch to expose a function GetCurrentTransactionWALVolume() that
gives the total number of bytes written to WAL by current transaction.

User interface to this information discussed on separate thread, so
that we don't check the baby out with the bathwater when people
discuss UI pros and cons.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Jan 15, 2014 at 2:12 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> Short patch to expose a function GetCurrentTransactionWALVolume() that
> gives the total number of bytes written to WAL by current transaction.

Could you tell me the use case of this function? ISTM that it's less
useful in the real
world because it reports the information of WAL generated only in my own current
transaction. For example, the monitoring tool cannot see that
information because
it's the different session from the target.

Regards,

--
Fujii Masao

I send you my review comment.

2014-01-15 Simon Riggs <simon(at)2ndquadrant(dot)com>:

> Short patch to expose a function GetCurrentTransactionWALVolume() that
> gives the total number of bytes written to WAL by current transaction.
>
* It's simple and good feature. It is useful for system management, and
forecasting
server spec(especially disk volume) on the system needed.

* Overhead is nothing unless my test.

* Compile and unit tests are no problem.

User interface to this information discussed on separate thread, so
> that we don't check the baby out with the bathwater when people
> discuss UI pros and cons.
>
Did you get good opinion in other thread?
I'd like to use seeing WAL volume sql and init value of WAL volume sql.
Your patch seems to init the value when start up the server now.
If we have init function, we can see database activities in each hours in a
day from WAL volumes.
Now, we only see number of transactions and database volumes.
I'd like to see more detail activities from WAL volume in each minutes or
hours.
It might be good for performance improvement by hackers, too

Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center

On 31 January 2014 13:56, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Wed, Jan 15, 2014 at 2:12 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>> Short patch to expose a function GetCurrentTransactionWALVolume() that
>> gives the total number of bytes written to WAL by current transaction.
>
> Could you tell me the use case of this function? ISTM that it's less
> useful in the real
> world because it reports the information of WAL generated only in my own current
> transaction. For example, the monitoring tool cannot see that
> information because
> it's the different session from the target.

It's already possible to work out how much total WAL has been
generated. Monitoring tools may access that.

What this allows is user/plugin code to see how much WAL has been
generated in this transaction and take different user defined actions.
Those actions would be specific to the transaction, so this is more
about production control applications than overall system monitoring.
There are many possible ways to use that knowledge.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Hi Simon,

On 2014-01-14 17:12:35 +0000, Simon Riggs wrote:
> /*
> - * MarkCurrentTransactionIdLoggedIfAny
> + * ReportTransactionInsertedWAL
> *
> - * Remember that the current xid - if it is assigned - now has been wal logged.
> + * Remember that the current xid - if it is assigned - has now inserted WAL
> */
> void
> -MarkCurrentTransactionIdLoggedIfAny(void)
> +ReportTransactionInsertedWAL(uint32 insertedWALVolume)
> {
> + currentTransactionWALVolume += insertedWALVolume;
> if (TransactionIdIsValid(CurrentTransactionState->transactionId))
> CurrentTransactionState->didLogXid = true;
> }

Not a big fan of combining those two. One works on the toplevel
transaction, the other on the current subtransaction... The new name
also ignores that it's only taking effect if there's actually a
transaction in progress.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

(2014/02/15 23:04), Andres Freund wrote:
> Hi Simon,
>
> On 2014-01-14 17:12:35 +0000, Simon Riggs wrote:
>> /*
>> - * MarkCurrentTransactionIdLoggedIfAny
>> + * ReportTransactionInsertedWAL
>> *
>> - * Remember that the current xid - if it is assigned - now has been wal logged.
>> + * Remember that the current xid - if it is assigned - has now inserted WAL
>> */
>> void
>> -MarkCurrentTransactionIdLoggedIfAny(void)
>> +ReportTransactionInsertedWAL(uint32 insertedWALVolume)
>> {
>> + currentTransactionWALVolume += insertedWALVolume;
>> if (TransactionIdIsValid(CurrentTransactionState->transactionId))
>> CurrentTransactionState->didLogXid = true;
>> }
>
> Not a big fan of combining those two. One works on the toplevel
> transaction, the other on the current subtransaction... The new name
> also ignores that it's only taking effect if there's actually a
> transaction in progress.
Oh, yes. I don't have good idea, but we need to change function name or add new
function for WAL adding volume. If it will be fixed, I set ready for commiter,
because I cannot see any bad point in this patch.

Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center