Re: How to generate specific WAL records?

Hi,

Does anyone know how to generate the following WAL records from psql?

I'm now fixing pg_lesslog, which I reported a bug. Now code is
almost okay and I'd like to add test tools to show pg_lesslog can
handle all the WAL record correctly.

XLOG_MULTIXACT_ZERO_OFF_PAGE
XLOG_MULTIXACT_ZERO_MEM_PAGE
XLOG_MULTIXACT_CREATE_ID
XLOG_HEAP_LOCK
XLOG_HEAP_INIT_PAGE
XLOG_BTREE_INSERT_META
XLOG_BTREE_DELETE_PAGE
XLOG_BTREE_DELETE_PAGE_META
XLOG_BTREE_DELETE_PAGE_HALF
XLOG_GIN_CREATE_PTREE
XLOG_GIN_SPLIT
XLOG_GIN_VACUUM_PAGE
XLOG_GIN_DELETE_PAGE
XLOG_GIN_DELETE_LISTPAGE
XLOG_GIST_PAGE_DELETE

So far, I'm using conventional BTREE, btree_gin and btree_gist for
the test, as well as 2PC and savepoint.

Any information is welcome.

Thank you very much in advance;
------
Koichi Suzuki

Koichi Suzuki escribió:
> Hi,
>
> Does anyone know how to generate the following WAL records from psql?
>
> I'm now fixing pg_lesslog, which I reported a bug. Now code is
> almost okay and I'd like to add test tools to show pg_lesslog can
> handle all the WAL record correctly.
>
> XLOG_MULTIXACT_ZERO_OFF_PAGE
> XLOG_MULTIXACT_ZERO_MEM_PAGE
> XLOG_MULTIXACT_CREATE_ID

For these, you need to do SELECT FOR SHARE of a tuple in more than one
concurrent session.

I think HEAP_LOCK is SELECT FOR UPDATE.

> XLOG_BTREE_DELETE_PAGE
> XLOG_BTREE_DELETE_PAGE_META
> XLOG_BTREE_DELETE_PAGE_HALF

These are caused during vacuum of a btree. Create an index, populate
it, then remove all items. Then vacuum twice.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro-san;

Thank you for a great advice. I successfully generated all the WAL
records listed below. By deleteing and VACUUMing a table with
btree_gist based index, I was successful to generate all the GIST WAL
records.

Still have the following WAL record to create:

CLOG_TRUNCATE
XLOG_HEAP_INIT_PAGE
XLOG_BTREE_INSERT_META
XLOG_GIN_CREATE_PTREE
XLOG_GIN_DELETE_PAGE

I have to run good amount of transactions before VACUUM FREEZE to
create CLOG_TRUNCATE. I'll try.

It's very helpful if anybody knows how to generate XLOG_HEAP_INIT_PAGE
and XLOG_BTREE_INSERT_META.

I'll ask Oleg-san how to generate GIN-related WALs.

Thank you very much;
----------
Koichi Suzuki

2010/4/14 Alvaro Herrera <alvherre(at)commandprompt(dot)com>:
> Koichi Suzuki escribió:
>> Hi,
>>
>> Does anyone know how to generate the following WAL records from psql?
>>
>> I'm now fixing pg_lesslog, which I reported a bug.   Now code is
>> almost okay and I'd like to add test tools to show pg_lesslog can
>> handle all the WAL record correctly.
>>
>> XLOG_MULTIXACT_ZERO_OFF_PAGE
>> XLOG_MULTIXACT_ZERO_MEM_PAGE
>> XLOG_MULTIXACT_CREATE_ID
>
> For these, you need to do SELECT FOR SHARE of a tuple in more than one
> concurrent session.
>
> I think HEAP_LOCK is SELECT FOR UPDATE.
>
>
>> XLOG_BTREE_DELETE_PAGE
>> XLOG_BTREE_DELETE_PAGE_META
>> XLOG_BTREE_DELETE_PAGE_HALF
>
> These are caused during vacuum of a btree.  Create an index, populate
> it, then remove all items.  Then vacuum twice.
>
> --
> Alvaro Herrera                                http://www.CommandPrompt.com/
> PostgreSQL Replication, Consulting, Custom Development, 24x7 support
>

On Wed, 2010-04-14 at 11:13 +0900, Koichi Suzuki wrote:

> Thank you for a great advice. I successfully generated all the WAL
> records listed below. By deleteing and VACUUMing a table with
> btree_gist based index, I was successful to generate all the GIST WAL
> records.

It would be a very useful test case to publish. There are already a
number of standby related tests, this would be very useful for general
case testing, not just for pg_lesslog.

--
Simon Riggs www.2ndQuadrant.com

Thanks for encouraging comment. I'm still struggling to generate
remaing WAL records.
----------
Koichi Suzuki

2010/4/14 Simon Riggs <simon(at)2ndquadrant(dot)com>:
> On Wed, 2010-04-14 at 11:13 +0900, Koichi Suzuki wrote:
>
>> Thank you for a great advice.   I successfully generated all the WAL
>> records listed below.   By deleteing and VACUUMing a table with
>> btree_gist based index, I was successful to generate all the GIST WAL
>> records.
>
> It would be a very useful test case to publish. There are already a
> number of standby related tests, this would be very useful for general
> case testing, not just for pg_lesslog.
>
> --
>  Simon Riggs           www.2ndQuadrant.com
>
>

Koichi Suzuki <koichi(dot)szk(at)gmail(dot)com> wrote:
> 2010/4/14 Simon Riggs <simon(at)2ndquadrant(dot)com>:

>> It would be a very useful test case to publish.

> I'm still struggling to generate remaing WAL records.

Sure, but when you've got it all, please share. I'd like to see us
have a much larger set of tests than the "make check" regression
tests which would get run less frequently but test a lot more. This
sounds like a good one to include.

-Kevin

Kevin Grittner wrote:
> Koichi Suzuki <koichi(dot)szk(at)gmail(dot)com> wrote:
> > 2010/4/14 Simon Riggs <simon(at)2ndquadrant(dot)com>:
>
> >> It would be a very useful test case to publish.
>
> > I'm still struggling to generate remaing WAL records.
>
> Sure, but when you've got it all, please share. I'd like to see us
> have a much larger set of tests than the "make check" regression
> tests which would get run less frequently but test a lot more. This
> sounds like a good one to include.

Agreed. This test belongs in our CVS tree so we can maintain it as we
add new WAL types.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

Now I've tested almost all WAL record, including CLOG_TRUNCATE and all
the GIST and GIN-related WALs.

I'm still struggling to find how to have XLOG_HEAP_INIT_PAGE and
XLOG_BTREE_INSERT_META. I'm trying to find how to have these WAL
records but it's a great help if anyone suggests me how.

Thank you very much in advance;
----------
Koichi Suzuki

2010/4/15 Bruce Momjian <bruce(at)momjian(dot)us>:
> Kevin Grittner wrote:
>> Koichi Suzuki <koichi(dot)szk(at)gmail(dot)com> wrote:
>> > 2010/4/14 Simon Riggs <simon(at)2ndquadrant(dot)com>:
>>
>> >> It would be a very useful test case to publish.
>>
>> > I'm still struggling to generate remaing WAL records.
>>
>> Sure, but when you've got it all, please share.  I'd like to see us
>> have a much larger set of tests than the "make check" regression
>> tests which would get run less frequently but test a lot more.  This
>> sounds like a good one to include.
>
> Agreed.  This test belongs in our CVS tree so we can maintain it as we
> add new WAL types.
>
> --
>  Bruce Momjian  <bruce(at)momjian(dot)us>        http://momjian.us
>  EnterpriseDB                             http://enterprisedb.com
>