pgbench filler columns

While looking at the compressibility of WAL files generated by pgbench,
which is close to 90%, I first thought its because of the "filler" column
in the accounts table. But a comment in pgbench.c says:

/*
* Note: TPC-B requires at least 100 bytes per row, and the "filler"
* fields in these table declarations were intended to comply with that.
* But because they default to NULLs, they don't actually take any
space.
* We could fix that by giving them non-null default values. However,
that
* would completely break comparability of pgbench results with prior
* versions. Since pgbench has never pretended to be fully TPC-B
* compliant anyway, we stick with the historical behavior.
*/

The comment about them being NULL and hence not taking up any space is
added by commit b7a67c2840f193f in response to this bug report:
http://www.postgresql.org/message-id/200710170810.l9H8A76I080568@wwwmaster.postgresql.org

But I find it otherwise. On my machine, accounts table can only fit 62
tuples in a page with default fillfactor. The following queries show that
filler column is NOT NULL, but set to empty string. I have tested on 8.2,
8.4 and master and they all show the same behavior. So I don't know if that
bug report itself was wrong or if I am reading the comment wrong.

postgres=# select count(*) from pgbench_accounts where filler IS NULL;
count
-------
0
(1 row)

postgres=# select count(*) from pgbench_accounts where filler = '';
count
--------
100000
(1 row)

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

On Thu, Sep 26, 2013 at 2:05 PM, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>wrote:

> While looking at the compressibility of WAL files generated by pgbench,
> which is close to 90%, I first thought its because of the "filler" column
> in the accounts table. But a comment in pgbench.c says:
>
> /*
> * Note: TPC-B requires at least 100 bytes per row, and the "filler"
> * fields in these table declarations were intended to comply with
> that.
> * But because they default to NULLs, they don't actually take any
> space.
> * We could fix that by giving them non-null default values. However,
> that
> * would completely break comparability of pgbench results with prior
> * versions. Since pgbench has never pretended to be fully TPC-B
> * compliant anyway, we stick with the historical behavior.
> */
>
> The comment about them being NULL and hence not taking up any space is
> added by commit b7a67c2840f193f in response to this bug report:
>
> http://www.postgresql.org/message-id/200710170810.l9H8A76I080568@wwwmaster.postgresql.org
>
>
On a more careful look, it seems the original bug report complained about
all tables except accounts. And all other tables indeed have "filler" as
NULL. But the way comment is written it seems as if it applies to all DDLs.
Should we just fix the comment and say its applicable for all tables except
accounts ?

Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

On Thu, Sep 26, 2013 at 03:23:30PM +0530, Pavan Deolasee wrote:
> On Thu, Sep 26, 2013 at 2:05 PM, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>wrote:
>
> > While looking at the compressibility of WAL files generated by pgbench,
> > which is close to 90%, I first thought its because of the "filler" column
> > in the accounts table. But a comment in pgbench.c says:
> >
> > /*
> > * Note: TPC-B requires at least 100 bytes per row, and the "filler"
> > * fields in these table declarations were intended to comply with
> > that.
> > * But because they default to NULLs, they don't actually take any
> > space.
> > * We could fix that by giving them non-null default values. However,
> > that
> > * would completely break comparability of pgbench results with prior
> > * versions. Since pgbench has never pretended to be fully TPC-B
> > * compliant anyway, we stick with the historical behavior.
> > */
> >
> > The comment about them being NULL and hence not taking up any space is
> > added by commit b7a67c2840f193f in response to this bug report:
> >
> > http://www.postgresql.org/message-id/200710170810.l9H8A76I080568@wwwmaster.postgresql.org
> >
> >
> On a more careful look, it seems the original bug report complained about
> all tables except accounts. And all other tables indeed have "filler" as
> NULL. But the way comment is written it seems as if it applies to all DDLs.

Agreed.

> Should we just fix the comment and say its applicable for all tables except
> accounts ?

Please do.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com

On Thu, Sep 26, 2013 at 7:20 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:

> On Thu, Sep 26, 2013 at 03:23:30PM +0530, Pavan Deolasee wrote:
>
> > Should we just fix the comment and say its applicable for all tables
> except
> > accounts ?
>
> Please do.
>
>
How about something like this ? Patch attached.

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

On Fri, Sep 27, 2013 at 4:03 PM, Pavan Deolasee
<pavan(dot)deolasee(at)gmail(dot)com> wrote:
> On Thu, Sep 26, 2013 at 7:20 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:
>>
>> On Thu, Sep 26, 2013 at 03:23:30PM +0530, Pavan Deolasee wrote:
>>
>> > Should we just fix the comment and say its applicable for all tables
>> > except
>> > accounts ?
>>
>> Please do.
>>
>
> How about something like this ? Patch attached.

Thanks! Committed.

Regards,

--
Fujii Masao