Re: pgstat documentation tables

Right now we have a single table on
http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS
that lists all our statistics views. It makes for difficult to parse
descriptions like "One row only, showing cluster-wide statistics from
the background writer: number of scheduled checkpoints, requested
checkpoints, buffers written by checkpoints and cleaning scans, and
the number of times the background writer stopped a cleaning scan
because it had written too many buffers. Also includes statistics
about the shared buffer pool, including buffers written by backends
(that is, not by the background writer), how many times those backends
had to execute their own fsync calls (normally the background writer
handles those even when the backend does its own write), total buffers
allocated, and time of last statistics reset."

I'd like to turn that into one table for each view, with two columns,
one being the name the other one being the description. That'll also
make it possible to expand on the descriptions without making it
completley unreadable, should we want to.

Comments?Objections?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 01/15/2012 03:51 AM, Magnus Hagander wrote:
> I'd like to turn that into one table for each view, with two columns,
> one being the name the other one being the description. That'll also
> make it possible to expand on the descriptions without making it
> completley unreadable, should we want to.

Scott Mead's pg_stat_activity refactoring patch already was forced
toward doing something similar for its documentation to be readable
(maybe exactly the same as you were thinking, can't tell because my
SGML-fu is weak) . I feel doing this for every view in there is
inevitable, just a matter of when to bite on the problem. This whole
section is barely usable in its current form. I strongly hope the
pg_stat_activity patch goes in, which will be close to a compelling
event to motivate fixing the rest of them in a similar way. I would
advocate leading with that one until it seems right, then adopting the
same style of reorg to the rest until they're all converted.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On Sun, Jan 15, 2012 at 10:35, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> On 01/15/2012 03:51 AM, Magnus Hagander wrote:
>>
>> I'd like to turn that into one table for each view, with two columns,
>> one being the name the other one being the description. That'll also
>> make it possible to expand on the descriptions without making it
>> completley unreadable, should we want to.
>
>
> Scott Mead's pg_stat_activity refactoring patch already was forced toward
> doing something similar for its documentation to be readable (maybe exactly
> the same as you were thinking, can't tell because my SGML-fu is weak) .  I
> feel doing this for every view in there is inevitable, just a matter of when
> to bite on the problem.  This whole section is barely usable in its current
> form.  I strongly hope the pg_stat_activity patch goes in, which will be
> close to a compelling event to motivate fixing the rest of them in a similar
> way.  I would advocate leading with that one until it seems right, then
> adopting the same style of reorg to the rest until they're all converted.

Yes, the pg_stat_activity patch will make it more important - but I
want to do it for *all* of them...

I'll be happy to do the mechanical work on turning them into table,
but yes, I was planning to look at the pg_stat_activity patch first
all along :)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> Right now we have a single table on
> http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS
> that lists all our statistics views ...
> I'd like to turn that into one table for each view,

Please follow the style already used for system catalogs; ie I think
there should be a summary table with one entry per view, and then a
separate description and table-of-columns for each view.

regards, tom lane

On 01/15/2012 12:20 PM, Tom Lane wrote:
> Please follow the style already used for system catalogs; ie I think
> there should be a summary table with one entry per view, and then a
> separate description and table-of-columns for each view.

Yes, that's a perfect precedent. I think the easiest path forward here
is to tweak the updated pg_stat_activity documentation, since that's
being refactoring first anyway. That can be reformatted until it looks
just like the system catalog documentation. And then once that's done,
the rest of them can be converted over to follow the same style. I'd be
willing to work on doing that in a way that improves what is documented,
too. The difficulty of working with the existing tables has been the
deterrent for improving that section to me.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

Bruce had a patch to turn SGML descriptions of system view into comments
via some Perl program or something. He posted it many moons ago and I
haven't seen an updated version. Bruce, do you have something to say on
this topic?

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On sön, 2012-01-15 at 12:20 -0500, Tom Lane wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > Right now we have a single table on
> > http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS
> > that lists all our statistics views ...
> > I'd like to turn that into one table for each view,
>
> Please follow the style already used for system catalogs; ie I think
> there should be a summary table with one entry per view, and then a
> separate description and table-of-columns for each view.

If the tables had proper table and column comments, you might even be
able to generate the SGML documentation automatically.

On Mon, Jan 16, 2012 at 19:41, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On sön, 2012-01-15 at 12:20 -0500, Tom Lane wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> > Right now we have a single table on
>> > http://www.postgresql.org/docs/devel/static/monitoring-stats.html#MONITORING-STATS-VIEWS
>> > that lists all our statistics views ...
>> > I'd like to turn that into one table for each view,
>>
>> Please follow the style already used for system catalogs; ie I think
>> there should be a summary table with one entry per view, and then a
>> separate description and table-of-columns for each view.
>
> If the tables had proper table and column comments, you might even be
> able to generate the SGML documentation automatically.

Well, I'd expect some of those columns to get (at least over time)
significantly more detailed information than they have now. Certainly
more than you'd put in comments in the catalogs. And having some sort
of combination there seems to overcomplicate things...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Jan 16, 2012 at 01:01:50PM -0300, Alvaro Herrera wrote:
>
>
> Bruce had a patch to turn SGML descriptions of system view into comments
> via some Perl program or something. He posted it many moons ago and I
> haven't seen an updated version. Bruce, do you have something to say on
> this topic?

Yes, I was waiting to get feedback on that but it seems the 9.2 release
slipped right by me on that. I guess I will try to pick it up for 9.3.

How do people feel about pulling text out of the SGML docs and loading
it into the database as table and column comments?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> How do people feel about pulling text out of the SGML docs and loading
> it into the database as table and column comments?

I'm not thrilled by that proposal. The length limits on comments are
very much shorter than what is sensible to use in catalogs.sgml (at
least if you want the comments to look passable in \d displays). And
I don't want people trying to serve two different use-cases when they
write those.

Perhaps it'd work to pull column comments from the C header files
--- any same-line comment in catalog/pg_*.h is probably short enough to
serve this purpose usefully.

regards, tom lane

On Feb 8, 2012 5:32 AM, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > How do people feel about pulling text out of the SGML docs and loading
> > it into the database as table and column comments?
>
> I'm not thrilled by that proposal. The length limits on comments are
> very much shorter than what is sensible to use in catalogs.sgml (at
> least if you want the comments to look passable in \d displays). And
> I don't want people trying to serve two different use-cases when they
> write those.

Yes, I'd rather see longer than shorter descriptions in the docs, and this
would turn that backwards...

>
> Perhaps it'd work to pull column comments from the C header files
> --- any same-line comment in catalog/pg_*.h is probably short enough to
> serve this purpose usefully.

That could work a lot better..

/Magnus

On Tue, Feb 07, 2012 at 11:29:12PM -0500, Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > How do people feel about pulling text out of the SGML docs and loading
> > it into the database as table and column comments?
>
> I'm not thrilled by that proposal. The length limits on comments are
> very much shorter than what is sensible to use in catalogs.sgml (at
> least if you want the comments to look passable in \d displays). And
> I don't want people trying to serve two different use-cases when they
> write those.

I was thinking of just grabbing the first X characters from the SGML
comments.

> Perhaps it'd work to pull column comments from the C header files
> --- any same-line comment in catalog/pg_*.h is probably short enough to
> serve this purpose usefully.

Yes, that would be simpler.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, Feb 8, 2012 at 17:13, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> On Tue, Feb 07, 2012 at 11:29:12PM -0500, Tom Lane wrote:
>> Bruce Momjian <bruce(at)momjian(dot)us> writes:
>> > How do people feel about pulling text out of the SGML docs and loading
>> > it into the database as table and column comments?
>>
>> I'm not thrilled by that proposal.  The length limits on comments are
>> very much shorter than what is sensible to use in catalogs.sgml (at
>> least if you want the comments to look passable in \d displays).  And
>> I don't want people trying to serve two different use-cases when they
>> write those.
>
> I was thinking of just grabbing the first X characters from the SGML
> comments.

So that it wouldn't necessarily even make for a complete sentence?
That seems like a really bad idea...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Jan 16, 2012 at 02:03, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> On 01/15/2012 12:20 PM, Tom Lane wrote:
>>
>> Please follow the style already used for system catalogs; ie I think
>> there should be a summary table with one entry per view, and then a
>> separate description and table-of-columns for each view.
>
>
> Yes, that's a perfect precedent.  I think the easiest path forward here is
> to tweak the updated pg_stat_activity documentation, since that's being
> refactoring first anyway.  That can be reformatted until it looks just like
> the system catalog documentation.  And then once that's done, the rest of
> them can be converted over to follow the same style.  I'd be willing to work
> on doing that in a way that improves what is documented, too.  The
> difficulty of working with the existing tables has been the deterrent for
> improving that section to me.

I've applied a patch that does this now. Hopefully, I didn't create
too many spelling errors or such :-)

I also applied a separate patch that folded the list of functions into
the list of views, since that's where they are called, as a way to
reduce duplicate documentation. I did it as a spearate patch to make
it easier to back out if people think that was a bad idea...

I didn't add any new documentation at this point - I wanted to get
this patch in before it starts conflicting with work others are
potentially doing. I also noticed there were some stats functions that
were undocumented - the view fields were listed, but not the functions
themselves. I haven't added those either...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Sat, Feb 25, 2012 at 9:33 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, Jan 16, 2012 at 02:03, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
>> On 01/15/2012 12:20 PM, Tom Lane wrote:
>>>
>>> Please follow the style already used for system catalogs; ie I think
>>> there should be a summary table with one entry per view, and then a
>>> separate description and table-of-columns for each view.
>>
>>
>> Yes, that's a perfect precedent.  I think the easiest path forward here is
>> to tweak the updated pg_stat_activity documentation, since that's being
>> refactoring first anyway.  That can be reformatted until it looks just like
>> the system catalog documentation.  And then once that's done, the rest of
>> them can be converted over to follow the same style.  I'd be willing to work
>> on doing that in a way that improves what is documented, too.  The
>> difficulty of working with the existing tables has been the deterrent for
>> improving that section to me.
>
> I've applied a patch that does this now. Hopefully, I didn't create
> too many spelling errors or such :-)
>
> I also applied a separate patch that folded the list of functions into
> the list of views, since that's where they are called, as a way to
> reduce duplicate documentation. I did it as a spearate patch to make
> it easier to back out if people think that was a bad idea...

I think it's a little awkward this way; maybe it would be better as a
separate table column. Or maybe it was better the way it was; I'm not
sure. Or maybe we could have a separate table that just gives the
equivalences between stats table-column pairs and functions. Of those
ideas, I think I like "separate table in the same column" the best.

Also, I wonder if we should promote section 27.2.2.1. Other Statistics
Functions to 27.2.3.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Feb 27, 2012 at 04:44, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sat, Feb 25, 2012 at 9:33 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> On Mon, Jan 16, 2012 at 02:03, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
>>> On 01/15/2012 12:20 PM, Tom Lane wrote:
>>>>
>>>> Please follow the style already used for system catalogs; ie I think
>>>> there should be a summary table with one entry per view, and then a
>>>> separate description and table-of-columns for each view.
>>>
>>>
>>> Yes, that's a perfect precedent.  I think the easiest path forward here is
>>> to tweak the updated pg_stat_activity documentation, since that's being
>>> refactoring first anyway.  That can be reformatted until it looks just like
>>> the system catalog documentation.  And then once that's done, the rest of
>>> them can be converted over to follow the same style.  I'd be willing to work
>>> on doing that in a way that improves what is documented, too.  The
>>> difficulty of working with the existing tables has been the deterrent for
>>> improving that section to me.
>>
>> I've applied a patch that does this now. Hopefully, I didn't create
>> too many spelling errors or such :-)
>>
>> I also applied a separate patch that folded the list of functions into
>> the list of views, since that's where they are called, as a way to
>> reduce duplicate documentation. I did it as a spearate patch to make
>> it easier to back out if people think that was a bad idea...
>
> I think it's a little awkward this way; maybe it would be better as a
> separate table column.  Or maybe it was better the way it was; I'm not

The problem with a separate column is that it makes the table very
wide (some of those functions have very long name). And it won't be
possible to get an auto-linebreak in the function names, because
they're too long.

> sure.  Or maybe we could have a separate table that just gives the
> equivalences between stats table-column pairs and functions.  Of those
> ideas, I think I like "separate table in the same column" the best.

That one would at least work. You mean basically:

|pg_stat_database.xact_commit|pg_stat_get_db_xact_commit|
|pg_stat_database.xact_rollback|pg_stat_get_db_xact_rollback|

etc etc for each column/function, right?

> Also, I wonder if we should promote section 27.2.2.1. Other Statistics
> Functions to 27.2.3.

I was considering that, but given that 27.2.2 is "viewing statistics",
it does seem like a sub-section to that.. Though maybe if we make the
lis tof views into their own section *as well*, so we have one sectoin
for "how to view it", one for "views" and one for "other functions" it
would make more sense.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Feb 27, 2012 at 5:22 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> The problem with a separate column is that it makes the table very
> wide (some of those functions have very long name).

Yeah, that's one thing I don't like about what you actually did,
either - it made some of the tables much wider.

>> sure.  Or maybe we could have a separate table that just gives the
>> equivalences between stats table-column pairs and functions.  Of those
>> ideas, I think I like "separate table in the same column" the best.
>
> That one would at least work. You mean basically:
>
> |pg_stat_database.xact_commit|pg_stat_get_db_xact_commit|
> |pg_stat_database.xact_rollback|pg_stat_get_db_xact_rollback|
>
> etc etc for each column/function, right?

Yeah.

>> Also, I wonder if we should promote section 27.2.2.1. Other Statistics
>> Functions to 27.2.3.
>
> I was considering that, but given that 27.2.2 is "viewing statistics",
> it does seem like a sub-section to that.. Though maybe if we make the
> lis tof views into their own section *as well*, so we have one sectoin
> for "how to view it", one for "views" and one for "other functions" it
> would make more sense.

My thought was that a good half of those "other statistics functions"
don't have all that much to do with viewing anything, so the logical
argument for that section to need to be under "viewing statistics"
seems a bit weak.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Feb 27, 2012 at 14:36, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Mon, Feb 27, 2012 at 5:22 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> The problem with a separate column is that it makes the table very
>> wide (some of those functions have very long name).
>
> Yeah, that's one thing I don't like about what you actually did,
> either - it made some of the tables much wider.

Uh, can you give me an example of one? While they will all become
wider if you actually need to, I don't see any of those tables as wide
enough to cause any trouble that automatic linebreaks don't fix?

>>> sure.  Or maybe we could have a separate table that just gives the
>>> equivalences between stats table-column pairs and functions.  Of those
>>> ideas, I think I like "separate table in the same column" the best.
>>
>> That one would at least work. You mean basically:
>>
>> |pg_stat_database.xact_commit|pg_stat_get_db_xact_commit|
>> |pg_stat_database.xact_rollback|pg_stat_get_db_xact_rollback|
>>
>> etc etc for each column/function, right?
>
> Yeah.

Ok, I'll see if I can put something like that together and see what it
looks like. Unless somebody else objects to that way?

>>> Also, I wonder if we should promote section 27.2.2.1. Other Statistics
>>> Functions to 27.2.3.
>>
>> I was considering that, but given that 27.2.2 is "viewing statistics",
>> it does seem like a sub-section to that.. Though maybe if we make the
>> lis tof views into their own section *as well*, so we have one sectoin
>> for "how to view it", one for "views" and one for "other functions" it
>> would make more sense.
>
> My thought was that a good half of those "other statistics functions"
> don't have all that much to do with viewing anything, so the logical
> argument for that section to need to be under "viewing statistics"
> seems a bit weak.

True.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Feb 27, 2012 at 11:32 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, Feb 27, 2012 at 14:36, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Mon, Feb 27, 2012 at 5:22 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> The problem with a separate column is that it makes the table very
>>> wide (some of those functions have very long name).
>>
>> Yeah, that's one thing I don't like about what you actually did,
>> either - it made some of the tables much wider.
>
> Uh, can you give me an example of one? While they will all become
> wider if you actually need to, I don't see any of those tables as wide
> enough to cause any trouble that automatic linebreaks don't fix?

It's not so much that they become wide enough to line-wrap; it's just
that it's harder to read. For example, table 27-5,
pg_stat_database_conflicts view, is now about twice the width that it
was before, and the additional sentence is mostly junk, because 95% of
people reading this won't care a whit. Plus, for those who do care,
the phrasing of the sentence (this value can also be returned
directly...) isn't really altogether clear, at least IMHO.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Mon, Feb 27, 2012 at 11:32 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> On Mon, Feb 27, 2012 at 14:36, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> Yeah, that's one thing I don't like about what you actually did,
>>> either - it made some of the tables much wider.

>> Uh, can you give me an example of one? While they will all become
>> wider if you actually need to, I don't see any of those tables as wide
>> enough to cause any trouble that automatic linebreaks don't fix?

> It's not so much that they become wide enough to line-wrap; it's just
> that it's harder to read.

Something else to keep in mind is that PDF-format docs are not terribly
forgiving of wide tables --- have you looked at what these look like in
PDF?

regards, tom lane

On Mon, Feb 27, 2012 at 18:00, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Mon, Feb 27, 2012 at 11:32 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> On Mon, Feb 27, 2012 at 14:36, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>> Yeah, that's one thing I don't like about what you actually did,
>>>> either - it made some of the tables much wider.
>
>>> Uh, can you give me an example of one? While they will all become
>>> wider if you actually need to, I don't see any of those tables as wide
>>> enough to cause any trouble that automatic linebreaks don't fix?
>
>> It's not so much that they become wide enough to line-wrap; it's just
>> that it's harder to read.
>
> Something else to keep in mind is that PDF-format docs are not terribly
> forgiving of wide tables --- have you looked at what these look like in
> PDF?

I have not, and I can't seem to build them:
openjade -D . -D . -c
/usr/share/sgml/docbook/stylesheet/dsssl/modular/catalog -d
./stylesheet.dsl -t tex -V tex-backend -i output-print -i
include-index -V texpdf-output -V '%paper-type%'=A4 -o
postgres-A4.tex-pdf postgres.sgml
openjade:./stylesheet.dsl:678:2:E: flow object not accepted by port;
only display flow objects accepted
make: *** [postgres-A4.tex-pdf] Segmentation fault
make: *** Deleting file `postgres-A4.tex-pdf'

does that work for others, or did we break something globally in it?

I'm not sure I've ever tried to built it on Ubuntu 12.10 before, so it
could be that..

(and same error on -US)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Feb 27, 2012 at 18:00, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Something else to keep in mind is that PDF-format docs are not terribly
>> forgiving of wide tables --- have you looked at what these look like in
>> PDF?

> I have not, and I can't seem to build them:
> openjade -D . -D . -c
> /usr/share/sgml/docbook/stylesheet/dsssl/modular/catalog -d
> ./stylesheet.dsl -t tex -V tex-backend -i output-print -i
> include-index -V texpdf-output -V '%paper-type%'=A4 -o
> postgres-A4.tex-pdf postgres.sgml
> openjade:./stylesheet.dsl:678:2:E: flow object not accepted by port;
> only display flow objects accepted

Oh, not that again :-(. We've never figured out exactly what triggers
that AFAIR.

> does that work for others, or did we break something globally in it?

FWIW, I built all the back-branch versions in both US and A4 formats
last week on Fedora 16. Don't recall trying HEAD though.

> I'm not sure I've ever tried to built it on Ubuntu 12.10 before, so it
> could be that..

FWIW, all the recent reports of this seem to be on Ubuntu or Debian.
Don't know if that means few of us use Fedora or if it means Fedora has
fixed this. I wonder a bit about this patch Fedora is carrying:
http://pkgs.fedoraproject.org/gitweb/?p=openjade.git;a=blob;f=openjade-1.3.2-gcc46.patch;h=962ac19ffea6bb6434ebb1a482d5604088192c7a;hb=fce180174c911b5982a5b1a45e09912c910e9bbf
which is stated to be for a recent gcc change, but it does seem to
have to do with creation of flow objects ...

regards, tom lane

On Mon, Feb 27, 2012 at 18:22, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Mon, Feb 27, 2012 at 18:00, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Something else to keep in mind is that PDF-format docs are not terribly
>>> forgiving of wide tables --- have you looked at what these look like in
>>> PDF?
>
>> I have not, and I can't seem to build them:
>> openjade  -D . -D . -c
>> /usr/share/sgml/docbook/stylesheet/dsssl/modular/catalog -d
>> ./stylesheet.dsl -t tex -V tex-backend -i output-print -i
>> include-index -V texpdf-output -V '%paper-type%'=A4 -o
>> postgres-A4.tex-pdf postgres.sgml
>> openjade:./stylesheet.dsl:678:2:E: flow object not accepted by port;
>> only display flow objects accepted
>
> Oh, not that again :-(.  We've never figured out exactly what triggers
> that AFAIR.

:-O

>> does that work for others, or did we break something globally in it?
>
> FWIW, I built all the back-branch versions in both US and A4 formats
> last week on Fedora 16.  Don't recall trying HEAD though.

I only tried HEAD. Trying 9.1 now, and I get the same crash there. Can
you (or someone) try it on Fedora with the latest tip of 9.1,
including the release notes, if those broke it?

>> I'm not sure I've ever tried to built it on Ubuntu 12.10 before, so it
>> could be that..
>
> FWIW, all the recent reports of this seem to be on Ubuntu or Debian.
> Don't know if that means few of us use Fedora or if it means Fedora has
> fixed this.  I wonder a bit about this patch Fedora is carrying:
> http://pkgs.fedoraproject.org/gitweb/?p=openjade.git;a=blob;f=openjade-1.3.2-gcc46.patch;h=962ac19ffea6bb6434ebb1a482d5604088192c7a;hb=fce180174c911b5982a5b1a45e09912c910e9bbf
> which is stated to be for a recent gcc change, but it does seem to
> have to do with creation of flow objects ...

My ubuntu has gcc 4.6.1, so AFAICT it would be affected.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Feb 27, 2012 at 18:28, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, Feb 27, 2012 at 18:22, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> On Mon, Feb 27, 2012 at 18:00, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>> Something else to keep in mind is that PDF-format docs are not terribly
>>>> forgiving of wide tables --- have you looked at what these look like in
>>>> PDF?
>>
>>> I have not, and I can't seem to build them:
>>> openjade  -D . -D . -c
>>> /usr/share/sgml/docbook/stylesheet/dsssl/modular/catalog -d
>>> ./stylesheet.dsl -t tex -V tex-backend -i output-print -i
>>> include-index -V texpdf-output -V '%paper-type%'=A4 -o
>>> postgres-A4.tex-pdf postgres.sgml
>>> openjade:./stylesheet.dsl:678:2:E: flow object not accepted by port;
>>> only display flow objects accepted
>>
>> Oh, not that again :-(.  We've never figured out exactly what triggers
>> that AFAIR.
>
> :-O
>
>>> does that work for others, or did we break something globally in it?
>>
>> FWIW, I built all the back-branch versions in both US and A4 formats
>> last week on Fedora 16.  Don't recall trying HEAD though.
>
> I only tried HEAD. Trying 9.1 now, and I get the same crash there. Can
> you (or someone) try it on Fedora with the latest tip of 9.1,
> including the release notes, if those broke it?

actually, that can't be it - Devrim built the 9.1 PDFs fine on a
Fedora box, and there hasn't been any activity on 9.1 stable since.
Must be the Ubuntu-is-broken thing then :(

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Feb 27, 2012 at 18:22, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> FWIW, I built all the back-branch versions in both US and A4 formats
>> last week on Fedora 16. Don't recall trying HEAD though.

> I only tried HEAD. Trying 9.1 now, and I get the same crash there. Can
> you (or someone) try it on Fedora with the latest tip of 9.1,
> including the release notes, if those broke it?

What I was building *was* the release notes, specifically to see if it
would work or not, because we've had problems before with the PDF builds
failing unexpectedly. Should be exactly the same docs as 9.1 branch
tip. (Rechecks...) In fact, I later rebuilt postgres-US.pdf from the
9.1.3 tarball, so that definitely was the released bits.

This doesn't really prove that Fedora has a fix for the bug, of course.
We have noted before that Fedora's style sheets sometimes result in page
breaks at different places from other distros, and it would not be
terribly surprising if this bug were page-break-sensitive.

regards, tom lane

On 02/27/2012 12:22 PM, Tom Lane wrote:
> FWIW, all the recent reports of this seem to be on Ubuntu or Debian.
> Don't know if that means few of us use Fedora or if it means Fedora has
> fixed this.

I just tried building postgres-US.pdf on a RHEL-derived system,
Scientific Linux 6 using openjade-1.3.2-36.el6. That gave me lots of
"Overfull hbox" errors, then died like this:

! pdfTeX error (ext4): \pdfendlink ended up in different nesting level
than \pd
fstartlink.
<to be read again>
\endgroup \set(at)typeset@protect
l.895119 ...rticular type modifier value.\endPar{}
\endNode{}\Node%
! ==> Fatal error occurred, no output PDF file produced!

Here are some Debian/Ubuntu platforms that all run into the other problem:

Ubuntu 9.04, openjade 1.4devel1-19: flow error
Debian Squeeze, openjade 1.4devel1-19 : flow error

I always assumed that the reason this didn't work, but the Fedoras did,
was because of a difference between 1.3 and 1.4. Maybe a forward port
of that 1.3 patch would help. I'm not sure what state RHEL6 is really
in though, I don't fully trust my SL6 system yet for this task.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On Mon, February 27, 2012 18:30, Magnus Hagander wrote:
>>
>>>> does that work for others, or did we break something globally in it?
>>>

FWIW: I build A4 pdf's for HEAD often (say, weekly), on centos 5; it has always worked this last
year or so (I did tweak tex (?) parameters, long ago).

I built the A4 pdf just now again without problem.

Centos 5.7, openjade 1.3.2-27.

Here are some observations on these pg-stat tables:

Table 27-1. Standard Statistics views: looks OK in the pdf (HEAD).

Table 27-2 has some of the longer GUC/function names sticking out of the table frame. not pretty
but it still works, they are still readable as a whole.

Table 27-3 (pg_stat_bgwriter_view) it does not look good: some GUC/function names not only stick
outsize the table frame but are truncated (and thus become unreadable).

I think it would be better if somehow the widths of the 3 table-columns ('Column', 'Type',
'Description') would not be 3 columns of the same width, like they seem to be now: the 'Type'
column take fully one third of page-width, for short words like 'text' and 'bigint'. The
'Column' values too are almost always more narrow than 1/3 of the page.

Erik Rijkers

Greg Smith <greg(at)2ndQuadrant(dot)com> writes:
> I just tried building postgres-US.pdf on a RHEL-derived system,
> Scientific Linux 6 using openjade-1.3.2-36.el6. That gave me lots of
> "Overfull hbox" errors, then died like this:

> ! pdfTeX error (ext4): \pdfendlink ended up in different nesting level
> than \pdfstartlink.

This one is a known issue having to do with <link> text that crosses a
page boundary. It's fairly annoying but we know how to work around it:
tweak the text to avoid having a <link> fall right there, or better
shorten the <link> text enough so it doesn't cross a line end.

If you got as far as the TeX run, though, you're not seeing the openjade
crash. I rather suspect that one is at bottom a similar sort of issue,
but since it provides no feedback at all about where in the document it
is, it's hard to do much with it.

regards, tom lane

Greg Smith <greg(at)2ndQuadrant(dot)com> writes:
> Here are some Debian/Ubuntu platforms that all run into the other problem:
> Ubuntu 9.04, openjade 1.4devel1-19: flow error
> Debian Squeeze, openjade 1.4devel1-19 : flow error

> I always assumed that the reason this didn't work, but the Fedoras did,
> was because of a difference between 1.3 and 1.4.

Nah, I don't think so. The oldest reports of this in our archives are
from 7.4 era:
http://archives.postgresql.org/pgsql-docs/2003-12/msg00024.php
which could not have been openjade 1.4.

regards, tom lane

On 02/27/2012 03:39 PM, Tom Lane wrote:
> Greg Smith<greg(at)2ndQuadrant(dot)com> writes:
>> Here are some Debian/Ubuntu platforms that all run into the other problem:
>> Ubuntu 9.04, openjade 1.4devel1-19: flow error
>> Debian Squeeze, openjade 1.4devel1-19 : flow error
>
>> I always assumed that the reason this didn't work, but the Fedoras did,
>> was because of a difference between 1.3 and 1.4.
>
> Nah, I don't think so. The oldest reports of this in our archives are
> from 7.4 era:
> http://archives.postgresql.org/pgsql-docs/2003-12/msg00024.php
> which could not have been openjade 1.4.

I'm not so sure about that...that's 2003, and I've found RPM packages
including 1.4-devel going back to what looks like late 2001:
http://rpm.pbone.net/index.php3/stat/4/idpl/3369792/dir/turbolinux/com/OpenJade-1.4devel-2.i586.rpm.html

The Postgres documentation documentation says "Downgrading to openjade
1.3 should get rid of this error." right now; at least I had a good
basis for the assumption I was making. Other ideas suggesting that's
the case include the discussion you were involved in at
http://sourceforge.net/mailarchive/message.php?msg_id=26959520 as well
as the long open bug at
https://bugs.launchpad.net/ubuntu/+source/openjade/+bug/12431 (which has
a small test case showing the same error).

Actually, check this out. I've attached the test.docbook from that
Ubuntu bug report. If I just run docbook2pdf on my Debian system, it
works fine:

$ docbook2pdf test.docbook
Using catalogs: /etc/sgml/catalog
Using stylesheet: /usr/share/docbook-utils/docbook-utils.dsl#print
Working on: /home/gsmith/test.docbook
jade:/usr/share/sgml/declaration/xml.dcl:31:27:W: characters in the
document character set with numbers exceeding 65535 not supported
Done.

And I can view the resulting PDF. But the test case in the bug report
says it's reproducible using "-p" to specify an alternate parser. If I
do that, sure enough the bug pops up:

$ docbook2pdf -p /usr/bin/openjade-1.4devel test.docbook
Using catalogs: /etc/sgml/catalog
Using stylesheet: /usr/share/docbook-utils/docbook-utils.dsl#print
Working on: /home/gsmith/test.docbook
/usr/bin/openjade-1.4devel:/usr/share/sgml/docbook/stylesheet/dsssl/modular/print/dbttlpg.dsl:2722:6:E:
flow object not accepted by port; only display flow objects accepted
Done.

That's despite the fact that my installed openjade is the same one I'm
referring to with the alternate parser:

$ which openjade
/usr/bin/openjade
$ ls -l /usr/bin/openjade
lrwxrwxrwx 1 root root 26 Jan 10 04:57 /usr/bin/openjade ->
/etc/alternatives/openjade
$ ls -l /etc/alternatives/openjade
lrwxrwxrwx 1 root root 26 Jan 10 04:57 /etc/alternatives/openjade ->
/usr/bin/openjade-1.4devel

And the bug is still there without the symlink coming into place:

$ docbook2pdf -p /usr/bin/openjade test.docbook
Using catalogs: /etc/sgml/catalog
Using stylesheet: /usr/share/docbook-utils/docbook-utils.dsl#print
Working on: /home/gsmith/test.docbook
/usr/bin/openjade:/usr/share/sgml/docbook/stylesheet/dsssl/modular/print/dbttlpg.dsl:2722:6:E:
flow object not accepted by port; only display flow objects accepted
Done.

Now that's just bizarre--that the problem comes and goes based on
whether you specific the parser by name. The same test works fine in
all incarnations on my SL6 system, so this simple case seems to follow
the systems where the problem is and isn't in my testing so far. Makes
me wonder if this is more of a memory stomping type of bug rather than a
functional one. If this simple test case holds up as representative of
the PostgreSQL problem, that would seem an easier thing to get fixed
upstream.

There is an open bug in this area for Debian too:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=521148

That gets reproduced like this (file test2.docbook also attached):

$ openjade -t tex -d /usr/share/docbook-utils/docbook-utils.dsl#print
/usr/share/sgml/declaration/xml.dcl test2.docbook
openjade:test2.docbook:6:12:E: document type does not allow element
"glossary" here
openjade:test2.docbook:6:12:E: end tag for "glossary" which is not finished
openjade:test2.docbook:6:12:E: end tag for "glossary" which is not finished
openjade:test2.docbook:7:10:E: end tag for "chapter" which is not finished
openjade:/usr/share/sgml/docbook/stylesheet/dsssl/modular/print/dbcompon.dsl:205:3:E:
flow object not accepted by port; only display flow objects accepted

This suggests whatever is going wrong here doesn't just impact PDF
creation. And this one also works fine on the openjade 1.3 based SL6
server here, just need a different path for the library files:

$ openjade -t tex -d
/usr/share/sgml/docbook/utils-0.6.14/docbook-utils.dsl#print
/usr/share/sgml/xml.dcl test2.docbook

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com