Switching to XML

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi All:

I read a thread from July explaining the current status of moving to XML
[1].

Peter, you mention that if someone has a clear need you would be open to
switching. Let me explain the problems I've encountered with SGML.

A few months ago, I split up the 8.1 pdf into 2 volumes so that I could
get a hardbound copy from Lulu. You can get those here [2-3] (I'm not
making any money, it's the pure cost of production). For 8.2, I wanted
to go deeper and be able to distribute the final version to bookstores.
In order to do that, each volume needs to be under 700 pages.

I started work on modifying the SGML to use a set of 3 volumes, split at
roughly 500 page intervals. I wanted to generate individual ToC's and
indexes for each volume. I started to modify the SGML to include a
"role" attribute for each indexterm, to tell what volume it was part of.
This was done with a simple sed script. When I went to generate the
actual indexes, I hit a brick wall. Apparently, the SGML toolchain
cannot handle typed indexes as described here [4]. Only the XML
toolchain currently handles them.

I am working with Joshua Drake on creating a Lulu account for the
fundraising group. We would then move the volumes I did for 8.1 to their
account, and raise the price so that any profit went to them. The
ability to order the 8.2 manual from bookstores could increase the
visibility of the project as a whole.

David Blewett

1.
http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7
2. http://www.lulu.com/content/455020
3. http://www.lulu.com/content/464949
4. http://www.xml.com/pub/a/2004/07/14/dbndx.html?page=3
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFeXIOZmlc6wNjtLYRAo2yAKCZT1NbsklCd8djADdv48MuLELG7wCfYYG3
ypUw8LU3g++GaY6Dz2vUi+I=
=lC+c
-----END PGP SIGNATURE-----

On Fri, 2006-12-08 at 09:09 -0500, David Blewett wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi All:
>
> I read a thread from July explaining the current status of moving to XML
> [1].

To add to this, it would take minutes to transform XML versus the days
it takes the SGML.

Joshua D. Drake

>
> Peter, you mention that if someone has a clear need you would be open to
> switching. Let me explain the problems I've encountered with SGML.
>
> A few months ago, I split up the 8.1 pdf into 2 volumes so that I could
> get a hardbound copy from Lulu. You can get those here [2-3] (I'm not
> making any money, it's the pure cost of production). For 8.2, I wanted
> to go deeper and be able to distribute the final version to bookstores.
> In order to do that, each volume needs to be under 700 pages.
>
> I started work on modifying the SGML to use a set of 3 volumes, split at
> roughly 500 page intervals. I wanted to generate individual ToC's and
> indexes for each volume. I started to modify the SGML to include a
> "role" attribute for each indexterm, to tell what volume it was part of.
> This was done with a simple sed script. When I went to generate the
> actual indexes, I hit a brick wall. Apparently, the SGML toolchain
> cannot handle typed indexes as described here [4]. Only the XML
> toolchain currently handles them.
>
> I am working with Joshua Drake on creating a Lulu account for the
> fundraising group. We would then move the volumes I did for 8.1 to their
> account, and raise the price so that any profit went to them. The
> ability to order the 8.2 manual from bookstores could increase the
> visibility of the project as a whole.
>
> David Blewett
>
> 1.
> http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7
> 2. http://www.lulu.com/content/455020
> 3. http://www.lulu.com/content/464949
> 4. http://www.xml.com/pub/a/2004/07/14/dbndx.html?page=3
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFFeXIOZmlc6wNjtLYRAo2yAKCZT1NbsklCd8djADdv48MuLELG7wCfYYG3
> ypUw8LU3g++GaY6Dz2vUi+I=
> =lC+c
> -----END PGP SIGNATURE-----
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

David Blewett wrote:
> I started work on modifying the SGML to use a set of 3 volumes, split
> at roughly 500 page intervals. I wanted to generate individual ToC's
> and indexes for each volume. I started to modify the SGML to include
> a "role" attribute for each indexterm, to tell what volume it was
> part of. This was done with a simple sed script. When I went to
> generate the actual indexes, I hit a brick wall. Apparently, the SGML
> toolchain cannot handle typed indexes as described here [4]. Only the
> XML toolchain currently handles them.

But no one is forcing you to use "the SGML toolchain". I requote the
message you cited:

First of all, moving to DocBook XML will not do anything in the way of
improving our output processing abilities. Any tool that you can use
on DocBook SGML can also be used on DocBook XML and vice versa.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter,

> First of all, moving to DocBook XML will not do anything in the way of
> improving our output processing abilities. Any tool that you can use
> on DocBook SGML can also be used on DocBook XML and vice versa.
>

As I said then, this is absolutely untrue. OpenOffice.org, for example,
works with DocBook XML but not SGML. There are also a plethora of XML
editing and publishing tools which can been used for Docbook XML which
are not available for SGML. A simple look at this page:
http://wiki.docbook.org/topic/DocBookAuthoringTools
.... shows that there are more than twice as many authoring tools which
support only XML as support SGML -- and that most of the tools which
support SGML are out-of-maintenance.

--Josh Berkus

On Fri, 2006-12-08 at 13:36 -0500, Josh Berkus wrote:
> Peter,
>
> > First of all, moving to DocBook XML will not do anything in the way of
> > improving our output processing abilities. Any tool that you can use
> > on DocBook SGML can also be used on DocBook XML and vice versa.
> >
>
> As I said then, this is absolutely untrue.

You are correct Josh. I am not sure why Peter ignores this simple fact.

> OpenOffice.org, for example,
> works with DocBook XML but not SGML. There are also a plethora of XML
> editing and publishing tools which can been used for Docbook XML which
> are not available for SGML. A simple look at this page:
> http://wiki.docbook.org/topic/DocBookAuthoringTools
> .... shows that there are more than twice as many authoring tools which
> support only XML as support SGML -- and that most of the tools which
> support SGML are out-of-maintenance.

Yep.

Sincerely,

Joshua D. Drake

>
> --Josh Berkus
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Josh Berkus wrote:
> As I said then, this is absolutely untrue. OpenOffice.org, for
> example, works with DocBook XML but not SGML.

What does "works" mean?

> There are also a
> plethora of XML editing and publishing tools which can been used for
> Docbook XML which are not available for SGML.

That has nothing to do with processing.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Fri, 2006-12-08 at 21:15 +0100, Peter Eisentraut wrote:
> Josh Berkus wrote:
> > As I said then, this is absolutely untrue. OpenOffice.org, for
> > example, works with DocBook XML but not SGML.
>
> What does "works" mean?

You can create, edit, convert, save, and open docbook xml in
OpenOfice.org.

>
> > There are also a
> > plethora of XML editing and publishing tools which can been used for
> > Docbook XML which are not available for SGML.
>
> That has nothing to do with processing.

You are correct but XML does give us more modern and efficient toolsets
for processing. XSLT, FOP + Xerces for example.

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> You can create, edit, convert, save, and open docbook xml in
> OpenOfice.org.

Sure, there are more editing options with DocBook XML. No one disputes
that. But the question at hand was about processing the DocBook.

> You are correct but XML does give us more modern and efficient
> toolsets for processing. XSLT, FOP + Xerces for example.

You can do that right now.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > You can create, edit, convert, save, and open docbook xml in
> > OpenOfice.org.
>
> Sure, there are more editing options with DocBook XML. No one disputes
> that. But the question at hand was about processing the DocBook.

Yes which is generated from our use of SGML which is the core of this
problem and the core of the question as a whole.

SGML is making working with the documentation *harder*.

We have people that *DO NOT* contribute because of this SGML
requirement. They have what I consider extremely valid reasons, namely
it is dumb to require a writer to use emacs or write tags explictly.

Hell, the only reason I have even bothered to contribute what little I
have to the docs is because I wrote a book in SGML, thus it is a no
brainer to me. Others aren't so tortured as to have done the same.

There is a long standing support within the community to move to XML
including:

Josh Berkus
Josh Drake
Robert Treat
Andrew Dunslane
David Blewett
David Fetter
Devrim Gunduz
Darcy Buskermolen

And that is just from #postgresql

The french team also uses Docbook XML and they can generate a PDF in 30
minutes... it takes us DAYS because of the SGML.

How about we stopping using assembling and move to C shall we?

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Fri, 2006-12-08 at 15:26, Joshua D. Drake wrote:
> On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> > Joshua D. Drake wrote:
> > > You can create, edit, convert, save, and open docbook xml in
> > > OpenOfice.org.
> >
> > Sure, there are more editing options with DocBook XML. No one disputes
> > that. But the question at hand was about processing the DocBook.
>
> Yes which is generated from our use of SGML which is the core of this
> problem and the core of the question as a whole.
>
> SGML is making working with the documentation *harder*.
>
> We have people that *DO NOT* contribute because of this SGML
> requirement. They have what I consider extremely valid reasons, namely
> it is dumb to require a writer to use emacs or write tags explictly.

I'm one. I had the vacuuming stuff about written, but couldn't figure
out the tool set this summer, and didn't have the time. Does OO make it
pretty much painless to work with? I couldn't find any tools that I
felt comfortable using for SGML / Docbooks.

On Fri, 2006-12-08 at 15:45 -0600, Scott Marlowe wrote:
> On Fri, 2006-12-08 at 15:26, Joshua D. Drake wrote:
> > On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> > > Joshua D. Drake wrote:
> > > > You can create, edit, convert, save, and open docbook xml in
> > > > OpenOfice.org.
> > >
> > > Sure, there are more editing options with DocBook XML. No one disputes
> > > that. But the question at hand was about processing the DocBook.
> >
> > Yes which is generated from our use of SGML which is the core of this
> > problem and the core of the question as a whole.
> >
> > SGML is making working with the documentation *harder*.
> >
> > We have people that *DO NOT* contribute because of this SGML
> > requirement. They have what I consider extremely valid reasons, namely
> > it is dumb to require a writer to use emacs or write tags explictly.
>
> I'm one. I had the vacuuming stuff about written, but couldn't figure
> out the tool set this summer, and didn't have the time. Does OO make it
> pretty much painless to work with? I couldn't find any tools that I
> felt comfortable using for SGML / Docbooks.

OO treats Docbook like a normal document. You will however loose styles
(like bold, italic). It does support tables, it understands
transformation from things like sect1 (from OO heading1) etc...

The style loss is to be expected because Docbook doesn't contain
representation data. That belongs to a style sheet.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Fri, 2006-12-08 at 15:51, Joshua D. Drake wrote:
> On Fri, 2006-12-08 at 15:45 -0600, Scott Marlowe wrote:
> > On Fri, 2006-12-08 at 15:26, Joshua D. Drake wrote:
> > > On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> > > > Joshua D. Drake wrote:
> > > > > You can create, edit, convert, save, and open docbook xml in
> > > > > OpenOfice.org.
> > > >
> > > > Sure, there are more editing options with DocBook XML. No one disputes
> > > > that. But the question at hand was about processing the DocBook.
> > >
> > > Yes which is generated from our use of SGML which is the core of this
> > > problem and the core of the question as a whole.
> > >
> > > SGML is making working with the documentation *harder*.
> > >
> > > We have people that *DO NOT* contribute because of this SGML
> > > requirement. They have what I consider extremely valid reasons, namely
> > > it is dumb to require a writer to use emacs or write tags explictly.
> >
> > I'm one. I had the vacuuming stuff about written, but couldn't figure
> > out the tool set this summer, and didn't have the time. Does OO make it
> > pretty much painless to work with? I couldn't find any tools that I
> > felt comfortable using for SGML / Docbooks.
>
> OO treats Docbook like a normal document. You will however loose styles
> (like bold, italic). It does support tables, it understands
> transformation from things like sect1 (from OO heading1) etc...
>
> The style loss is to be expected because Docbook doesn't contain
> representation data. That belongs to a style sheet.

and let me add that I'm not really anti-sgml docbook, I just couldn't
find a "starter set" for editing the stuff. It seemed like everything I
found on docbook xml was written for people who already use docbook xml.

> > OO treats Docbook like a normal document. You will however loose styles
> > (like bold, italic). It does support tables, it understands
> > transformation from things like sect1 (from OO heading1) etc...
> >
> > The style loss is to be expected because Docbook doesn't contain
> > representation data. That belongs to a style sheet.
>
> and let me add that I'm not really anti-sgml docbook, I just couldn't
> find a "starter set" for editing the stuff. It seemed like everything I
> found on docbook xml was written for people who already use docbook xml.

Nor am I anti-sgml. I am however anti-noncontribution, if people are not
contributing because of our sgml but would if it is xml, that is a no
brainer.

Further, here is a real world problem that our toolset creates...

I take 5 minutes, change the stylesheet for SGML. I want to see what my
changes will look like... 3 days later, I will know.

That is stupid. If it was XML, it would be 30 minutes. That is a
workable timeframe.

Sincerely,

Joshua D. Drake

>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
>>> OO treats Docbook like a normal document. You will however loose styles
>>> (like bold, italic). It does support tables, it understands
>>> transformation from things like sect1 (from OO heading1) etc...
>>>
>>> The style loss is to be expected because Docbook doesn't contain
>>> representation data. That belongs to a style sheet.
>>>
>> and let me add that I'm not really anti-sgml docbook, I just couldn't
>> find a "starter set" for editing the stuff. It seemed like everything I
>> found on docbook xml was written for people who already use docbook xml.
>>
>
> Nor am I anti-sgml. I am however anti-noncontribution, if people are not
> contributing because of our sgml but would if it is xml, that is a no
> brainer.
>
> Further, here is a real world problem that our toolset creates...
>
> I take 5 minutes, change the stylesheet for SGML. I want to see what my
> changes will look like... 3 days later, I will know.
>
> That is stupid. If it was XML, it would be 30 minutes. That is a
> workable timeframe.
>
>
> Sincerely,
>
> Joshua D. Drake
>
>
>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 7: You can help support the PostgreSQL project by donating at
>>
>> http://www.postgresql.org/about/donate
>>
>>
I am not an expert in any of this but would say the one time I had
something I would have added to a doc (windows Vs *nix differences in
copy command) I was in knots with the whole SGML thing and figured it
would be quicker to just post an answer on the mailing list so it would
be immortalized, it should not be so hard to contribute. I feel there
are people who maybe cannot code or add in that way but could add to
docs if it was easier. I had an easier time setting up a pgAdmin build
than figuring out the SGML stuff.

Oisin

On 08/12/06, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> There is a long standing support within the community to move to XML
> including:
>
> Josh Berkus
> Josh Drake
> Robert Treat
> Andrew Dunslane
> David Blewett
> David Fetter
> Devrim Gunduz
> Darcy Buskermolen
>
> And that is just from #postgresql
>

And now in Chile, I wrote an app to translate postgres documentation
(l10n.postgresql.cl). I took SGML sources, transform to XML and to POT
files

The results are good! but is it not the best. To move from SGML to
XML is a hard work.

>

--
http://www.advogato.org/person/mgonzalez/

Joshua D. Drake a écrit :
> On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
>> Joshua D. Drake wrote:
>>> You can create, edit, convert, save, and open docbook xml in
>>> OpenOfice.org.
>> Sure, there are more editing options with DocBook XML. No one disputes
>> that. But the question at hand was about processing the DocBook.
>
> Yes which is generated from our use of SGML which is the core of this
> problem and the core of the question as a whole.
>
> SGML is making working with the documentation *harder*.
>

+1

> We have people that *DO NOT* contribute because of this SGML
> requirement. They have what I consider extremely valid reasons, namely
> it is dumb to require a writer to use emacs or write tags explictly.
>
> Hell, the only reason I have even bothered to contribute what little I
> have to the docs is because I wrote a book in SGML, thus it is a no
> brainer to me. Others aren't so tortured as to have done the same.
>

I'm not so sure it will help you find more contributors. I'm part of a
project which aims to translate HOWTO from TLDP. They don't find
contributors and we too have really hard times to find contributors
despite the fact we try to only use DocBook XML (TLDP use DocBook SGML,
DocBook XML and LinuxDoc formats).

Did you try to use OpenOffice.org with DocBook ? I tried once and it was
a complete disaster. But it was a long time ago. I will try again this
week-end.

> There is a long standing support within the community to move to XML
> including:
>
> Josh Berkus
> Josh Drake
> Robert Treat
> Andrew Dunslane
> David Blewett
> David Fetter
> Devrim Gunduz
> Darcy Buskermolen
>
> And that is just from #postgresql
>
> The french team also uses Docbook XML and they can generate a PDF in 30
> minutes... it takes us DAYS because of the SGML.
>

In fact, we need 15 minutes to build HTML files and 10 minutes to build
PDF file. To be completely honest, I don't seem to be able to build PDF
file for 8.2.0 release. I must have made a mistake (or perhaps a lot of
:) ).

Regards.

--
Guillaume.
<!-- http://abs.traduc.org/
http://lfs.traduc.org/
http://docs.postgresqlfr.org/ -->

On Fri, 2006-12-08 at 13:26 -0800, Joshua D. Drake wrote:
<snip>
> Yes which is generated from our use of SGML which is the core of this
> problem and the core of the question as a whole.
>
> SGML is making working with the documentation *harder*.

>From a total outsider's point of view I have to disagree. It took me a
couple of minutes to figure out how to make the tiny change I did the
other day by looking at the rest of the sgml (managed to get the diff
the wrong way around but not the sgml :/).

> We have people that *DO NOT* contribute because of this SGML
> requirement. They have what I consider extremely valid reasons, namely
> it is dumb to require a writer to use emacs or write tags explictly.

Again would have to disagree - surely if someone really wants to
contribute they could provide their input in plain text, and someone on
the list could then integrate those contribs.

> Hell, the only reason I have even bothered to contribute what little I
> have to the docs is because I wrote a book in SGML, thus it is a no
> brainer to me. Others aren't so tortured as to have done the same.

I would hate to hand edit the stuff generated by something like
OpenOffice.org.

> There is a long standing support within the community to move to XML
> including:
>
> Josh Berkus
> Josh Drake
> Robert Treat
> Andrew Dunslane
> David Blewett
> David Fetter
> Devrim Gunduz
> Darcy Buskermolen
>
> And that is just from #postgresql
>
> The french team also uses Docbook XML and they can generate a PDF in 30
> minutes... it takes us DAYS because of the SGML.

Here I agree - 30 minutes vs days is a good reason - as long as editing
with an ascii editor is not taken away.

--
Regards
Theo

Joshua D. Drake wrote:
> Further, here is a real world problem that our toolset creates...
>
> I take 5 minutes, change the stylesheet for SGML. I want to see what
> my changes will look like... 3 days later, I will know.
>
> That is stupid. If it was XML, it would be 30 minutes. That is a
> workable timeframe.

I feel like I'm talkling to a wall here, but I'm going to say this one
last time:

Any processing toolchain that you can use with DocBook XML you can use
with DocBook SGML and vice versa.

If you know of a way to create a PDF off DocBook XML in 30 minutes,
please tell us.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> As I said then, this is absolutely untrue. OpenOffice.org, for example,
> works with DocBook XML but not SGML. There are also a plethora of XML
> editing and publishing tools which can been used for Docbook XML which
> are not available for SGML. A simple look at this page:
> http://wiki.docbook.org/topic/DocBookAuthoringTools
> .... shows that there are more than twice as many authoring tools which
> support only XML as support SGML -- and that most of the tools which
> support SGML are out-of-maintenance.

This is confusing authoring tools (ie, stuff for more or less WYSIWYG
editing of the document source) with output generation tools.

As for authoring tools, show me one that produces SGML or XML that's
reasonably readable, and I might worry about allowing people to use it.
Most of the ones I've seen would render the doc sources unreadable for
anyone not using an authoring tool (possibly even the very same
authoring tool). We are not going to move in that direction
because it would piss off the people who do the bulk of the work now.

regards, tom lane

>> The french team also uses Docbook XML and they can generate a PDF in 30
>> minutes... it takes us DAYS because of the SGML.

Has anyone looked into actually fixing the performance problem?

oprofile results for jade trying to produce tex output from our docs are
suggestive of a localized performance issue:

samples % symbol name
2082917 98.5829 OpenJade_DSSSL::PairNodeListObj::nodeListFirst(OpenJade_DSSSL::EvalContext&, OpenJade_DSSSL::Interpreter&)
9713 0.4597 OpenJade_DSSSL::PairNodeListObj::nodeListRest(OpenJade_DSSSL::EvalContext&, OpenJade_DSSSL::Interpreter&)
5019 0.2375 OpenJade_DSSSL::AppendSosofoObj::traceSubObjects(Collector&) const
3571 0.1690 Collector::collect()
1938 0.0917 OpenJade_DSSSL::FlowObj::traceSubObjects(Collector&) const

I attached to the process with gdb and found it nested four thousand (!)
call levels deep in OpenJade_DSSSL::PairNodeListObj::nodeListFirst and
OpenJade_DSSSL::PairNodeListObj::nodeListRest calls. Meanwhile, looking
at the output-so-far-emitted makes me think it was working on a fairly
large <programlisting> example. The last little bit is:

{asis}\def\InputWhitespaceTreatment%
{preserve}}\Seq%
{}\Seq%
{}~~~~\endSeq{}/*
\Seq%
{}~~~~\endSeq{}~*~testlibpq2.c
\Seq%
{}~~~~\endSeq{}~*~~~~~~Test~of~the~asynchronous~notification~interface
\Seq%
{}~~~~\endSeq{}~*
\Seq%
{}~~~~\endSeq{}~*~Start~this~program,~then~from~psql~in~another~window~do
\Seq%
{}~~~~\endSeq{}~*~~~NOTIFY~TBL2;
\Seq%
{}~~~~\endSeq{}~*~Repeat~four~times~to~get~this~program~to~exit.
\Seq%
{}~~~~\endSeq{}~*
\Seq%
{}~~~~\endSeq{}~*~Or,~if~you~want~to~get~fancy,~try~this:
\Seq%
{}~~~~\endSeq{}~*~populate~a~database~with~th

What it looks like to me is that there is some bit of stupidity that is
producing a deeply nested list representation of a <programlisting>
section, probably one list level per character in the text, making the
runtime O(N^2) or worse in the length of the <programlisting>. (The
particular example it's stuck on here is about 10K characters.)

Since jade does not go into this kind of spiral when producing html
output from the same sources, I suggest that it's not jade's fault,
but rather crummy coding in the sgml-to-tex conversion scripts it's
using. I don't know enough about those to know where to look, but maybe
someone here does?

regards, tom lane

>
> > Hell, the only reason I have even bothered to contribute what little I
> > have to the docs is because I wrote a book in SGML, thus it is a no
> > brainer to me. Others aren't so tortured as to have done the same.
>
> I would hate to hand edit the stuff generated by something like
> OpenOffice.org.

As would I.

> >
> > The french team also uses Docbook XML and they can generate a PDF in 30
> > minutes... it takes us DAYS because of the SGML.
>
> Here I agree - 30 minutes vs days is a good reason - as long as editing
> with an ascii editor is not taken away.

I would never expect to take editing away.. I personally would likely
just use bluefish, not Openoffice.

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

> If you know of a way to create a PDF off DocBook XML in 30 minutes,
> please tell us.

See the post on this thread from :

Guillaume Lelarge

10 minutes to generate a pdf.

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> > If you know of a way to create a PDF off DocBook XML in 30 minutes,
> > please tell us.
>
> See the post on this thread from :
>
> Guillaume Lelarge
>
> 10 minutes to generate a pdf.

Right. So why don't we just use that?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sun, 2006-12-10 at 00:32 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > > If you know of a way to create a PDF off DocBook XML in 30 minutes,
> > > please tell us.
> >
> > See the post on this thread from :
> >
> > Guillaume Lelarge
> >
> > 10 minutes to generate a pdf.
>
> Right. So why don't we just use that?

How about the fact that we are putting undue burden on our fellow
regional projects?

How about it isn't in english?

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> On Sun, 2006-12-10 at 00:32 +0100, Peter Eisentraut wrote:
> > Joshua D. Drake wrote:
> > > > If you know of a way to create a PDF off DocBook XML in 30
> > > > minutes, please tell us.
> > >
> > > See the post on this thread from :
> > >
> > > Guillaume Lelarge
> > >
> > > 10 minutes to generate a pdf.
> >
> > Right. So why don't we just use that?
>
> How about the fact that we are putting undue burden on our fellow
> regional projects?
>
> How about it isn't in english?

Well, I didn't actually find a post titled "10 minutes to generate a
pdf." anywhere, so I assumed you referred to another one of his posts
which described how they used FOP to build PDF documentation. That
method does not burden anyone unduly and it is in English. But maybe
we're not talking about the same thing.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Tom Lane wrote:
> Since jade does not go into this kind of spiral when producing html
> output from the same sources, I suggest that it's not jade's fault,
> but rather crummy coding in the sgml-to-tex conversion scripts it's
> using.

Right. I fixed that, so now it takes about 15 minutes to build the
whole thing.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Tom Lane wrote:

>
> As for authoring tools, show me one that produces SGML or XML that's
> reasonably readable, and I might worry about allowing people to use it.
> Most of the ones I've seen would render the doc sources unreadable for
> anyone not using an authoring tool (possibly even the very same
> authoring tool). We are not going to move in that direction
> because it would piss off the people who do the bulk of the work now.
>

+1

With regards to the current setup discouraging contribution - I've done
one (or maybe two) doc submissions and ISTR that it was not very
difficult - even as a complete SGML newbie - to get up and running, make
my changes and build the (HTML) docs.

Cheers

Mark

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Tom Lane wrote:
>> Since jade does not go into this kind of spiral when producing html
>> output from the same sources, I suggest that it's not jade's fault,
>> but rather crummy coding in the sgml-to-tex conversion scripts it's
>> using.

> Right. I fixed that, so now it takes about 15 minutes to build the
> whole thing.

Actually, I just finished fixing what seems to be the underlying problem
in jade: it's got a spectacularly bad implementation of linked lists.
In PG-code terms, if the list is built by successive lappends(), then
both lfirst() and lnext() take O(N) time for an N-element list, thus
iterating through the list in the usual way is O(N^2) ... and not even
with a reasonably small multiplier, because it stresses the hell out of
memory allocation and garbage collection while it's at it. (Which may
well mean that it's really more like O(N^3), since the garbage collector
will iterate over every allocated object every so often.)

With the patch it takes me about 5 minutes to do the jade step of the
PDF build, using this morning's SGML sources. (I don't know how to set
the TeX configuration to get the pdfjadetex steps to go through, so I
dunno about total time.)

However, I have no idea what it'll take to get this patch propagated
into the copies people actually use, so your fix sounds good for the
short term.

regards, tom lane

diff -cr openjade-1.3.2.orig/style/ELObj.cxx openjade-1.3.2/style/ELObj.cxx
*** openjade-1.3.2.orig/style/ELObj.cxx Fri Jan 11 10:48:38 2002
--- openjade-1.3.2/style/ELObj.cxx Sat Dec 9 21:58:36 2006
***************
*** 1044,1049 ****
--- 1044,1054 ----
return new (interp) ReverseNodeListObj(this);
}

+ NodeListObj *NodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
+ {
+ return new (interp) HeadNodeListObj(this, newTail, interp);
+ }
+
long NodeListObj::nodeListLength(EvalContext &context, Interpreter &interp)
{
NodeListObj *nl = this;
***************
*** 1217,1236 ****

NodeListObj *PairNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
{
! if (!head_ || !head_->nodeListFirst(context, interp))
return tail_->nodeListRest(context, interp);
NodeListObj *tem = head_->nodeListRest(context, interp);
ELObjDynamicRoot protect(interp, tem);
! return new (interp) PairNodeListObj(tem, tail_);
}

NodeListObj *PairNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
{
! if (!head_ || !head_->nodeListFirst(context, interp))
return tail_->nodeListChunkRest(context, interp, chunk);
NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
ELObjDynamicRoot protect(interp, tem);
! return new (interp) PairNodeListObj(tem, tail_);
}

void PairNodeListObj::traceSubObjects(Collector &c) const
--- 1222,1260 ----

NodeListObj *PairNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
{
! if (!head_)
! return tail_->nodeListRest(context, interp);
! if (!head_->nodeListFirst(context, interp)) {
! head_ = 0;
return tail_->nodeListRest(context, interp);
+ }
NodeListObj *tem = head_->nodeListRest(context, interp);
ELObjDynamicRoot protect(interp, tem);
! if (!tem || !tem->nodeListFirst(context, interp))
! return tail_;
! return tem->nodeListAppend(tail_, interp);
}

NodeListObj *PairNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
{
! if (!head_)
! return tail_->nodeListChunkRest(context, interp, chunk);
! if (!head_->nodeListFirst(context, interp)) {
! head_ = 0;
return tail_->nodeListChunkRest(context, interp, chunk);
+ }
NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
ELObjDynamicRoot protect(interp, tem);
! if (!tem || !tem->nodeListFirst(context, interp))
! return tail_;
! return tem->nodeListAppend(tail_, interp);
! }
!
! NodeListObj *PairNodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
! {
! NodeListObj *tem = new (interp) PairNodeListObj(tail_, newTail);
! ELObjDynamicRoot protect(interp, tem);
! return new (interp) PairNodeListObj(head_, tem);
}

void PairNodeListObj::traceSubObjects(Collector &c) const
***************
*** 1239,1244 ****
--- 1263,1384 ----
c.trace(tail_);
}

+ CellNodeListObj::CellNodeListObj(NodeListObj *head)
+ : head_(head), next_(0)
+ {
+ hasSubObjects_ = 1;
+ }
+
+ NodePtr CellNodeListObj::nodeListFirst(EvalContext &context, Interpreter &interp)
+ {
+ if (head_) {
+ NodePtr nd(head_->nodeListFirst(context, interp));
+ if (nd)
+ return nd;
+ head_ = 0;
+ }
+ if (next_)
+ return next_->nodeListFirst(context, interp);
+ return NodePtr();
+ }
+
+ NodeListObj *CellNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
+ {
+ if (!head_) {
+ if (next_)
+ return next_->nodeListRest(context, interp);
+ return interp.makeEmptyNodeList();
+ }
+ if (!head_->nodeListFirst(context, interp)) {
+ head_ = 0;
+ if (next_)
+ return next_->nodeListRest(context, interp);
+ return interp.makeEmptyNodeList();
+ }
+ NodeListObj *tem = head_->nodeListRest(context, interp);
+ ELObjDynamicRoot protect(interp, tem);
+ if (!tem || !tem->nodeListFirst(context, interp)) {
+ if (next_)
+ return next_;
+ return interp.makeEmptyNodeList();
+ }
+ if (next_)
+ return tem->nodeListAppend(next_, interp);
+ return tem;
+ }
+
+ NodeListObj *CellNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
+ {
+ if (!head_) {
+ if (next_)
+ return next_->nodeListChunkRest(context, interp, chunk);
+ chunk = 0;
+ return interp.makeEmptyNodeList();
+ }
+ if (!head_->nodeListFirst(context, interp)) {
+ head_ = 0;
+ if (next_)
+ return next_->nodeListChunkRest(context, interp, chunk);
+ chunk = 0;
+ return interp.makeEmptyNodeList();
+ }
+ NodeListObj *tem = head_->nodeListChunkRest(context, interp, chunk);
+ ELObjDynamicRoot protect(interp, tem);
+ if (!tem || !tem->nodeListFirst(context, interp)) {
+ if (next_)
+ return next_;
+ return interp.makeEmptyNodeList();
+ }
+ if (next_)
+ return tem->nodeListAppend(next_, interp);
+ return tem;
+ }
+
+ void CellNodeListObj::traceSubObjects(Collector &c) const
+ {
+ c.trace(head_);
+ c.trace(next_);
+ }
+
+ HeadNodeListObj::HeadNodeListObj(NodeListObj *first, NodeListObj *second, Interpreter &interp)
+ : first_(0), last_(0)
+ {
+ hasSubObjects_ = 1;
+ ELObjDynamicRoot protect(interp, this);
+ first_ = new (interp) CellNodeListObj(first);
+ last_ = new (interp) CellNodeListObj(second);
+ first_->next_ = last_;
+ }
+
+ NodePtr HeadNodeListObj::nodeListFirst(EvalContext &context, Interpreter &interp)
+ {
+ return first_->nodeListFirst(context, interp);
+ }
+
+ NodeListObj *HeadNodeListObj::nodeListRest(EvalContext &context, Interpreter &interp)
+ {
+ return first_->nodeListRest(context, interp);
+ }
+
+ NodeListObj *HeadNodeListObj::nodeListChunkRest(EvalContext &context, Interpreter &interp, bool &chunk)
+ {
+ return first_->nodeListChunkRest(context, interp, chunk);
+ }
+
+ NodeListObj *HeadNodeListObj::nodeListAppend(NodeListObj *newTail, Interpreter &interp)
+ {
+ CellNodeListObj *tem = new (interp) CellNodeListObj(newTail);
+ last_->next_ = tem;
+ last_ = tem;
+ return this;
+ }
+
+ void HeadNodeListObj::traceSubObjects(Collector &c) const
+ {
+ c.trace(first_);
+ // first_ will recurse to the rest
+ }
+
ReverseNodeListObj::ReverseNodeListObj(NodeListObj *nl)
: nl_(nl), reversed_(0)
{
diff -cr openjade-1.3.2.orig/style/ELObj.h openjade-1.3.2/style/ELObj.h
*** openjade-1.3.2.orig/style/ELObj.h Fri Jan 11 10:48:38 2002
--- openjade-1.3.2/style/ELObj.h Sat Dec 9 20:14:23 2006
***************
*** 426,431 ****
--- 426,432 ----
virtual NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &chunk);
virtual NodePtr nodeListRef(long, EvalContext &, Interpreter &);
virtual NodeListObj *nodeListReverse(EvalContext &, Interpreter &);
+ virtual NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
virtual long nodeListLength(EvalContext &, Interpreter &);
virtual bool suppressError();
};
***************
*** 493,504 ****
--- 494,533 ----
NodePtr nodeListFirst(EvalContext &, Interpreter &);
NodeListObj *nodeListRest(EvalContext &, Interpreter &);
NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+ NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
void traceSubObjects(Collector &) const;
private:
NodeListObj *head_; // may be null
NodeListObj *tail_;
};

+ class CellNodeListObj : public NodeListObj {
+ public:
+ CellNodeListObj(NodeListObj *);
+ NodePtr nodeListFirst(EvalContext &, Interpreter &);
+ NodeListObj *nodeListRest(EvalContext &, Interpreter &);
+ NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+ void traceSubObjects(Collector &) const;
+ private:
+ NodeListObj *head_; // may be null
+ CellNodeListObj *next_; // may be null
+ friend class HeadNodeListObj; // needs access to next_ link
+ };
+
+ class HeadNodeListObj : public NodeListObj {
+ public:
+ HeadNodeListObj(NodeListObj *, NodeListObj *, Interpreter &);
+ NodePtr nodeListFirst(EvalContext &, Interpreter &);
+ NodeListObj *nodeListRest(EvalContext &, Interpreter &);
+ NodeListObj *nodeListChunkRest(EvalContext &, Interpreter &, bool &);
+ NodeListObj *nodeListAppend(NodeListObj *, Interpreter &);
+ void traceSubObjects(Collector &) const;
+ private:
+ // there are always at least two cells in the list, hence first_ != last_
+ CellNodeListObj *first_;
+ CellNodeListObj *last_;
+ };
+
inline
bool ELObj::equal(ELObj &obj1, ELObj &obj2)
{
diff -cr openjade-1.3.2.orig/style/primitive.cxx openjade-1.3.2/style/primitive.cxx
*** openjade-1.3.2.orig/style/primitive.cxx Wed Mar 13 07:21:22 2002
--- openjade-1.3.2/style/primitive.cxx Sat Dec 9 21:48:01 2006
***************
*** 3846,3870 ****

DEFPRIMITIVE(NodeList, argc, argv, context, interp, loc)
{
! if (argc == 0)
return interp.makeEmptyNodeList();
! int i = argc - 1;
! NodeListObj *nl = argv[i]->asNodeList();
if (!nl)
return argError(interp, loc,
! InterpreterMessages::notANodeList, i, argv[i]);
! if (i > 0) {
! ELObjDynamicRoot protect(interp, nl);
! for (;;) {
! i--;
NodeListObj *tem = argv[i]->asNodeList();
if (!tem)
! return argError(interp, loc,
! InterpreterMessages::notANodeList, i, argv[i]);
! nl = new (interp) PairNodeListObj(tem, nl);
! if (i == 0)
! break;
! protect = nl;
}
}
return nl;
--- 3846,3868 ----

DEFPRIMITIVE(NodeList, argc, argv, context, interp, loc)
{
! if (argc <= 0)
return interp.makeEmptyNodeList();
! NodeListObj *nl = argv[0]->asNodeList();
if (!nl)
return argError(interp, loc,
! InterpreterMessages::notANodeList, 0, argv[0]);
! if (argc > 1) {
! ELObjDynamicRoot protect(interp);
! ELObjDynamicRoot protect2(interp);
! for (int i = 1; i < argc; i++) {
! protect = nl;
NodeListObj *tem = argv[i]->asNodeList();
if (!tem)
! return argError(interp, loc,
! InterpreterMessages::notANodeList, i, argv[i]);
! protect2 = tem;
! nl = nl->nodeListAppend(tem, interp);
}
}
return nl;

I wrote:
> With the patch it takes me about 5 minutes to do the jade step of the
> PDF build, using this morning's SGML sources.

Ooops, that was with an -O0 build for debugging. After rebuilding with
the normal -O2 optimization, it takes about 70 seconds, just about
exactly the same as for the HTML-output case.

regards, tom lane

On Sun, 2006-12-10 at 03:13 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > On Sun, 2006-12-10 at 00:32 +0100, Peter Eisentraut wrote:
> > > Joshua D. Drake wrote:
> > > > > If you know of a way to create a PDF off DocBook XML in 30
> > > > > minutes, please tell us.
> > > >
> > > > See the post on this thread from :
> > > >
> > > > Guillaume Lelarge
> > > >
> > > > 10 minutes to generate a pdf.
> > >
> > > Right. So why don't we just use that?
> >
> > How about the fact that we are putting undue burden on our fellow
> > regional projects?
> >
> > How about it isn't in english?
>
> Well, I didn't actually find a post titled "10 minutes to generate a
> pdf." anywhere, so I assumed you referred to another one of his posts
> which described how they used FOP to build PDF documentation. That
> method does not burden anyone unduly and it is in English. But maybe
> we're not talking about the same thing.

I meant, *their* xml file is not english. So it doesn't do any good.
Remember this whole current thread started because we are trying to
create print ready documentation, which we currently do not have.

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

I wrote:
> With the patch it takes me about 5 minutes to do the jade step of the
> PDF build, using this morning's SGML sources. (I don't know how to set
> the TeX configuration to get the pdfjadetex steps to go through, so I
> dunno about total time.)

OK, I read Peter's notes about suitable TeX configuration, and with
that plus suitably patched openjade I get this from a standing start:

$ time make postgres.pdf
... much output ...
Output written on postgres.pdf (1670 pages, 13561269 bytes).
Transcript written on postgres.log.

real 8m23.949s
user 8m14.263s
sys 0m9.241s
$

But my goodness it produces a lot of TeX warnings :-(. I can understand
that no one was gonna look into those with a three-day turnaround time,
but maybe now we can think about cleaning them up? Personally I
haven't used TeX/LaTeX in anger in about ten years, but most of what
scrolled past looked easily fixable, and at least some of it would
translate to visible ugliness in the output.

regards, tom lane

On Sat, 2006-12-09 at 09:21 +0100, Guillaume Lelarge wrote:
> Joshua D. Drake a écrit :
> > On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> >> Joshua D. Drake wrote:
> >>> You can create, edit, convert, save, and open docbook xml in
> >>> OpenOfice.org.
> >> Sure, there are more editing options with DocBook XML. No one disputes
> >> that. But the question at hand was about processing the DocBook.
> >
> > Yes which is generated from our use of SGML which is the core of this
> > problem and the core of the question as a whole.
> >
> > SGML is making working with the documentation *harder*.
> >
>
> +1

I believe this point is being overlooked is people's willingness to code
a solution ;).

Guillaume could you please tell us *why* you ported the SGML to XML
before doing your translation and processing? Perhaps that will carry
more weight to this discussion then my hand waving.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Fri, 2006-12-08 at 22:49 -0300, Mario wrote:
> On 08/12/06, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> > There is a long standing support within the community to move to XML
> > including:
> >
> > Josh Berkus
> > Josh Drake
> > Robert Treat
> > Andrew Dunslane
> > David Blewett
> > David Fetter
> > Devrim Gunduz
> > Darcy Buskermolen
> >
> > And that is just from #postgresql
> >
>
> And now in Chile, I wrote an app to translate postgres documentation
> (l10n.postgresql.cl). I took SGML sources, transform to XML and to POT
> files
>
> The results are good! but is it not the best. To move from SGML to
> XML is a hard work.

I believe this point is being overlooked is people's willingness to code
a solution ;).

Mario could you please tell us *why* you ported the SGML to XML
before doing your translation and processing? Perhaps that will carry
more weight to this discussion then my hand waving.

Sincerely,

Joshua D. Drake

>
> >
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sat, Dec 09, 2006 at 09:21:12AM +0100, Guillaume Lelarge wrote:
> Joshua D. Drake a écrit :
> > On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
> >> Joshua D. Drake wrote:
> >>> You can create, edit, convert, save, and open docbook xml in
> >>> OpenOfice.org.
> >> Sure, there are more editing options with DocBook XML. No one disputes
> >> that. But the question at hand was about processing the DocBook.
> >
> > Yes which is generated from our use of SGML which is the core of this
> > problem and the core of the question as a whole.
> >
> > SGML is making working with the documentation *harder*.
>
> +1

Thanks to Peter and Tom for making the PDF build faster, but the more
general problem, which is that the SGML does not actually do the same
things that XML does, no matter how many times Peter so asserts,
remains.

In addition to the long-standing problem that there is no way to edit
the SGML docs with any known GUI tool, we have a particular use case,
namely producing a multi-volume set suitable for printing as books.

> > We have people that *DO NOT* contribute because of this SGML
> > requirement. They have what I consider extremely valid reasons,
> > namely it is dumb to require a writer to use emacs or write tags
> > explictly.

Peter, if you have a working example of a GUI tool that can be used
with the SGML source in its current form, the burden of proof is on
you to demonstrate it. Another flat assertion from you of some kind
of mathematical equivalence between SGML and XML will *not* do the
trick.

> > Hell, the only reason I have even bothered to contribute what
> > little I have to the docs is because I wrote a book in SGML, thus
> > it is a no brainer to me. Others aren't so tortured as to have
> > done the same.
>
> I'm not so sure it will help you find more contributors. I'm part of
> a project which aims to translate HOWTO from TLDP. They don't find
> contributors and we too have really hard times to find contributors
> despite the fact we try to only use DocBook XML (TLDP use DocBook
> SGML, DocBook XML and LinuxDoc formats).
>
> Did you try to use OpenOffice.org with DocBook ? I tried once and it
> was a complete disaster. But it was a long time ago. I will try
> again this week-end.

I gave it a try post-patches, and it's still a disaster 3284 pages of
un-rendered XML.

> > There is a long standing support within the community to move to XML
> > including:
> >
> > Josh Berkus
> > Josh Drake
> > Robert Treat
> > Andrew Dunslane
> > David Blewett
> > David Fetter
> > Devrim Gunduz
> > Darcy Buskermolen
> >
> > And that is just from #postgresql
> >
> > The french team also uses Docbook XML and they can generate a PDF
> > in 30 minutes... it takes us DAYS because of the SGML.
>
> In fact, we need 15 minutes to build HTML files and 10 minutes to
> build PDF file. To be completely honest, I don't seem to be able to
> build PDF file for 8.2.0 release. I must have made a mistake (or
> perhaps a lot of :) ).

It'll be nice to have the document building cycle shorter, but the
point here is that we need to enter the 21st century. That Tom found
a need to fork a document tool, i.e. take ownership of a whole large
piece of software, that being what forking means, is a neon sign that
means, "we're stuck with broken tools."

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

> In addition to the long-standing problem that there is no way to edit
> the SGML docs with any known GUI tool, we have a particular use case,
> namely producing a multi-volume set suitable for printing as books.

Well yes and no. There isn't really any good WYSIWYG tools, but there
are plenty of graphical validating editors for Docbook XML. I actually
don't agree with JoshBs argument on this one as most major FOSS (Gnome,
KDE, Linux) have moved to Docbook XML without the need of graphical
editors.

I do think the tools in general can be a little daunting and perhaps a
pgfoundry project that had a perl script that would download and
configure a self contained jail of tools for working with the PostgreSQL
docs would be good.

>
> > > We have people that *DO NOT* contribute because of this SGML
> > > requirement. They have what I consider extremely valid reasons,
> > > namely it is dumb to require a writer to use emacs or write tags
> > > explictly.
>
> Peter, if you have a working example of a GUI tool that can be used
> with the SGML source in its current form, the burden of proof is on
> you to demonstrate it. Another flat assertion from you of some kind
> of mathematical equivalence between SGML and XML will *not* do the
> trick.

I have to agree here, the reality is that the tools that are designed to
work with XML work with XML. Yes you *can* in theory make them work with
SGML, and some may work directly out of the box (Emacs would for editing
for example), not all of them would.

> > Did you try to use OpenOffice.org with DocBook ? I tried once and it
> > was a complete disaster. But it was a long time ago. I will try
> > again this week-end.
>
> I gave it a try post-patches, and it's still a disaster 3284 pages of
> un-rendered XML.

Well keep in mind that all XML is unrendered :). OpenOffice will do
things like this:

<emphasis>Hi</emphasis>
<para></para>
<para></para>
Bye

Which equates to:

Hi

Bye

We don't do that in the PostgreSQL sgml (thank goodness) so it comes out
really narly :). However it wouldn't take much to clean up small
sections for people.

> > In fact, we need 15 minutes to build HTML files and 10 minutes to
> > build PDF file. To be completely honest, I don't seem to be able to
> > build PDF file for 8.2.0 release. I must have made a mistake (or
> > perhaps a lot of :) ).
>
> It'll be nice to have the document building cycle shorter, but the
> point here is that we need to enter the 21st century. That Tom found
> a need to fork a document tool, i.e. take ownership of a whole large
> piece of software, that being what forking means, is a neon sign that
> means, "we're stuck with broken tools."

That is a big concern for me as well. I don't think Tom expects us to
maintain the code, and we can certainly try to get the patch submitted
to the open jade project which would definitely help. However, the
movement of the open jade projects appears basically dead.

Moving to XML based tools does give us a more modern and developed
toolset.

Sincerely,

Joshua D. Drake

>
> Cheers,
> D
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

David Fetter <david(at)fetter(dot)org> writes:
> In addition to the long-standing problem that there is no way to edit
> the SGML docs with any known GUI tool,

The point not in evidence is that there exists a GUI tool we'd accept
for editing XML-format docs. Can you point to some that don't mess up
XML source code to the point of being unreadable?

> ... That Tom found
> a need to fork a document tool, i.e. take ownership of a whole large
> piece of software, that being what forking means, is a neon sign that
> means, "we're stuck with broken tools."

Um, I spend all day every day on making sometimes-comparable
improvements in Postgres. Will you abandon Postgres as soon as
someone points out a(nother) serious performance bug in it?

As for "fork", I have no intention of forking anything --- that patch
is already submitted upstream.

regards, tom lane

Joshua D. Drake wrote:
> I do think the tools in general can be a little daunting and perhaps
> a pgfoundry project that had a perl script that would download and
> configure a self contained jail of tools for working with the
> PostgreSQL docs would be good.

Do you find any deficiencies in the setup instructions we have in the
documentation?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

David Fetter wrote:
> Peter, if you have a working example of a GUI tool that can be used
> with the SGML source in its current form, the burden of proof is on
> you to demonstrate it. Another flat assertion from you of some kind
> of mathematical equivalence between SGML and XML will *not* do the
> trick.

Well, most of you are obviously led into twisting every single thread on
this mailing list related to some sort of tool trouble into a thread
that SGML is completely unusable for unrelated reasons such as that
there are no editing tools or that it takes too long to build. I have
no interest in these sorts of uninformed discussions.

The question here was by David Blewett about creating multiple indexes.
Has anyone bothered to look into that? My experimentation, by the way,
shows that it works exactly as described in the cited document.

> It'll be nice to have the document building cycle shorter, but the
> point here is that we need to enter the 21st century. That Tom found
> a need to fork a document tool, i.e. take ownership of a whole large
> piece of software, that being what forking means, is a neon sign that
> means, "we're stuck with broken tools."

And which nonbroken tools for creating print output do you have up your
sleeve in the new XML world? And what is so wonderous about these
tools that we can't use them today?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Joshua D. Drake a écrit :
> On Sat, 2006-12-09 at 09:21 +0100, Guillaume Lelarge wrote:
>> Joshua D. Drake a écrit :
>>> On Fri, 2006-12-08 at 21:58 +0100, Peter Eisentraut wrote:
>>>> Joshua D. Drake wrote:
>>>>> You can create, edit, convert, save, and open docbook xml in
>>>>> OpenOfice.org.
>>>> Sure, there are more editing options with DocBook XML. No one disputes
>>>> that. But the question at hand was about processing the DocBook.
>>> Yes which is generated from our use of SGML which is the core of this
>>> problem and the core of the question as a whole.
>>>
>>> SGML is making working with the documentation *harder*.
>>>
>> +1
>
> I believe this point is being overlooked is people's willingness to code
> a solution ;).
>
> Guillaume could you please tell us *why* you ported the SGML to XML
> before doing your translation and processing? Perhaps that will carry
> more weight to this discussion then my hand waving.
>

I use DocBook XML a lot more than SGML on some translation's projects.

I switch to DocBook XML because I find it easier to customize
stylesheets. Take a look at DSSSL and take a look at XSLT : DSSL just
hurts my eyes, I found XSLT more simple to understand *and* customize.
There's been a lot of posts on this subject on some of the Linux From
Scratch project. And they've done a pretty good job with HTML and PDF
builts. So I took their stylesheets, Makefiles. I tried to understand
what they do, customize a little more to get what the french team have now.

So first argument, the one that makes me switch, is ease of
customization. Second one is also really important to me. XSLT
stylesheets and xsltproc are under active development. You can take a
look here :
http://sourceforge.net/project/showfiles.php?group_id=21935#files
DSSSL 1.79 has been packaged one 2004-11-03, XSL 1.71.1 has been
released on 2006-10-19. And don't get me started on tools. I've seen
recent releases of xsltproc, none of jade, openjade or any of those tools.

Last point : just look at other free projects. Take a look at Gnome and
KDE for example. They all use DocBook XML with customized stylesheets. I
don't mean that because lots of projects use DocBook XML, they all are
right and us wrong. I don't even mean that we should switch to follow
new buzz. I just mean people use it, publish information on what they do
and how they do it. Can someone find more informations on how to build
documentation with SGML than he can with XML ?

I don't want to say we can't do all these with DocBook SGML. Tom fixed
some stuff on openjade to make it process quickier. Great. We can also
customize DSSSL stylesheets. We can, we can... yes we can do a lot of
things with SGML and DSSSL. When I say "We", I should say "You" because
I don't want to get this burden. I mean, fixing a tool is great... but I
really prefer to see Tom working on PostgreSQL new astounding features
than on an old unmaintained and unused tool. I can't code as Tom. But I
can help on documentation with tools, but tools that are maintained. You
want to put more documentation on the manual but don't want to write it
with DocBook XML ? I'm OK with it. Just give me a plain ASCII file or an
openoffice file, I will handle the tagging, check correctness of the XML
file, build the documentation.

I say "I" but perhaps "we" would be more appropriate.

Perhaps it is time to create a real documentation team. More people will
be able to get involved with this great project.

--
Guillaume.
<!-- http://abs.traduc.org/
http://lfs.traduc.org/
http://docs.postgresqlfr.org/ -->

Joshua D. Drake wrote:
> On Fri, 2006-12-08 at 22:49 -0300, Mario wrote:

> > And now in Chile, I wrote an app to translate postgres documentation
> > (l10n.postgresql.cl). I took SGML sources, transform to XML and to POT
> > files
> >
> > The results are good! but is it not the best. To move from SGML to
> > XML is a hard work.
>
> I believe this point is being overlooked is people's willingness to code
> a solution ;).
>
> Mario could you please tell us *why* you ported the SGML to XML
> before doing your translation and processing? Perhaps that will carry
> more weight to this discussion then my hand waving.

The reason is that he uses xml2pot, which AFAIK only handles XML and not
SGML.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Guillaume Lelarge wrote:
<snip>

> Second one is also really important to me. XSLT
> stylesheets and xsltproc are under active development. You can take a
> look here :
> http://sourceforge.net/project/showfiles.php?group_id=21935#files
> DSSSL 1.79 has been packaged one 2004-11-03, XSL 1.71.1 has been
> released on 2006-10-19. And don't get me started on tools. I've seen
> recent releases of xsltproc, none of jade, openjade or any of those tools.

<snip>

>
> I don't want to say we can't do all these with DocBook SGML. Tom fixed
> some stuff on openjade to make it process quickier. Great. We can also
> customize DSSSL stylesheets. We can, we can... yes we can do a lot of
> things with SGML and DSSSL. When I say "We", I should say "You" because
> I don't want to get this burden. I mean, fixing a tool is great... but I
> really prefer to see Tom working on PostgreSQL new astounding features
> than on an old unmaintained and unused tool.

I think this is a good point. I've heard a saying in #postgresql on
IRC: "First rate database, second rate tools..." I think that is a
shame.

David
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfBx2Zmlc6wNjtLYRCOI4AJ9/v0+k3+zNDS/LjwPzYSM70ILBtwCeImbA
z8T/bF8aE/qOV1eUnHeFyCc=
=Hf45
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Peter Eisentraut wrote:
> > The question here was by David Blewett about creating multiple
indexes.
> > Has anyone bothered to look into that? My experimentation, by
the way,
> > shows that it works exactly as described in the cited document.

Peter:

Can you explain the process to me then? I couldn't figure out how to
generate different HTML.index files, or how to split the one
generated into different volumeindex.sgml files. Can you enlighten me?

David

PS:
Sorry if this gets posted twice, I'm having issues this morning...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfB01Zmlc6wNjtLYRCCeAAJ9DULf4miTCGljCNy37pm6YMaN5/ACgjELj
kplVuFLBstJeXFwM8N087u8=
=N3Zu
-----END PGP SIGNATURE-----

On Sun, 2006-12-10 at 09:43 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > I do think the tools in general can be a little daunting and perhaps
> > a pgfoundry project that had a perl script that would download and
> > configure a self contained jail of tools for working with the
> > PostgreSQL docs would be good.
>
> Do you find any deficiencies in the setup instructions we have in the
> documentation?

Where are they in the documentation?

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sun, 2006-12-10 at 10:08 +0100, Peter Eisentraut wrote:
> David Fetter wrote:
> > Peter, if you have a working example of a GUI tool that can be used
> > with the SGML source in its current form, the burden of proof is on
> > you to demonstrate it. Another flat assertion from you of some kind
> > of mathematical equivalence between SGML and XML will *not* do the
> > trick.
>
> Well, most of you are obviously led into twisting every single thread on
> this mailing list related to some sort of tool trouble into a thread
> that SGML is completely unusable for unrelated reasons such as that
> there are no editing tools or that it takes too long to build. I have
> no interest in these sorts of uninformed discussions.

Bah! That is certainly not what is happening here. No one has said that
they are completely unusable. Even I have said they are usable but that
they can be daunting.

The point of this discussion comes down to some very simple things.

1. Tools. SGML tools are not as actively developed as the XML ones. No
you can not use XML tools with SGML as effectively.

If I am wrong on #1 Peter, fine. Prove it. You keep telling us we are
wrong but are unwilling to share where or how.

2. We have two regional projects that take a lot of hard earned time to
work around the lackluster state of our source documentation, namely
SGML. This hard earned time would better be spent on other things and
could be if you would stop stonewalling and let us move to XML.

3. You are the only one that is aruging *for* staying SGML.

>
> The question here was by David Blewett about creating multiple indexes.
> Has anyone bothered to look into that? My experimentation, by the way,
> shows that it works exactly as described in the cited document.

Which cited document? However this still doesn't solve #2

Lastly, even if I am completely cranked on #1 and #3, #2 is reason
enough to move to XML regardless of your opinion.

We need to move to XML for the regional projects.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> 1. Tools. SGML tools are not as actively developed as the XML ones.
> No you can not use XML tools with SGML as effectively.
>
> If I am wrong on #1 Peter, fine. Prove it. You keep telling us we
> are wrong but are unwilling to share where or how.

Well, such statements are quite impossible to reason about because
a "tool" can be just about anything. There is no doubt that SGML tools
are not as actively developed as the XML ones. That could, for
example, be because SGML is much older and stabilized and the tools
have matured.

But none of this interests me unless someone can come up with a specific
completion of the following sentence: "We need to do $ACTION because
that allows us to use tool $TOOL, which is more efficient at doing
$TASK than the currently used tool."

The only completion I have heard about so far is ACTION='switch source
to XML', TOOL='OpenOffice', and TASK='editing'. Others have pointed
out that while OpenOffice can indeed edit DocBook XML, it wouldn't be
very useful for that purpose in our case. But I'm interested in
hearing more about that.

> 2. We have two regional projects that take a lot of hard earned time
> to work around the lackluster state of our source documentation,
> namely SGML.

I know of one regional project that has done documentation work, the
French one. (I'm sorry that I missed the other one.) Their reason for
moving to XML was (a) to use FOP for (b) producing print output more
efficiently. But you can use FOP right now, so (a) is not a good
reason, and we have fixed the print output generation now, so (b) isn't
a reason either.

> This hard earned time would better be spent on other
> things and could be if you would stop stonewalling and let us move to
> XML.

I'm not stonewalling anything. If someone wants to prepare a case for
moving to XML, be my guest. I'm just here to debunk the reasons for
moving to XML which are wrong.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Joshua D. Drake wrote:
> > Do you find any deficiencies in the setup instructions we have in
> > the documentation?
>
> Where are they in the documentation?

Appendix G

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sun, 2006-12-10 at 17:37 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > 1. Tools. SGML tools are not as actively developed as the XML ones.
> > No you can not use XML tools with SGML as effectively.
> >
> > If I am wrong on #1 Peter, fine. Prove it. You keep telling us we
> > are wrong but are unwilling to share where or how.
>
> Well, such statements are quite impossible to reason about because
> a "tool" can be just about anything. There is no doubt that SGML tools
> are not as actively developed as the XML ones. That could, for
> example, be because SGML is much older and stabilized and the tools
> have matured.

O.k. again... what tools? Either provide some specific solutions or back
out of the conversation.

The standard tools used in the FOSS arena (specifically linux) for
Docbook are OpenJade and DSSSL. OpenJade sees very little development.

What tools do you use? Are they the same listed in Appendix G (I am
assuming so)

>
> But none of this interests me unless someone can come up with a specific
> completion of the following sentence: "We need to do $ACTION because
> that allows us to use tool $TOOL, which is more efficient at doing
> $TASK than the currently used tool."

We need to do migrate our docs to XML because that allows us to use
xml-po, which is more efficient at helping translation than the
currently used process.

Close enough?

> > 2. We have two regional projects that take a lot of hard earned time
> > to work around the lackluster state of our source documentation,
> > namely SGML.
>
> I know of one regional project that has done documentation work, the
> French one. (I'm sorry that I missed the other one.) Their reason for
> moving to XML was (a) to use FOP for (b) producing print output more
> efficiently. But you can use FOP right now, so (a) is not a good
> reason, and we have fixed the print output generation now, so (b) isn't
> a reason either.

No, they also wanted easier stylesheet maintenance.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sun, 2006-12-10 at 17:37 +0100, Peter Eisentraut wrote:

> I'm not stonewalling anything. If someone wants to prepare a case for
> moving to XML, be my guest. I'm just here to debunk the reasons for
> moving to XML which are wrong.
>

O.k. how about this:

We want to move to XML. We have community members that want to move to
XML. We have regional projects that want to move to XML.

You can not debunk any of those reasons.

The fact is, they want to.

Until you can come up with a reason why we *shouldn't* move to XML. I
believe this issue is settled.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sun, 2006-12-10 at 17:38 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > > Do you find any deficiencies in the setup instructions we have in
> > > the documentation?
> >
> > Where are they in the documentation?
>
> Appendix G

These are good instruction, but not anywhere as simple as:

Download this script.
Run this script.
type make, make pdf, make rtf etc...

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Peter Eisentraut a écrit :
> Joshua D. Drake wrote:
>> 2. We have two regional projects that take a lot of hard earned time
>> to work around the lackluster state of our source documentation,
>> namely SGML.
>
> I know of one regional project that has done documentation work, the
> French one. (I'm sorry that I missed the other one.) Their reason for
> moving to XML was (a) to use FOP for (b) producing print output more
> efficiently. But you can use FOP right now, so (a) is not a good
> reason, and we have fixed the print output generation now, so (b) isn't
> a reason either.
>

We didn't switch to use FOP. We switched to build better PDF files
(better --> prettier). Perhaps we could have done the same with SGML but
I had a great way to do it with XML and XSLT (and FOP), a way I can
understand, customize, enhance, etc.

The last PDF available on the website has some bad issues. Just take a
look at table 8-13 (page 805), table 8-18 (page 113). One more time, I
don't say you can't fix this in SGML (and I think we already talk of
this some time ago). But *I* won't do it. And I think some people are
willing to work on this but they also want to have XML documentation.

Perhaps we can switch to XML just because people who want to get
involved with PostgreSQL documentation want to use XML ?

>> This hard earned time would better be spent on other
>> things and could be if you would stop stonewalling and let us move to
>> XML.
>
> I'm not stonewalling anything. If someone wants to prepare a case for
> moving to XML, be my guest. I'm just here to debunk the reasons for
> moving to XML which are wrong.
>

--
Guillaume.
<!-- http://abs.traduc.org/
http://lfs.traduc.org/
http://docs.postgresqlfr.org/ -->

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> Until you can come up with a reason why we *shouldn't* move to XML. I
> believe this issue is settled.

How about "the people who are actually doing the work today don't
want to change to XML"?

I'm not very excited by hopes that all sorts of documentation manpower
will suddenly materialize if only the docs were in XML not SGML. *That*
is the part of this argument for which no evidence has been produced.
Where are the people who know the code well enough to document it
but refuse to work on the docs because they're not XML?

(And no, Peter is not the only one objecting.)

regards, tom lane

Joshua D. Drake wrote:
> O.k. again... what tools? Either provide some specific solutions or
> back out of the conversation.

No, that is up to you, because you started handwaving about "tools". I
know my tools, but then I'm not the one complaining that we don't have
enough or the right ones or whatever.

> We need to do migrate our docs to XML because that allows us to use
> xml-po, which is more efficient at helping translation than the
> currently used process.

Well, that seems reasonable, but I looked around for something
called "xmlpo" or "xml-po", but I could find neither a binary nor a
package by that name on my system. Googling suggests that it is a
KDE-specific tool. Can someone clarify?

> No, they also wanted easier stylesheet maintenance.

$EDITOR stylesheet.xsl

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sun, 2006-12-10 at 12:11 -0500, Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> > Until you can come up with a reason why we *shouldn't* move to XML. I
> > believe this issue is settled.
>
> How about "the people who are actually doing the work today don't
> want to change to XML"?

O.k. that's obviously reasonable enough but nobody has stepped up and
said. "Hey, I am doing the work here, I don't want to use XML."

I can easily provide the counter argument that says, "Why don't the
users of SGML want to migrate to XML?" Not one person has stepped up and
said it will cause me undue amounts of grief, pain, or even... I just
hate XML.

>
> I'm not very excited by hopes that all sorts of documentation manpower
> will suddenly materialize if only the docs were in XML not SGML.

I am not a believe of suddenly materializing manpower either ;). However
I do know that the regional projects would be greatly appreciative.

> *That*
> is the part of this argument for which no evidence has been produced.
> Where are the people who know the code well enough to document it
> but refuse to work on the docs because they're not XML?

Where are the people that know the code well enough to document it but
refuse to work on the docs because they are XML?

I am not trying to be sarcastic here, honestly but not even in this
email have you said: "I don't like XML, I don't want to move to it
because..."

>
> (And no, Peter is not the only one objecting.)

O.k. why then? Why are you objecting? Peter has yet to provide a reason
either?

What we have here is a dozen or so people that want to move to XML and
have stated publicly that they want to do so, as well as the reasons
why.

We also have two people who have said, that they want to stay with SGML
but have provided zero reason why.

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:

> We want to move to XML. We have community members that want to move
> to XML. We have regional projects that want to move to XML.

"Because I want to" is not a reason in my mind.

> Until you can come up with a reason why we *shouldn't* move to XML. I
> believe this issue is settled.

I have previously listed the issues in favor and against. Check the
start of this thread.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Joshua D. Drake wrote:
> These are good instruction, but not anywhere as simple as:
>
> Download this script.
> Run this script.
> type make, make pdf, make rtf etc...

Then start writing that script.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Guillaume Lelarge wrote:
> We didn't switch to use FOP. We switched to build better PDF files
> (better --> prettier). Perhaps we could have done the same with SGML
> but I had a great way to do it with XML and XSLT (and FOP), a way I
> can understand, customize, enhance, etc.

Well, we've got XSLT stylesheets, so use them.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sun, 2006-12-10 at 18:26 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > These are good instruction, but not anywhere as simple as:
> >
> > Download this script.
> > Run this script.
> > type make, make pdf, make rtf etc...
>
> Then start writing that script.

I plan on it. That is why I brought it up.

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

>
> > We need to do migrate our docs to XML because that allows us to use
> > xml-po, which is more efficient at helping translation than the
> > currently used process.
>
> Well, that seems reasonable, but I looked around for something
> called "xmlpo" or "xml-po", but I could find neither a binary nor a
> package by that name on my system. Googling suggests that it is a
> KDE-specific tool. Can someone clarify?

Sorry, xml2pot

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

> > Until you can come up with a reason why we *shouldn't* move to XML. I
> > believe this issue is settled.
>
> I have previously listed the issues in favor and against. Check the
> start of this thread.

The only thing I see is that you don't want to move because all the
tools that are available for docbook sgml are also available for docbook
xml.

Am I missing anything?

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> The only thing I see is that you don't want to move because all the
> tools that are available for docbook sgml are also available for
> docbook xml.

Link posted by David Blewett:
http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7
--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sun, 2006-12-10 at 19:07 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > The only thing I see is that you don't want to move because all the
> > tools that are available for docbook sgml are also available for
> > docbook xml.
>
> Link posted by David Blewett:
> http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7

Ahh that would explain why I couldn't find it. :). Thanks.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> Sorry, xml2pot

OK, found it. Works fine.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On 10/12/06, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
>
> >
> > Well, that seems reasonable, but I looked around for something
> > called "xmlpo" or "xml-po", but I could find neither a binary nor a
> > package by that name on my system. Googling suggests that it is a
> > KDE-specific tool. Can someone clarify?
>
> Sorry, xml2pot
>

no, is xml2po and you can find in debian in a package called gnome-doc-utils

>

--
http://www.advogato.org/person/mgonzalez/

On 10/12/06, Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> >
> > Mario could you please tell us *why* you ported the SGML to XML
> > before doing your translation and processing? Perhaps that will carry
> > more weight to this discussion then my hand waving.
>
> The reason is that he uses xml2pot, which AFAIK only handles XML and not
> SGML.
>

Yes, that's right, I used and I'm using xml2po. Cause, we wanted to
create an easy way to translate manuals and the only tool I know for
that is xml2po. That tool AFAIK works with XML only, that's way I did
that port and also we wanted to translate online, so the current
configuration is

SGML -> XML -> POT -> WEBAPP -> PO_TRANSLATED -> XML_TRANSLATED

AND from the XML_TRANSLATED we are able to (thanks to openjade)
generate the HTML files in +10 minutes

>

--
http://www.advogato.org/person/mgonzalez/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Peter Eisentraut wrote:
> But none of this interests me unless someone can come up with a specific
> completion of the following sentence: "We need to do $ACTION because
> that allows us to use tool $TOOL, which is more efficient at doing
> $TASK than the currently used tool."

Sorry to butt in, but I'm still not sure how to do multiple indexes
in one volume using the DSSSL stylesheets. The docs I pointed to
were for the XSL stylesheets. After some discussion in #docbook, it
was said that it was a recent addition to those stylesheets.

David
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfLHPZmlc6wNjtLYRCGp8AJ0aVEbI/Ztlx5UsAY7KSW7vWz6IqACgudT5
VuD8FcZzajFUMNgAdPCPXJE=
=AD66
-----END PGP SIGNATURE-----

Am Montag, 11. Dezember 2006 02:18 schrieb David Blewett:
> Sorry to butt in, but I'm still not sure how to do multiple indexes
> in one volume using the DSSSL stylesheets.

I'm not sure either.

> The docs I pointed to were for the XSL stylesheets.

Right. So using the XSL(T) stylesheet instead of the DSSSL stylesheet would
be a step in the right direction. Actually the 8.2 branch currently only has
an XSLT stylesheet for HTML output. In 8.3devel I've added one for XSL-FO
output, which you can invoke by "make postgres-{A4|US}.fo". (You will
probably want yet a different paper format for your book.)

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Folks,

Tom and Peter correctly point out that discussion of production tools and
authoring tools are separate, and only come together if there are tools for
XML which solve both issues, an assertion which is not yet proven.

I am one of the champions of XML simply because I know for a fact that there
are no real authoring tools* for SGML. However, I do *not* know that the
tools for XML produce acceptable output.

So, questions to answer:

1) Are there enhanced tools for Docbook XML, WYSWYG or otherwise, which make
doc authoring easier and produce correct output for PostgreSQL Docs?
1) a) is there some way we can try various tools and check output?

2) If (1), would switching to XML make doc writing with Emacs/VIM harder?

3) If (1), will having authoring tools available increase the number of doc
contributors?

Regarding (3), I believe that having authoring tools would increase our pool
based on two things:
a) Anectodal evidence, including several people (like me) who were prevented
from doing significant doc clean-up by the amount of labor required to
restructure large SGML documents.
b) That the vast majority of the OSS world is no longer using SGML, meaning
that even for hand-editing people who work on other projects will be more
familiar with XML and thus better able to edit it.

While these two may not be enough to answer "should we migrate", they are
enough to make it worth trying to find the answer to (1), since if all of the
Docbook XML authoring tools are crappy the whole discussion is moot.

So, can someone help me come up with a way to validate a Docbook XML version
of the PostgreSQL docs so that I (and others, hopefully) can start testing
tools?

* = an authoring tool is one which makes generation of the document
easier/faster than hand-editing text files. No such tool exists for SGML --
even the Emacs toolkit merely does validation.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Peter Eisentraut wrote:
> Right. So using the XSL(T) stylesheet instead of the DSSSL stylesheet would
> be a step in the right direction. Actually the 8.2 branch currently only has
> an XSLT stylesheet for HTML output. In 8.3devel I've added one for XSL-FO
> output, which you can invoke by "make postgres-{A4|US}.fo". (You will
> probably want yet a different paper format for your book.)

Excuse my ignorance, but I thought XSLT stylesheets can only be used
on XML files?

David Blewett

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfZ8KZmlc6wNjtLYRCEsTAJ0YF/5/enV2w7ibvjSYENTdKjZs3QCgophl
tZVTZsrFceHD3f+KKn6Drqo=
=zhwv
-----END PGP SIGNATURE-----

> So, questions to answer:
>
> 1) Are there enhanced tools for Docbook XML, WYSWYG or otherwise, which make
> doc authoring easier and produce correct output for PostgreSQL Docs?

Anything that you produce from a WYSWYG editor is going to have to be
massaged to work with PostgreSQL.Org docs.

> 1) a) is there some way we can try various tools and check output?

OpenOffice.Org, WordPerfect, Kword, Abiword. The first two (last I
checked) were the only mature software supporting Docbook output.

>
> 2) If (1), would switching to XML make doc writing with Emacs/VIM harder?

Absolutely not.

>
> 3) If (1), will having authoring tools available increase the number of doc
> contributors?

It might. However, getting authors will always be difficult. Having
clear direction from the website on how to contribute documentation.
This should include how to contribute if you don't know Docbook. Right
now, you have to dig all the way to Appendix G (who reads apepedixes?).

Something like this would also be helpful:

http://www.tldp.org/LDP/LDP-Author-Guide/html/index.html

> Regarding (3), I believe that having authoring tools would increase our pool
> based on two things:
> a) Anectodal evidence, including several people (like me) who were prevented
> from doing significant doc clean-up by the amount of labor required to
> restructure large SGML documents.

Well to be honest, there isn't really anyway around that that I know of.
OpenOffice.Org will allow you to do what you want, but someone is going
to have to clean up after you.

Now, that is not neccessarily a road block. It is certainly plausible
that someone could write a section (although I think plain text would be
better here) or rewrite a section and submit it for factoring. Factoring
would involve any Docbook that needed to be done.

Usually the Docbook that is required is trivial.

> b) That the vast majority of the OSS world is no longer using SGML, meaning
> that even for hand-editing people who work on other projects will be more
> familiar with XML and thus better able to edit it.

Well that is certainly true.

> So, can someone help me come up with a way to validate a Docbook XML version
> of the PostgreSQL docs so that I (and others, hopefully) can start testing
> tools?

>From the source tree:

make postgres.xml

Then validate as you see fit. Just FYI, it does validate.

>
>
> * = an authoring tool is one which makes generation of the document
> easier/faster than hand-editing text files. No such tool exists for SGML --
> even the Emacs toolkit merely does validation.
>

What is it specifically you are looking for then? Because that is what
Docbook is, XML (or sgml) the only thing you need to do is validate.

http://www.commandprompt.com/community/emacs_sgml/

If you use OpenOffice you will have to set up a style to correct mirror
the PostgreSQL Docbook usage.

Is that perhaps useful? An OpenOffice style that people could download
that is documented and can be used for creating reasonably useful
Docbook for the community?

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Mon, 2006-12-11 at 13:10 -0500, David Blewett wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Peter Eisentraut wrote:
> > Right. So using the XSL(T) stylesheet instead of the DSSSL stylesheet would
> > be a step in the right direction. Actually the 8.2 branch currently only has
> > an XSLT stylesheet for HTML output. In 8.3devel I've added one for XSL-FO
> > output, which you can invoke by "make postgres-{A4|US}.fo". (You will
> > probably want yet a different paper format for your book.)
>
> Excuse my ignorance, but I thought XSLT stylesheets can only be used
> on XML files?

AFAIK they can only be used on XML files, but you can get XML from the
postgresql sgml sources by using make postgres.xml .

Of course your changes that you make from release to release for volumes
won't be permanent and we will continue to waste time because people
have an odd affection for SGML like it was some kind of pet dog.

Joshua D. Drake

>
> David Blewett
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFFfZ8KZmlc6wNjtLYRCEsTAJ0YF/5/enV2w7ibvjSYENTdKjZs3QCgophl
> tZVTZsrFceHD3f+KKn6Drqo=
> =zhwv
> -----END PGP SIGNATURE-----
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
>> 1) a) is there some way we can try various tools and check output?

> OpenOffice.Org, WordPerfect, Kword, Abiword. The first two (last I
> checked) were the only mature software supporting Docbook output.

I tried opening postgres.xml in OpenOffice 2.0, and it just showed me
the raw text and markup --- no indication that it understood xml at all.
Is there some special incantation needed?

regards, tom lane

On Mon, 2006-12-11 at 13:34 -0500, Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> >> 1) a) is there some way we can try various tools and check output?
>
> > OpenOffice.Org, WordPerfect, Kword, Abiword. The first two (last I
> > checked) were the only mature software supporting Docbook output.
>
> I tried opening postgres.xml in OpenOffice 2.0, and it just showed me
> the raw text and markup --- no indication that it understood xml at all.
> Is there some special incantation needed?

I found that I had to remove the <? xml declartion and only have the
docbook dtd declartion.

Warning, it is ugly :) but it does indeed work.

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Sun, 2006-12-10 at 19:07 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > The only thing I see is that you don't want to move because all the
> > tools that are available for docbook sgml are also available for
> > docbook xml.
>
> Link posted by David Blewett:
> http://groups.google.com/group/pgsql.docs/browse_thread/thread/37ff3a011bb705d7

Per this link Peter you cite:
"""
- Marked sections don't work anymore; would need to use DocBook-specific
profiling mechanism, which isn't all that elegant.

- More typing: Things like <abc>foo</> and other abbreviations won't
work anymore; all attributes would need to be quoted, etc.

- doc/src/sgml/*.sgml will look silly for filenames.

- Someone would need to do the conversion. I understand that the
French
translation team might have patches available.

So that is it. In my mind, there is no clear winner, but if someone has
a concrete need for XML, I don't see a problem with it.
"""

Your first disadvantage I can not comment on.

Your second disadvantage is fixed with any reasonable editor including
emacs.

Your third comment is a filename change.

Your fourth is resolved, we have the conversion per make postgres.xml.
We could even reasonably break it all out again.

I would add to advantages that:

1. We wouldn't have to have this discussion every six months :)

2. It will make life easier for regional projects as they are already
going through the pain of moving to XML for each release.

3. It would help some sub projects including the printed documentation
project that Blewitt and I are working on because we could submit our
patches as XML directly back to the community.

Lastly: There are several who have mentioned a concrete need. Is the
need solved with the existing tools? Yes, but not without extra steps.

So I would ask that you either give us further reason why you don't want
to move to XML or honor what you say above, in that you don't have a
problem with it.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> On Mon, 2006-12-11 at 13:34 -0500, Tom Lane wrote:
>> I tried opening postgres.xml in OpenOffice 2.0, and it just showed me
>> the raw text and markup --- no indication that it understood xml at all.
>> Is there some special incantation needed?

> I found that I had to remove the <? xml declartion and only have the
> docbook dtd declartion.

> Warning, it is ugly :) but it does indeed work.

Only for exceedingly small values of "work". I tried that, and got
a display showing all the text run together in a single paragraph.
I changed one word and re-saved, and got a file that had basically
nothing to do with the original --- it's been arbitrarily reformatted,
and most of the original markup is gone.

Perhaps later versions are better, but the OO version currently available
in Fedora 5 appears utterly useless for our purposes.

Can you show me an authoring tool that does *not* think it's OK to
mangle the low-level text in "semantically irrelevant" ways? We might
as well forget about the concept of a doc patch if people start using
tools like this --- if a one-word change results in diffs across the
whole file, the tool is not usable.

regards, tom lane

> Can you show me an authoring tool that does *not* think it's OK to
> mangle the low-level text in "semantically irrelevant" ways?

No. :) I mentioned this previously. Any word processor is going to blow
stuff away in an ugly way. The closest we could get is:

Create a custom style in OpenOffice that works in a similar fashion to
PostgreSQL.

Use Tidy and or a custom script to clean after the fact.

IMHO the only tools for Docbook that are usable are editors, preferrably
validating editors such as Emacs, Bluefish (actually Word Perfect does
this too). We are not going to get what we want out of WYSIWYG.

> We might
> as well forget about the concept of a doc patch if people start using
> tools like this --- if a one-word change results in diffs across the
> whole file, the tool is not usable.

You are correct, we would have to run some kind of clean script and then
diff between the two versions.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Josh,

> Anything that you produce from a WYSWYG editor is going to have to be
> massaged to work with PostgreSQL.Org docs.

*sigh* too bad Lyx only writes DocBook and doesn't read it.

> > * = an authoring tool is one which makes generation of the document
> > easier/faster than hand-editing text files. No such tool exists for
> > SGML -- even the Emacs toolkit merely does validation.
>
> What is it specifically you are looking for then? Because that is what
> Docbook is, XML (or sgml) the only thing you need to do is validate.

"Authoring Tool" means "not always hand-editing tags". Right now, I can't
do anything with Emacs SGML that I couldn't do with Wordpad or Pico,
except validate.

Let me give you an example of a doc change which needing to hand-edit SGML
prevented me from making: I wanted to clean up runtime.sgml by adding an
alpha index, and clearly breaking out data like defaults, set time, etc.
This meant changing 70% of the document, including some significant
re-arranging.

After 5 hours of editing, I discovered that I was only 1/3 done ... and
that there had been incompatible changes in CVS in the meantime. I gave
up.

If the docs were WYSWYG-editable or if we had any authoring tool which
would automatically fix tags, I would have been able to make the changes
and submit a patch in < 4 hours, not 15 hours+ it would have taken to
complete the project.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

> "Authoring Tool" means "not always hand-editing tags". Right now, I can't
> do anything with Emacs SGML that I couldn't do with Wordpad or Pico,
> except validate.

That is certainly not true. Emacs will correctly associate your tags
without having to write those tags, if you use Emacs correctly. As will
Bluefish.

http://www.commandprompt.com/community/emacs_sgml/x108

Bluefish is a little easier with this IMHO but it gives the same
capabilities.

>
> Let me give you an example of a doc change which needing to hand-edit SGML
> prevented me from making: I wanted to clean up runtime.sgml by adding an
> alpha index, and clearly breaking out data like defaults, set time, etc.
> This meant changing 70% of the document, including some significant
> re-arranging.
>
> After 5 hours of editing, I discovered that I was only 1/3 done ... and
> that there had been incompatible changes in CVS in the meantime. I gave
> up.

Well the CVS issue will be one that always exists.

>
> If the docs were WYSWYG-editable or if we had any authoring tool which
> would automatically fix tags,

O.k. well you will you lost my support here. I guarantee you I would not
submit patches if I had to work with problems associated with doing
things in OpenOffice.Org format. I would not have a problem helping get
a clean up script together that would work with Docbook output from
OpenOffice.org though.

Docbook is specifically designed for technical documentation.

> I would have been able to make the changes
> and submit a patch in < 4 hours, not 15 hours+ it would have taken to
> complete the project.

Wait... You want a tool to automatically fix tags? You do need to know
Docbook to write the docs JoshB. That means you need to know which tags
are relevant to what. Something like emacs will make this easier because
it will tell you what tags are valid for each section of the document
you are in.

Sincerely,

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> Your fourth is resolved, we have the conversion per make postgres.xml.

We do not have that --- have you looked at the output? It's nigh
unreadable. Same problem as with the authoring tools: that software
doesn't think that formatting of the XML source text is important.

regards, tom lane

On Mon, 2006-12-11 at 14:53 -0500, Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> > Your fourth is resolved, we have the conversion per make postgres.xml.
>
> We do not have that --- have you looked at the output? It's nigh
> unreadable.

You are correct which is actually the argument that Gleu made to me
about moving to XML. I did find however that if you ran postgres.xml
through tidy it cleaned things up nicely.

Secondly the make postgres.xml would be a one time thing. I would
happilly take the time to get postgres.xml into a format that was
reasonable and similar to how we currently do the postgres.sgml docs.

> Same problem as with the authoring tools: that software
> doesn't think that formatting of the XML source text is important.

Well not authoring editors though. Emacs or Bluefish won't blow that
stuff away (just realized Bluefish doesn't validate :( ).

>
> regards, tom lane
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Josh,

> Wait... You want a tool to automatically fix tags? You do need to know
> Docbook to write the docs JoshB. That means you need to know which tags
> are relevant to what. Something like emacs will make this easier because
> it will tell you what tags are valid for each section of the document
> you are in.

Sure, but that doesn't do any good if you want to cut-and-paste a section
and fix the tags.

Anyway, from the sound of it whether or not there are better authoring
tools for XML than SGML is "undetermined pending further investigation."

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> Secondly the make postgres.xml would be a one time thing.

Agreed, the cost of conversion is one-time ... but it's not small.
Aside from getting the files themselves converted, there's the effort
for people to find, install, and learn suitable tools, not to mention
come up to speed on the differences between xml and sgml. (I assume
there are some significant ones, else why are we having this discussion?)

The real problem here is that you've still failed to establish any
sizable benefit from converting. As best I can tell at the moment,
the acceptable options for editing XML will be about the same as they
are for SGML: emacs, and not a lot else. I don't really see why I
should have to start spelling out every closing tag for a no-op like
that.

regards, tom lane

On Mon, 2006-12-11 at 15:07 -0500, Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> > Secondly the make postgres.xml would be a one time thing.
>
> Agreed, the cost of conversion is one-time ... but it's not small.
> Aside from getting the files themselves converted, there's the effort
> for people to find, install, and learn suitable tools, not to mention
> come up to speed on the differences between xml and sgml. (I assume
> there are some significant ones, else why are we having this discussion?)

Well this actually comes back to one of Peter's points :). You shouldn't
need to update your tools. If you installed Docbook tools, it is going
to come with DSSSL and XSL. OpenJade will process the XML as will any of
the current db2, docbook2 etc.. scripts.

As far as learning the differences between xml and sgml, they are very
subtle because it is still Docbook.

http://www.oreilly.com/catalog/docbook/chapter/book/appb.html
http://www.docbook.org/tdg/en/html/ch02.html

For most things that anyone would be doing, they will see extremely
little difference. I didn't even notice the difference because I never
used short tags.

> The real problem here is that you've still failed to establish any
> sizable benefit from converting. As best I can tell at the moment,
> the acceptable options for editing XML will be about the same as they
> are for SGML: emacs, and not a lot else.

Well basically, editing :) any editor although some make it easier as I
mentioned Emacs, BlueFish, Kate etc...

The benefits are (in my mind):

1. At least some of our regionals are already using XML. Let's make it a
little easier for them to do so.

2. It would make the new printed documentation project easier for David
Blewitt (the guy that is doing it)

3. DocBook XML is where FOSS documentation is moving or has moved
depending on the project. Should any project want to integrate any part
of our docs to their docs or vice versa, it will be easier for them if
we follow the similar standard of using Docbook XML.

4. Their is active and progressive development with XSL and Docbook.
There is not (that I can see) the same level with DSSSL and Docbbok.

5. The format objects development (FOP) seems to be better maintained
then the dblatex/db2latex/pdflatex stuff. FOP does not appear to work
with SGML.

6. Nobody has really come up with a reason *not* to move to XML that
can't easily be addressed.

7. And this may be ethereal but I do believe that if we were following
in the footsteps of other FOSS projects we may be able to get writers
from those projects. Even only 1 of them.

> I don't really see why I
> should have to start spelling out every closing tag for a no-op like
> that.

Fair enough but if you are using a decent editor, it will take care of
that for you.

To be honest. I do not care. I have zero preference to XML or SGML
because I know them both. I do however care, when I have regionals and
contributors trying to give to the community that are having difficulty
doing so *because* we use SGML versus XML.

It is enough for me, that I have PostgreSQLFR, Chile and the printed
book project all clammoring for XML support to give more weight to their
arguments. Each of the three have presented their specific problems with
the SGML. Not to mention people such as Andrew Dunstan and the others.

Maybe the people demand isn't the concern. Okay, but switching to XML is
not hurting anyone. It's not like we are trying to change languages...
only the DTD :)

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

> The real problem here is that you've still failed to
> establish any sizable benefit from converting. As best I can
> tell at the moment, the acceptable options for editing XML
> will be about the same as they are for SGML: emacs, and not a
> lot else. I don't really see why I should have to start
> spelling out every closing tag for a no-op like that.

Visual Studio has a *very* nice XML editor these days, and they don't do
SGML...

(ok, ok, that's a joke. It's true, sure, but it's not very relevant...)

//Magnus

On Mon, 2006-12-11 at 21:56 +0100, Magnus Hagander wrote:
> > The real problem here is that you've still failed to
> > establish any sizable benefit from converting. As best I can
> > tell at the moment, the acceptable options for editing XML
> > will be about the same as they are for SGML: emacs, and not a
> > lot else. I don't really see why I should have to start
> > spelling out every closing tag for a no-op like that.
>
> Visual Studio has a *very* nice XML editor these days, and they don't do
> SGML...
>
> (ok, ok, that's a joke. It's true, sure, but it's not very relevant...)

Actually it might be for Windows developers wishing to develop Windows
specific sections. How does it work with Docbook?

Sincerely,

Joshua D. Drake

>
> //Magnus
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
>> Secondly the make postgres.xml would be a one time thing.
>
> Agreed, the cost of conversion is one-time ... but it's not small.
> Aside from getting the files themselves converted, there's the effort
> for people to find, install, and learn suitable tools, not to mention
> come up to speed on the differences between xml and sgml. (I assume
> there are some significant ones, else why are we having this discussion?)
>
> The real problem here is that you've still failed to establish any
> sizable benefit from converting. As best I can tell at the moment,
> the acceptable options for editing XML will be about the same as they
> are for SGML: emacs, and not a lot else. I don't really see why I
> should have to start spelling out every closing tag for a no-op like
> that.
>
> regards, tom lane

In regards to cost of conversion: The osx tool that the makefile
uses to generate postgres.xml has more options available to tweak
the output (not resolve external files into the main file, no
newlines in tags, etc.) After adding a few of those options +
htmltidy, the result is pretty close to the original SGML, barring a
bare newline between <para>'s for example.

The benefit I'm talking about is the ability to do multiple indexes
in one document, which is not possible with DSSSL. In order to do
this, I would have to run the conversion *every time* I wanted to
work on a new release of the manual to bookstores.

David Blewett
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfci5Zmlc6wNjtLYRCFYwAJ45TMs/YY0RoxHi+oMCB7shPGghlQCfa+oA
hDRctNczvnrFweSV4yI2lG0=
=2NoW
-----END PGP SIGNATURE-----

On Mon, 2006-12-11 at 13:53, Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> > Your fourth is resolved, we have the conversion per make postgres.xml.
>
> We do not have that --- have you looked at the output? It's nigh
> unreadable. Same problem as with the authoring tools: that software
> doesn't think that formatting of the XML source text is important.

Would this help:

http://xmlindent.sourceforge.net/

I use it all the time at work, where we have LOTS of tools that make big
one line xml files for us.

David Blewett <david(at)dawninglight(dot)net> writes:
> The benefit I'm talking about is the ability to do multiple indexes
> in one document, which is not possible with DSSSL. In order to do
> this, I would have to run the conversion *every time* I wanted to
> work on a new release of the manual to bookstores.

Yeah, but for purely downstream work like that, what's the objection to
just running the osx conversion? It seems to take just a few seconds.

I'm still not seeing where we get return on our investment for
converting the master source files from one format to the other.

regards, tom lane

> > > The real problem here is that you've still failed to
> establish any
> > > sizable benefit from converting. As best I can tell at
> the moment,
> > > the acceptable options for editing XML will be about the same as
> > > they are for SGML: emacs, and not a lot else. I don't really see
> > > why I should have to start spelling out every closing tag for a
> > > no-op like that.
> >
> > Visual Studio has a *very* nice XML editor these days, and
> they don't
> > do SGML...
> >
> > (ok, ok, that's a joke. It's true, sure, but it's not very
> > relevant...)
>
> Actually it might be for Windows developers wishing to
> develop Windows specific sections. How does it work with Docbook?

It works just fine with any XML inlcluding docbook. It does your basic
syntax highlighting, autoindenting and things like that. You can teach
it about things like docbook to get autocomplete on tags and attributes,
but I don't know if anybody has done that (I certainly haven't).

//Magnus

"Magnus Hagander" <mha(at)sollentuna(dot)net> writes:
>>> Visual Studio has a *very* nice XML editor these days, and
>>> they don't do SGML...
>>>
>>> (ok, ok, that's a joke. It's true, sure, but it's not very
>>> relevant...)
>>
>> Actually it might be for Windows developers wishing to
>> develop Windows specific sections. How does it work with Docbook?

> It works just fine with any XML inlcluding docbook. It does your basic
> syntax highlighting, autoindenting and things like that. You can teach
> it about things like docbook to get autocomplete on tags and attributes,
> but I don't know if anybody has done that (I certainly haven't).

How is it on the point of not introducing random reformatting into the
XML source text?

regards, tom lane

> >>> Visual Studio has a *very* nice XML editor these days, and they
> >>> don't do SGML...
> >>>
> >>> (ok, ok, that's a joke. It's true, sure, but it's not very
> >>> relevant...)
> >>
> >> Actually it might be for Windows developers wishing to develop
> >> Windows specific sections. How does it work with Docbook?
>
> > It works just fine with any XML inlcluding docbook. It does
> your basic
> > syntax highlighting, autoindenting and things like that.
> You can teach
> > it about things like docbook to get autocomplete on tags and
> > attributes, but I don't know if anybody has done that (I
> certainly haven't).
>
> How is it on the point of not introducing random reformatting
> into the XML source text?

it doesn't do it, period. Just to be clear, it's a XML *source* editor.
Just like emacs, excpet prettier ;-)

(and just for the record, these days even the GUI editors in Visual
Studio generally do a very good job of not breaking your manually tuned
markup, but there is no GUI editor for docbook)

//Magnus

David Blewett a écrit :
> Peter Eisentraut wrote:
>> Right. So using the XSL(T) stylesheet instead of the DSSSL stylesheet would
>> be a step in the right direction. Actually the 8.2 branch currently only has
>> an XSLT stylesheet for HTML output. In 8.3devel I've added one for XSL-FO
>> output, which you can invoke by "make postgres-{A4|US}.fo". (You will
>> probably want yet a different paper format for your book.)
>
> Excuse my ignorance, but I thought XSLT stylesheets can only be used
> on XML files?
>

That's right. To use Peter's work, you need to checkout CVS HEAD, cd
into doc/src/sgml, "make postgres.xml" which build a valid XML file.

Next step depends on the output format you want to get. To get HTML
output from XSLT file, execute "make xslthtml". This target use
stylesheet.xsl which customizes a bit the XSLT stylesheets for XHTML
available from docbook.sourceforge.net. To get .fo output (required to
build a PDF file), "make postgres-A4.fo". This command will use the
stylesheet-fo.xsl file. This file is just a pointer to XSLT stylesheets
for PDF available from docbook.sourceforge.net. Simple but effective.

A summary :
* to build HTML manual with XSLT :
make postgres.xml
make xslthtml
* to build PDF manual with XSLT :
make postgres.xml
make postgres-A4.fo
fop.sh postgres-A4.fo postgres-A4.pdf

I think my fop install is broken because I'm not able to build the PDF
file...

Peter, if I said something stupid, please, tell it. This is just what I
understood from your mail and the Makefiles/xslt files you provide.

Regards.

--
Guillaume.
<!-- http://abs.traduc.org/
http://lfs.traduc.org/
http://docs.postgresqlfr.org/ -->

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Tom Lane wrote:
> Yeah, but for purely downstream work like that, what's the objection to
> just running the osx conversion? It seems to take just a few seconds.
>
> I'm still not seeing where we get return on our investment for
> converting the master source files from one format to the other.
>
> regards, tom lane

Does the designation of "downstream work" apply here? I was
intending to contribute back any changes required, to enable anyone
else to build the multiple volume version for printing. I wanted it
to become an easily maintainable way for the project to have an
up-to-date version of our manual available to bookstores, or
directly available through lulu.com.

I also would like to reiterate the fact that the localization
efforts would appreciate not having to re-do their work for each
release. Do we really want to continue considering this addition to
the project as well as their work to be "second-class citizens" wrt
the rest of the distribution?

David Blewett
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFfgg/Zmlc6wNjtLYRCK9pAKCtN+UDuiYMprloV6Kx9XM4X3jCXQCgvhZ3
sRgVSsSAwUhHhQ2aMMC4ehA=
=XA3B
-----END PGP SIGNATURE-----

David Blewett <david(at)dawninglight(dot)net> writes:
> Does the designation of "downstream work" apply here? I was
> intending to contribute back any changes required, to enable anyone
> else to build the multiple volume version for printing.

Changes required where? AFAICS the useful content of your work would
probably be in the stylesheets, which there is already an XML version
of.

> I also would like to reiterate the fact that the localization
> efforts would appreciate not having to re-do their work for each
> release.

How exactly does working from an osx translation rather than hand-edited
sources handicap them? Or even more to the point, which part of their
work don't they have to redo anyway? We certainly have never hesitated
to whack the documentation around in small ways and large...

regards, tom lane

Tom Lane a ecrit le 12/12/2006 02:47:
> David Blewett <david(at)dawninglight(dot)net> writes:
> [...]
>> I also would like to reiterate the fact that the localization
>> efforts would appreciate not having to re-do their work for each
>> release.
>
> How exactly does working from an osx translation rather than hand-edited
> sources handicap them? Or even more to the point, which part of their
> work don't they have to redo anyway? We certainly have never hesitated
> to whack the documentation around in small ways and large...
>

I don't know for others but I think I can speak for the french team :)

I also don't quite understand the whole point on the translation. I use
Kompare to... hmmm... compare one release to another. I merge *manually*
the differences, and then I translate them. I also need to transform
SGML tags to XML tags (</> stuff and no ending tags). Actually, the
merge/translate step is much more difficult.

If some projects prefer to use po (with xml2pot or po4all), "make
postgres.xml" is the required step. The XML file built this way seems
valid to me.

Regards.

--
Guillaume.

Joshua D. Drake wrote:
> 5. The format objects development (FOP) seems to be better maintained
> then the dblatex/db2latex/pdflatex stuff.

Check the facts. The latest stable FOP release was more than three
years ago. The latest stable dblatex release was less than a month
ago.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Joshua D. Drake wrote:
> Of course your changes that you make from release to release for
> volumes won't be permanent

Well, of course he'd make his changes in the sources, not in the
intermediate XML file. Or he is talking about a kind of changes which
I cannot fathom at the moment.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

David Blewett wrote:
> Excuse my ignorance, but I thought XSLT stylesheets can only be used
> on XML files?

FWIW, you can use XSLT stylesheets on SGML as well, but we don't do
that.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Guillaume Lelarge wrote:
> A summary :
> * to build HTML manual with XSLT :
> make postgres.xml
> make xslthtml

Just the last command should be enough.

> * to build PDF manual with XSLT :
> make postgres.xml
> make postgres-A4.fo

Same here.

> fop.sh postgres-A4.fo postgres-A4.pdf
>
> I think my fop install is broken because I'm not able to build the
> PDF file...

I have been using FOP on and off for many years and I've never seen it
produce any real document longer than two pages, so I'm not surprised.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut a ecrit le 12/12/2006 09:10:
> Guillaume Lelarge wrote:
>> fop.sh postgres-A4.fo postgres-A4.pdf
>>
>> I think my fop install is broken because I'm not able to build the
>> PDF file...
>
> I have been using FOP on and off for many years and I've never seen it
> produce any real document longer than two pages, so I'm not surprised.
>

It produces PDF file for the PostgreSQL french manual. I really have no
problem to build the 1098 pages PDF file for 8.1.5 release.

--
Guillaume.

Josh Berkus wrote:
> Anyway, from the sound of it whether or not there are better
> authoring tools for XML than SGML is "undetermined pending further
> investigation."

I'll be the first to welcome better editing options, but so far I have
only heard them being theorized about.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Tue, 2006-12-12 at 08:59 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > 5. The format objects development (FOP) seems to be better maintained
> > then the dblatex/db2latex/pdflatex stuff.
>
> Check the facts. The latest stable FOP release was more than three
> years ago.

Actually it was about 8 months ago. 0.92 Beta.

> The latest stable dblatex release was less than a month
> ago.

I have found dblatex to not work very well (as in at all) but that could
very well be my installation.

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

On Tue, 2006-12-12 at 09:10 +0100, Peter Eisentraut wrote:
> Guillaume Lelarge wrote:
> > A summary :
> > * to build HTML manual with XSLT :
> > make postgres.xml
> > make xslthtml
>
> Just the last command should be enough.
>
> > * to build PDF manual with XSLT :
> > make postgres.xml
> > make postgres-A4.fo
>
> Same here.
>
> > fop.sh postgres-A4.fo postgres-A4.pdf
> >
> > I think my fop install is broken because I'm not able to build the
> > PDF file...
>
> I have been using FOP on and off for many years and I've never seen it
> produce any real document longer than two pages, so I'm not surprised.

I was able to easily (once I got the environment sorted out) generate
the entire french documentation with FOP.

Joshua D. Drake

>
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua D. Drake wrote:
> > Check the facts. The latest stable FOP release was more than three
> > years ago.
>
> Actually it was about 8 months ago. 0.92 Beta.

That is not a stable release. And the fact that the latest beta is 8
months old doesn't impress me either.
--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Tue, 2006-12-12 at 20:27 +0100, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > > Check the facts. The latest stable FOP release was more than three
> > > years ago.
> >
> > Actually it was about 8 months ago. 0.92 Beta.
>
> That is not a stable release. And the fact that the latest beta is 8
> months old doesn't impress me either.

I am not going to argue with you further. I have better things to do.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
> > As I said then, this is absolutely untrue. OpenOffice.org, for example,
> > works with DocBook XML but not SGML. There are also a plethora of XML
> > editing and publishing tools which can been used for Docbook XML which
> > are not available for SGML. A simple look at this page:
> > http://wiki.docbook.org/topic/DocBookAuthoringTools
> > .... shows that there are more than twice as many authoring tools which
> > support only XML as support SGML -- and that most of the tools which
> > support SGML are out-of-maintenance.
>
> This is confusing authoring tools (ie, stuff for more or less WYSIWYG
> editing of the document source) with output generation tools.
>
> As for authoring tools, show me one that produces SGML or XML that's
> reasonably readable, and I might worry about allowing people to use it.
> Most of the ones I've seen would render the doc sources unreadable for
> anyone not using an authoring tool (possibly even the very same
> authoring tool). We are not going to move in that direction
> because it would piss off the people who do the bulk of the work now.

And diffs would be either very large or useless.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

I have read the two long XML threads, and see these issues for
converting to XML:

o authoring, in two parts:

o WYSIWYG - this seems unattainable because such editors are
going to modify the surrounding XML, which will affect
hand-edited changes

o tag assistance - this seems possible, e.g. tag completion,
drop down list of tags available for this section

o output generation - the XML toolchain is more widely used, but Tom has
fixed the performance issue we have had for years. The fact that the
tool maintainers were unable to fix it themselves reflects badly on
their maintenance of this toolchain.

o translation - does XML have better tools to assist in translating our
manuals, and is this better than outputting XML via our makefile?

It would be nice to use htmltidy to clean up the xml before each
release, but I wonder whether it understand the tags well enough to
provide better formatting than we do now manually.

Tom, I think we need that SGML patch placed on our ftp site and a
reference to it made in our documentation. I can do it of you want.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce,

> o WYSIWYG - this seems unattainable because such editors are
> going to modify the surrounding XML, which will affect
> hand-edited changes

It would be more accurate to say that we have not identified a WYSWYG tool
which does not mess up the source. There may be one, it would just take a
fair amount of testing to find it.

> o tag assistance - this seems possible, e.g. tag completion,
> drop down list of tags available for this section

Yes, and graphically-enhanced tag support using XML IDEs. Command-line tag
completion is available via Emacs-SGML, but there are a lot more tools
which do it for XML (including Emacs).

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

On Thu, 2006-12-14 at 14:58, Josh Berkus wrote:
> Bruce,
>
> > o WYSIWYG - this seems unattainable because such editors are
> > going to modify the surrounding XML, which will affect
> > hand-edited changes
>
> It would be more accurate to say that we have not identified a WYSWYG tool
> which does not mess up the source. There may be one, it would just take a
> fair amount of testing to find it.

Is this strictly a question of indentation, or one of actually mangling
tags and such?

Scott Marlowe <smarlowe(at)g2switchworks(dot)com> writes:
> On Thu, 2006-12-14 at 14:58, Josh Berkus wrote:
>> It would be more accurate to say that we have not identified a WYSWYG tool
>> which does not mess up the source. There may be one, it would just take a
>> fair amount of testing to find it.

> Is this strictly a question of indentation, or one of actually mangling
> tags and such?

What we need is something that does not change regions of the file that
the user did not intend to modify. I think horizonal-white-space-only
changes could be worked around if the user is careful to use diff
--ignore-space-change when submitting the patch, but I suspect that an
editor that thinks it can mangle whitespace will also figure that it can
change line boundaries, and then diff will never be able to extract any
signal from that noise.

regards, tom lane

On Thu, 2006-12-14 at 15:34, Tom Lane wrote:
> Scott Marlowe <smarlowe(at)g2switchworks(dot)com> writes:
> > On Thu, 2006-12-14 at 14:58, Josh Berkus wrote:
> >> It would be more accurate to say that we have not identified a WYSWYG tool
> >> which does not mess up the source. There may be one, it would just take a
> >> fair amount of testing to find it.
>
> > Is this strictly a question of indentation, or one of actually mangling
> > tags and such?
>
> What we need is something that does not change regions of the file that
> the user did not intend to modify. I think horizonal-white-space-only
> changes could be worked around if the user is careful to use diff
> --ignore-space-change when submitting the patch, but I suspect that an
> editor that thinks it can mangle whitespace will also figure that it can
> change line boundaries, and then diff will never be able to extract any
> signal from that noise.

Ahhhh. I see.

Well, I don't mind teaching myself some docbook, but the first time I
looked at adding to the docs I was overwhelmed with what I didn't know,
and didn't even see a starting point. I'll look in the appendices and
see what I can see.

I have applied the attached documentation patch to HEAD and 8.2.X to
document the use of Tom's patch to openjade to get ouput in a reasonable
amount of time.

---------------------------------------------------------------------------

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Tom Lane wrote:
> >> Since jade does not go into this kind of spiral when producing html
> >> output from the same sources, I suggest that it's not jade's fault,
> >> but rather crummy coding in the sgml-to-tex conversion scripts it's
> >> using.
>
> > Right. I fixed that, so now it takes about 15 minutes to build the
> > whole thing.
>
> Actually, I just finished fixing what seems to be the underlying problem
> in jade: it's got a spectacularly bad implementation of linked lists.
> In PG-code terms, if the list is built by successive lappends(), then
> both lfirst() and lnext() take O(N) time for an N-element list, thus
> iterating through the list in the usual way is O(N^2) ... and not even
> with a reasonably small multiplier, because it stresses the hell out of
> memory allocation and garbage collection while it's at it. (Which may
> well mean that it's really more like O(N^3), since the garbage collector
> will iterate over every allocated object every so often.)
>
> With the patch it takes me about 5 minutes to do the jade step of the
> PDF build, using this morning's SGML sources. (I don't know how to set
> the TeX configuration to get the pdfjadetex steps to go through, so I
> dunno about total time.)
>
> However, I have no idea what it'll take to get this patch propagated
> into the copies people actually use, so your fix sounds good for the
> short term.
>
> regards, tom lane
>
> diff -cr openjade-1.3.2.orig/style/ELObj.cxx openjade-1.3.2/style/ELObj.cxx
> *** openjade-1.3.2.orig/style/ELObj.cxx Fri Jan 11 10:48:38 2002
> --- openjade-1.3.2/style/ELObj.cxx Sat Dec 9 21:58:36 2006
> ***************
> *** 1044,1049 ****
> --- 1044,1054 ----

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Thu, Dec 14, 2006 at 04:34:06PM -0500, Tom Lane wrote:
> Scott Marlowe <smarlowe(at)g2switchworks(dot)com> writes:
> > On Thu, 2006-12-14 at 14:58, Josh Berkus wrote:
> >> It would be more accurate to say that we have not identified a
> >> WYSWYG tool which does not mess up the source. There may be one,
> >> it would just take a fair amount of testing to find it.
>
> > Is this strictly a question of indentation, or one of actually
> > mangling tags and such?
>
> What we need is something that does not change regions of the file
> that the user did not intend to modify. I think
> horizonal-white-space-only changes could be worked around if the
> user is careful to use diff --ignore-space-change when submitting
> the patch, but I suspect that an editor that thinks it can mangle
> whitespace will also figure that it can change line boundaries, and
> then diff will never be able to extract any signal from that noise.

Could post-processing with tidy clean this up? I'm pretty sure that
tidy's output is deterministic, at least if the editor hasn't mangled
tags...

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

> > What we need is something that does not change regions of the file
> > that the user did not intend to modify. I think
> > horizonal-white-space-only changes could be worked around if the
> > user is careful to use diff --ignore-space-change when submitting
> > the patch, but I suspect that an editor that thinks it can mangle
> > whitespace will also figure that it can change line boundaries, and
> > then diff will never be able to extract any signal from that noise.
>
> Could post-processing with tidy clean this up? I'm pretty sure that
> tidy's output is deterministic, at least if the editor hasn't mangled
> tags...
>

Yes tidy makes things pretty darn clean. However, there is one thing
that it won't do and that is remove extra tags. Thus something like
OpenOffice is still a no-op.

Joshua D. Drake

> Cheers,
> D
--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Am Freitag, 15. Dezember 2006 03:45 schrieb Bruce Momjian:
> I have applied the attached documentation patch to HEAD and 8.2.X to
> document the use of Tom's patch to openjade to get ouput in a reasonable
> amount of time.

This is no longer accurate in 8.3. A workaround has been installed in the
stylesheets to arrive at the same result.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Am Freitag, 15. Dezember 2006 03:45 schrieb Bruce Momjian:
> > I have applied the attached documentation patch to HEAD and 8.2.X to
> > document the use of Tom's patch to openjade to get ouput in a reasonable
> > amount of time.
>
> This is no longer accurate in 8.3. A workaround has been installed in the
> stylesheets to arrive at the same result.

OK, I have talked to Peter via IM and now understand that his change to
stylesheet.dsl avoids the bug Tom found and fixed, so instead of
mentioning the patch in the SGML docs, I added a comment in the
stylesheet.dsl file and pointed to the URL.

Also, the double-run of the PDF does not fix the index. Only a proper
bookindex.sgml does, that, so I removed the double runs and either Peter
or I will work on issuing a warning when the bookindex.sgml file isn't
proper, so people know to run it again.

Revert patch attached and applied to HEAD and 8.2.X.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Also, the double-run of the PDF does not fix the index. Only a
> proper bookindex.sgml does, that, so I removed the double runs and
> either Peter or I will work on issuing a warning when the
> bookindex.sgml file isn't proper, so people know to run it again.

Producing a warning isn't hard, but the question is where to put it so
it doesn't get buried in all the other output.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > Also, the double-run of the PDF does not fix the index. Only a
> > proper bookindex.sgml does, that, so I removed the double runs and
> > either Peter or I will work on issuing a warning when the
> > bookindex.sgml file isn't proper, so people know to run it again.
>
> Producing a warning isn't hard, but the question is where to put it so
> it doesn't get buried in all the other output.

I think the Makefile rule just emits it at the end of its processing.

Also, for testing, I was thinking you could use 'wc -l bookindex.sgml',
and if the file length is less thatn 25 lines, it is a fake file.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Tom Lane wrote:
> However, I have no idea what it'll take to get this patch propagated
> into the copies people actually use, so your fix sounds good for the
> short term.
>
> regards, tom lane

I submitted the patch to gentoo's bugzilla:
http://bugs.gentoo.org/show_bug.cgi?id=158605

David Blewett

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFieRjZmlc6wNjtLYRCH2GAJ4rfMXrcPAceJoL5rTPjmpY9EAndACcCfes
oxBVK5jqghkxKVXEbwRD8m8=
=bR1k
-----END PGP SIGNATURE-----