Re: Proposal: new border setting in psql

Here is a simple select output.

darcy=# select title_id, title_name from title;
title_id | title_name
----------+------------
2 | Mrs
3 | Ms
4 | Miss
(3 rows)

Now I change the border.

darcy=# \pset border 2
Border style is 2.
darcy=# select title_id, title_name from title;
+----------+------------+
| title_id | title_name |
+----------+------------+
| 2 | Mrs |
| 3 | Ms |
| 4 | Miss |
+----------+------------+
(3 rows)

I would like to propose a new border setting.

darcy=# \pset border 3
Border style is 3.
darcy=# select title_id, title_name from title;
+----------+------------+
| title_id | title_name |
+==========+============+
| 2 | Mrs |
+----------+------------+
| 3 | Ms |
+----------+------------+
| 4 | Miss |
+----------+------------+
(3 rows)

The reason for this is to allow the output to be fed directly into any
system using Restructured text as described in
http://docutils.sourceforge.net/docs/user/rst/quickref.html.

If there is any interest I will submit patches for code and
documentation.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> I would like to propose a new border setting.

That code is horrendously overcomplicated and unreadable already :-(
I'm not too eager to add more variants to it.

> The reason for this is to allow the output to be fed directly into any
> system using Restructured text as described in
> http://docutils.sourceforge.net/docs/user/rst/quickref.html.

Is that *really* going to work? What about quoting/escaping
conventions?

Also, how many of those "any systems" actually exist? Markup
conventions are a dime a dozen.

On the whole I think it ought to be sufficient to support XML output
for people who want easily-machine-readable query output.

regards, tom lane

On Thu, 21 Aug 2008 15:03:23 -0400
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > I would like to propose a new border setting.
>
> That code is horrendously overcomplicated and unreadable already :-(
> I'm not too eager to add more variants to it.

Actually, I already made the code changes and they were surprisingly
simple.

> > The reason for this is to allow the output to be fed directly into any
> > system using Restructured text as described in
> > http://docutils.sourceforge.net/docs/user/rst/quickref.html.
>
> Is that *really* going to work? What about quoting/escaping
> conventions?

ReST is pretty good with that stuff.

> Also, how many of those "any systems" actually exist? Markup
> conventions are a dime a dozen.

That I can't say. My impression was that it was reasonably well
known. However, while ReST was *my* reason for proposing this it was
also important to me that the format stand by itself. I think it
does. It also follows the documentation in that it is an extension to
border 2 but with more borders, just like border 2 is more than border
1, etc. It's a consistent progression.

> On the whole I think it ought to be sufficient to support XML output
> for people who want easily-machine-readable query output.

Perhaps. The problem is that it still means running it through an
external program. That's fine for scripted processes but not for ad
hoc queries.

Perhaps what we really need is the ability for users to install their
own formatting functions. After all, we can define everything else.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

Proposed formats don't look easier to read for humans.
I doubt that they are more common or easier to process by machines than just
COPY query TO STDOUT CSV;

> The reason for this is to allow the output to be fed directly into any
> system using Restructured text

The idea would be to use psql as backend for some other system?
Or what do you mean by fed directly?

On Thu, Aug 21, 2008 at 10:54 PM, D'Arcy J.M. Cain <darcy(at)druid(dot)net> wrote:

> On Thu, 21 Aug 2008 15:03:23 -0400
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > > I would like to propose a new border setting.
> >
> > That code is horrendously overcomplicated and unreadable already :-(
> > I'm not too eager to add more variants to it.
>
> Actually, I already made the code changes and they were surprisingly
> simple.
>
> > > The reason for this is to allow the output to be fed directly into any
> > > system using Restructured text as described in
> > > http://docutils.sourceforge.net/docs/user/rst/quickref.html.
> >
> > Is that *really* going to work? What about quoting/escaping
> > conventions?
>
> ReST is pretty good with that stuff.
>
> > Also, how many of those "any systems" actually exist? Markup
> > conventions are a dime a dozen.
>
> That I can't say. My impression was that it was reasonably well
> known. However, while ReST was *my* reason for proposing this it was
> also important to me that the format stand by itself. I think it
> does. It also follows the documentation in that it is an extension to
> border 2 but with more borders, just like border 2 is more than border
> 1, etc. It's a consistent progression.
>
> > On the whole I think it ought to be sufficient to support XML output
> > for people who want easily-machine-readable query output.
>
> Perhaps. The problem is that it still means running it through an
> external program. That's fine for scripted processes but not for ad
> hoc queries.
>
> Perhaps what we really need is the ability for users to install their
> own formatting functions. After all, we can define everything else.
>
> --
> D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
> http://www.druid.net/darcy/ | and a sheep voting on
> +1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

On Thu, 21 Aug 2008 23:22:28 +0300
"Asko Oja" <ascoja(at)gmail(dot)com> wrote:
> The idea would be to use psql as backend for some other system?
> Or what do you mean by fed directly?

No, I meant that one would do any ad hoc query and cut and paste the
output directly into a tracking tool that supports ReST. As I
explained in another message though, this is a nice side effect for me
and is the reason that I have an interest in coding it but it really is
a logical extension of what we have anyway. Look how simple the
documentation change would be. If you left out the extra example it's
a one line difference.

Index: src/sgml/ref/psql-ref.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/psql-ref.sgml,v
retrieving revision 1.209
diff -u -p -u -r1.209 psql-ref.sgml
--- src/sgml/ref/psql-ref.sgml 3 Jul 2008 03:37:16 -0000 1.209
+++ src/sgml/ref/psql-ref.sgml 21 Aug 2008 20:31:24 -0000
@@ -1570,7 +1570,7 @@ lo_import 152801
<acronym>HTML</acronym> mode, this will translate directly
into the <literal>border=...</literal> attribute, in the
others only values 0 (no border), 1 (internal dividing lines),
- and 2 (table frame) make sense.
+ 2 (table frame) and 3 (individual cells) make sense.
</para>
</listitem>
</varlistentry>
@@ -2973,6 +2973,22 @@ peter(at)localhost testdb=&gt; <userinput>S
+-------+--------+
(4 rows)

+peter(at)localhost testdb=&gt; <userinput>\pset border 3</userinput>
+Border style is 3.
+peter(at)localhost testdb=&gt; <userinput>SELECT * FROM my_table;</userinput>
++-------+--------+
+| first | second |
++=======+========+
+| 1 | one |
++-------+--------+
+| 2 | two |
++-------+--------+
+| 3 | three |
++-------+--------+
+| 4 | four |
++-------+--------+
+(4 rows)
+
peter(at)localhost testdb=&gt; <userinput>\pset border 0</userinput>
Border style is 0.
peter(at)localhost testdb=&gt; <userinput>SELECT * FROM my_table;</userinput>

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> On Thu, 21 Aug 2008 23:22:28 +0300
> "Asko Oja" <ascoja(at)gmail(dot)com> wrote:
>> The idea would be to use psql as backend for some other system?
>> Or what do you mean by fed directly?

> No, I meant that one would do any ad hoc query and cut and paste the
> output directly into a tracking tool that supports ReST.

There's still the question of whether this covers any needs that aren't
met just as well by XML or CSV output formats.

regards, tom lane

On Thu, 21 Aug 2008 20:36:51 -0400
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > No, I meant that one would do any ad hoc query and cut and paste the
> > output directly into a tracking tool that supports ReST.
>
> There's still the question of whether this covers any needs that aren't
> met just as well by XML or CSV output formats.

Well, we could remove all the display formats except XML. After all,
it can always be converted to any other format. Of course we wouldn't
do that. User convenience is all I'm thinking of.

What do you think of the idea of allowing user defined display
functions? Of course, that's a much bigger job.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

Tom Lane escribió:
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > On Thu, 21 Aug 2008 23:22:28 +0300
> > "Asko Oja" <ascoja(at)gmail(dot)com> wrote:
> >> The idea would be to use psql as backend for some other system?
> >> Or what do you mean by fed directly?
>
> > No, I meant that one would do any ad hoc query and cut and paste the
> > output directly into a tracking tool that supports ReST.
>
> There's still the question of whether this covers any needs that aren't
> met just as well by XML or CSV output formats.

I think it does -- I used to use the Latex output format for easy cut'n
pasting. ReST sounds like it serves the same purpose. If I had had to
use conversion to XML, it would have been rather painful.

What I wonder is whether this should be a border setting or a format
setting.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Thu, 21 Aug 2008 21:19:58 -0400
Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> Tom Lane escribió:
> > There's still the question of whether this covers any needs that aren't
> > met just as well by XML or CSV output formats.
>
> I think it does -- I used to use the Latex output format for easy cut'n
> pasting. ReST sounds like it serves the same purpose. If I had had to
> use conversion to XML, it would have been rather painful.

ReST is nice because it's almost plain text. In fact, a ReST document
source can easily be read raw.

> What I wonder is whether this should be a border setting or a format
> setting.

I don't see it as a format for two reasons. First, it isn't really a
different format. It's just the same format as "border 2" with a few
extra lines. Second, and following from the first, it's such a logical
progression from "border 0" to the proposed "border 3" that that syntax
makes more sense. In fact, the guide is inches away from describing
this behaviour already.

Besides, making it a border option adds 12 lines to the code, 5 of
which are blank. I wouldn't want to think about the changes if it was
a different setting.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Thu, Aug 21, 2008 at 11:18:24PM -0400, D'Arcy J.M. Cain wrote:
> > I think it does -- I used to use the Latex output format for easy cut'n
> > pasting. ReST sounds like it serves the same purpose. If I had had to
> > use conversion to XML, it would have been rather painful.
>
> ReST is nice because it's almost plain text. In fact, a ReST document
> source can easily be read raw.

I presume by ReST you mean this:
http://docutils.sourceforge.net/rst.html

Just guessing though. It's not referenced in wikipedia under that
acronym.

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.

On Fri, 22 Aug 2008 08:23:01 +0200
Martijn van Oosterhout <kleptog(at)svana(dot)org> wrote:
> On Thu, Aug 21, 2008 at 11:18:24PM -0400, D'Arcy J.M. Cain wrote:
> > ReST is nice because it's almost plain text. In fact, a ReST document
> > source can easily be read raw.
>
> I presume by ReST you mean this:
> http://docutils.sourceforge.net/rst.html

Yes. See the original message in this thread.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Thu, 21 Aug 2008 21:04:07 -0400
"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> wrote:
> > There's still the question of whether this covers any needs that aren't
> > met just as well by XML or CSV output formats.
>
> Well, we could remove all the display formats except XML. After all,
> it can always be converted to any other format. Of course we wouldn't
> do that. User convenience is all I'm thinking of.

Well, Tom has raised a question about its need and Asko has questioned
whether it should be under a different setting but so far no one has
outright rejected the proposal. Does anyone else have an opinion? I am
attaching a patch for further review.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> On Thu, 21 Aug 2008 21:04:07 -0400
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> wrote:
>
>>> There's still the question of whether this covers any needs that aren't
>>> met just as well by XML or CSV output formats.
>>>
>> Well, we could remove all the display formats except XML. After all,
>> it can always be converted to any other format. Of course we wouldn't
>> do that. User convenience is all I'm thinking of.
>>
>
> Well, Tom has raised a question about its need and Asko has questioned
> whether it should be under a different setting but so far no one has
> outright rejected the proposal. Does anyone else have an opinion? I am
> attaching a patch for further review.
>
>

In general I think I prefer machine readable formats to be produces by
the backend so they are available through all clients, not just psql.

That said, this has sufficiently low impact that I'm not going to be
vastly upset if we let it through.

I think we should probably confine ourselves to output formats that are
in very wide use or we'll be supporting a vast multitude. CSV and XML
both qualify here - not sure that ReST does.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> I think we should probably confine ourselves to output formats that are
> in very wide use or we'll be supporting a vast multitude. CSV and XML
> both qualify here - not sure that ReST does.

Yeah, that's the core of my objection.

Also, having now looked at the proposed patch, it seems clear that it
isn't addressing the issue of quoting/escaping at all; so I wonder how
this can be considered to be a safely machine-readable format.
In particular, the output seems to me to not even approximate the rules
laid down at
http://docutils.sourceforge.net/docs/user/rst/quickref.html
which among other things requires backslashing of literal asterisk,
backquote, vertical bar, and underscore in order to avoid textual data
looking like it matches the format's inline-markup constructs.

So, quite aside from the question of whether we care to support ReST,
my opinion is that this patch fails to do so, and a significantly more
invasive patch would be needed to do it.

regards, tom lane

On Sat, 23 Aug 2008 14:42:57 -0400
Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> In general I think I prefer machine readable formats to be produces by
> the backend so they are available through all clients, not just psql.

What do you mean by "machine readable?" XML?

> That said, this has sufficiently low impact that I'm not going to be
> vastly upset if we let it through.
>
> I think we should probably confine ourselves to output formats that are
> in very wide use or we'll be supporting a vast multitude. CSV and XML
> both qualify here - not sure that ReST does.

It isn't ReST. It just happens that this very simple and logical
progression of the border setting works for ReST. The display stands
on its own. This is not true of machine readable formats like XML. I
guess the question is, why border 2? Border 1 seems to be enough. In
fact border 0 has everything we absolutely need. Why have any borders?

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Sat, 23 Aug 2008 14:57:50 -0400
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Also, having now looked at the proposed patch, it seems clear that it
> isn't addressing the issue of quoting/escaping at all; so I wonder how
> this can be considered to be a safely machine-readable format.

It's not a machine readable format. It is a simple display with more
border lines. Just like "border 2" is like "border 1" with more border
lines. I'm just following the progression.

> In particular, the output seems to me to not even approximate the rules
> laid down at
> http://docutils.sourceforge.net/docs/user/rst/quickref.html

And there is no reason that it should.

> So, quite aside from the question of whether we care to support ReST,
> my opinion is that this patch fails to do so, and a significantly more
> invasive patch would be needed to do it.

I suppose it is my fault for mentioning ReST. That was the reason I
looked into this but that is not what the final proposal is. I too
would argue against making a munged output just to match one formatting
scheme. If I do a query and I need to modify it manually when I use it
in *any* third party program that's my personal issue. If "border 3"
happens to get me closer to my format that's great but it has to stand
on its own merit. I think that this proopsal does.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Sat, 2008-08-23 at 14:42 -0400, Andrew Dunstan wrote:
>
> D'Arcy J.M. Cain wrote:
> > On Thu, 21 Aug 2008 21:04:07 -0400
> > "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> wrote:
> >
> >>> There's still the question of whether this covers any needs that aren't
> >>> met just as well by XML or CSV output formats.
> >>>
> >> Well, we could remove all the display formats except XML. After all,
> >> it can always be converted to any other format. Of course we wouldn't
> >> do that. User convenience is all I'm thinking of.
> >>
> >
> > Well, Tom has raised a question about its need and Asko has questioned
> > whether it should be under a different setting but so far no one has
> > outright rejected the proposal. Does anyone else have an opinion? I am
> > attaching a patch for further review.
> >
> >
>
>
> In general I think I prefer machine readable formats to be produces by
> the backend so they are available through all clients, not just psql.

ReST is both human-readable format and machine readable format.

Where should this come from ?

> That said, this has sufficiently low impact that I'm not going to be
> vastly upset if we let it through.
>
> I think we should probably confine ourselves to output formats that are
> in very wide use or we'll be supporting a vast multitude. CSV and XML
> both qualify here - not sure that ReST does.

ReST is just one variant of TEXT - also a format which is in very wide
use :)

I mean, XML is just a meta-format, like TEXT, unless we start to
formalize our XML, provide DTD-s, etc.

----------------
Hannu

On Sun, Aug 24, 2008 at 2:00 AM, D'Arcy J.M. Cain <darcy(at)druid(dot)net> wrote:
> On Sat, 23 Aug 2008 14:57:50 -0400
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Also, having now looked at the proposed patch, it seems clear that it
>> isn't addressing the issue of quoting/escaping at all; so I wonder how
>> this can be considered to be a safely machine-readable format.
>
> It's not a machine readable format. It is a simple display with more
> border lines. Just like "border 2" is like "border 1" with more border
> lines. I'm just following the progression.

Personally I think it's rather nice to be able to have some extra
flexibility in how psql prints out data. Maybe, instead of the dry
and uninformative 'border 2', there could be a set of ouput control
options. Maybe I want the text aligned but with no border for
example.

merlin

On Sun, 24 Aug 2008 09:16:43 -0400
"Merlin Moncure" <mmoncure(at)gmail(dot)com> wrote:
> Personally I think it's rather nice to be able to have some extra
> flexibility in how psql prints out data. Maybe, instead of the dry
> and uninformative 'border 2', there could be a set of ouput control
> options. Maybe I want the text aligned but with no border for
> example.

You mean like "\pset border 0" does?

Personally I would love to see user defined display outputs for full
flexibility. Since we already have XML I would suggest using that as a
base and allow filters to process it before output.

That's a much larger job though.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> On Sat, 23 Aug 2008 14:57:50 -0400
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> So, quite aside from the question of whether we care to support ReST,
>> my opinion is that this patch fails to do so, and a significantly more
>> invasive patch would be needed to do it.

> I suppose it is my fault for mentioning ReST. That was the reason I
> looked into this but that is not what the final proposal is.

Well, if you can't just paste your output into ReST without having to
hand-munge it afterwards, then ISTM the argument for having this
additional bit of complexity in our printing routines really falls flat.

regards, tom lane

On Sun, 24 Aug 2008 13:22:38 -0400
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > I suppose it is my fault for mentioning ReST. That was the reason I
> > looked into this but that is not what the final proposal is.
>
> Well, if you can't just paste your output into ReST without having to
> hand-munge it afterwards, then ISTM the argument for having this
> additional bit of complexity in our printing routines really falls flat.

But Tom, you are still treating this as a ReST option. Please, pretend
that I never mentioned ReST. Consider this simply as a proposal to
make a logical extension to the "border [0|1|2]" setting. If you were
going to extend border to 3, what would you do? Adding extra row
dividers and turning dashes into equal signs for the existing row
divider seems pretty logical on its own without referencing any
external formats.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Aug 24, 2008, at 6:16 AM, Merlin Moncure wrote:

> On Sun, Aug 24, 2008 at 2:00 AM, D'Arcy J.M. Cain <darcy(at)druid(dot)net>
> wrote:
>> On Sat, 23 Aug 2008 14:57:50 -0400
>> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Also, having now looked at the proposed patch, it seems clear that
>>> it
>>> isn't addressing the issue of quoting/escaping at all; so I wonder
>>> how
>>> this can be considered to be a safely machine-readable format.
>>
>> It's not a machine readable format. It is a simple display with more
>> border lines. Just like "border 2" is like "border 1" with more
>> border
>> lines. I'm just following the progression.
>
>
> Personally I think it's rather nice to be able to have some extra
> flexibility in how psql prints out data. Maybe, instead of the dry
> and uninformative 'border 2', there could be a set of ouput control
> options. Maybe I want the text aligned but with no border for
> example.

Or maybe you want an external filter that eats csv or xml and excretes
what you want to see on the screen.

Cheers,
Steve

Steve Atkins <steve(at)blighty(dot)com> writes:
> On Aug 24, 2008, at 6:16 AM, Merlin Moncure wrote:
>> Personally I think it's rather nice to be able to have some extra
>> flexibility in how psql prints out data. Maybe, instead of the dry
>> and uninformative 'border 2', there could be a set of ouput control
>> options. Maybe I want the text aligned but with no border for
>> example.

> Or maybe you want an external filter that eats csv or xml and excretes
> what you want to see on the screen.

FWIW, I'd be for a redesign that allows more flexible control over
psql's output format(s). The patch at hand doesn't seem to offer
much advantage though ...

regards, tom lane

On Sat, 23 Aug 2008, Andrew Dunstan wrote:

> I think we should probably confine ourselves to output formats that are in
> very wide use or we'll be supporting a vast multitude. CSV and XML both
> qualify here - not sure that ReST does.

ReST is accepted by Trac, one of the more popular SCM+Project Management
tools available. And the Python Docutils package is pretty popular too,
with a growing infrastructure of packages springing up around it. For
example, it's trivial to take the ReST document, feed it through Sphinx
(the tool used to generate the Python documentation nowadays:
http://sphinx.pocoo.org/ ) , and turn the result into HTML, LaTeX, or PDF
files.

IMHO, while there are a ton of them out there (
http://en.wikipedia.org/wiki/Lightweight_markup_language ), there are only
two text markup languages besides HTML and XML-based ones that really
matter nowadays: ReST and the Mediawiki format.

I just checked D'Arcy's original table and it imports perfect into the
Trac system I use every day as valid rst code. Myself and at least
another half dozen people I work with would love to see that supported as
an output format--if the change isn't too obtrusive to the code base, of
course, which it doesn't sound like it is.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> On Sat, 23 Aug 2008, Andrew Dunstan wrote:
>> I think we should probably confine ourselves to output formats that are in
>> very wide use or we'll be supporting a vast multitude. CSV and XML both
>> qualify here - not sure that ReST does.

> ReST is accepted by Trac, one of the more popular SCM+Project Management
> tools available. And the Python Docutils package is pretty popular too,
> with a growing infrastructure of packages springing up around it. For
> example, it's trivial to take the ReST document, feed it through Sphinx
> (the tool used to generate the Python documentation nowadays:
> http://sphinx.pocoo.org/ ) , and turn the result into HTML, LaTeX, or PDF
> files.

You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
It outputs something that might pass for ReST, but only so long as there
are no special characters in the data.

regards, tom lane

On Fri, 29 Aug 2008, Tom Lane wrote:

> You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
> It outputs something that might pass for ReST, but only so long as there
> are no special characters in the data.

I agree that it's a bad idea to say explicitly that it's "ReST mode"
output, because it's not. But it works just fine for that purpose on
almost every table I would generate on a typical day. My databases are
mainly filled with plain alphanumeric text and numbers. If I can dump 99%
of them into ReST using this new border but 1% require me to manually
tweak by escaping some characters, that's still very useful to me. I'd
hate to see a focus on the corner cases drive this feature away.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> On Fri, 29 Aug 2008, Tom Lane wrote:
>> You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
>> It outputs something that might pass for ReST, but only so long as there
>> are no special characters in the data.

> I'd hate to see a focus on the corner cases drive this feature away.

Hmm ... the patch works for data that contains no backslashes,
asterisks, backquotes, vertical bars, nor underscores. Nor perhaps
other special characters that I might've missed in one cursory scan of
the ReST spec. I'm not sure which side of this should be considered a
"corner case"; but I am quite certain that anyone trying to pass data
into a ReST-reading application will soon be dissatisfied with this
patch.

regards, tom lane

On Fri, 2008-08-29 at 01:29 -0400, Tom Lane wrote:
> Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> > On Fri, 29 Aug 2008, Tom Lane wrote:
> >> You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
> >> It outputs something that might pass for ReST, but only so long as there
> >> are no special characters in the data.
>
> > I'd hate to see a focus on the corner cases drive this feature away.
>
> Hmm ... the patch works for data that contains no backslashes,
> asterisks, backquotes, vertical bars, nor underscores. Nor perhaps
> other special characters that I might've missed in one cursory scan of
> the ReST spec. I'm not sure which side of this should be considered a
> "corner case"; but I am quite certain that anyone trying to pass data
> into a ReST-reading application will soon be dissatisfied with this
> patch.

They are much more dissatisfied now, without this patch.

Escaping an occasional special character is something you have to do
anyway when writing markup. But adding rows of vertical bars and
aligning them using spaces is a much much more tedious task you really
would like to avoid.

I think the patch provides just the right gain for effort. Trying to
make it take care of all possible cases defined in ReST spec and still
output something that is esy to read as plain text would really be waste
of time.

---------------
Hannu

On Fri, 29 Aug 2008, Tom Lane wrote:

> the patch works for data that contains no backslashes, asterisks,
> backquotes, vertical bars, nor underscores.

These characters don't show up very much in real world data. You'll find
[-.,:;!(at)#$%&()"] in just about everything; those would be a problem.
Occasionally I'll see data files that use * in them, that's the main
source for the ones I was suggesting I'd have to tweak with a
search/replace (* -> \*) before the output would be useful. But [\`|_]?
Unless you're storing source code or similar computer-ish stuff in your
database, you just don't see those characters. The kind of regular text a
normal business dumps into a database just doesn't use them.

> I am quite certain that anyone trying to pass data into a ReST-reading
> application will soon be dissatisfied with this patch.

If anyone can suggest a common data source that a) makes sense to output
in tabular form and b) uses one of the special characters that's a problem
here, I'd grudgingly concede here. A quick search of the stuff I work on
finds filenames like "C:\WINDOWS" as the only thing I run into with any
frequency close to that category, and it's rare I'd be dumping something
with a filename into tabular output.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Le Friday 29 August 2008, Greg Smith a écrit :
> On Fri, 29 Aug 2008, Tom Lane wrote:
> > You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
> > It outputs something that might pass for ReST, but only so long as there
> > are no special characters in the data.
>
> I agree that it's a bad idea to say explicitly that it's "ReST mode"
> output, because it's not. But it works just fine for that purpose on
> almost every table I would generate on a typical day. My databases are
> mainly filled with plain alphanumeric text and numbers. If I can dump 99%
> of them into ReST using this new border but 1% require me to manually
> tweak by escaping some characters, that's still very useful to me. I'd
> hate to see a focus on the corner cases drive this feature away.

We use ReST a lot, and it will be very usefull to have this ouput.

>
> --
> * Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD
--
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org

Le Saturday 23 August 2008, D'Arcy J.M. Cain a écrit :
> On Thu, 21 Aug 2008 21:04:07 -0400
>
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> wrote:
> > > There's still the question of whether this covers any needs that aren't
> > > met just as well by XML or CSV output formats.
> >
> > Well, we could remove all the display formats except XML. After all,
> > it can always be converted to any other format. Of course we wouldn't
> > do that. User convenience is all I'm thinking of.
>
> Well, Tom has raised a question about its need and Asko has questioned
> whether it should be under a different setting but so far no one has
> outright rejected the proposal. Does anyone else have an opinion? I am
> attaching a patch for further review.

This patch is just good, not mentionned anywhere that it is *ReSt output*.
If needed, a 'case PRINT_REST' in print.c is still possible... or let user
define output format as suggested by D'arcy J.M. Cain.

--
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org

As stated above this format is mainly good for copy paste and may require
occasional manual tweaking.
Users should be people who use psql in their everyday work and on the other
hand need to publish data from database in some other places. Would you
please bring examples of some widespread applications that would be happy to
digest these formats?

regards,
Asko

PS: For me current capabilities of psql (\a and \f) have been quite enough
so far when pasting occasionally to wiki, openoffice or chat message.

On Fri, Aug 29, 2008 at 11:00 AM, Cédric Villemain <
cedric(dot)villemain(at)dalibo(dot)com> wrote:

> Le Friday 29 August 2008, Greg Smith a écrit :
> > On Fri, 29 Aug 2008, Tom Lane wrote:
> > > You're ignoring the fact that D'Arcy's patch doesn't output valid ReST.
> > > It outputs something that might pass for ReST, but only so long as
> there
> > > are no special characters in the data.
> >
> > I agree that it's a bad idea to say explicitly that it's "ReST mode"
> > output, because it's not. But it works just fine for that purpose on
> > almost every table I would generate on a typical day. My databases are
> > mainly filled with plain alphanumeric text and numbers. If I can dump
> 99%
> > of them into ReST using this new border but 1% require me to manually
> > tweak by escaping some characters, that's still very useful to me. I'd
> > hate to see a focus on the corner cases drive this feature away.
>
> We use ReST a lot, and it will be very usefull to have this ouput.
>
> >
> > --
> > * Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD
> --
> Cédric Villemain
> Administrateur de Base de Données
> Cel: +33 (0)6 74 15 56 53
> http://dalibo.com - http://dalibo.org
>

On Fri, 29 Aug 2008 01:29:14 -0400
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Hmm ... the patch works for data that contains no backslashes,
> asterisks, backquotes, vertical bars, nor underscores. Nor perhaps
> other special characters that I might've missed in one cursory scan of
> the ReST spec. I'm not sure which side of this should be considered a
> "corner case"; but I am quite certain that anyone trying to pass data
> into a ReST-reading application will soon be dissatisfied with this
> patch.

I think that your scan may have been a bit too cursory. Those
characters, while significant in ReST, only matter when used in very
specific ways. The following works just fine in my ReST application.

+----+-------+
| id | name |
+====+=======+
| 8 | T'est |
+----+-------+
| 9 | T*est |
+----+-------+
| 10 | T\est |
+----+-------+
| 11 | T`est |
+----+-------+
| 12 | T_est |
+----+-------+

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Fri, 29 Aug 2008 06:55:45 -0400
"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> wrote:
> On Fri, 29 Aug 2008 01:29:14 -0400
> I think that your scan may have been a bit too cursory. Those
> characters, while significant in ReST, only matter when used in very
> specific ways. The following works just fine in my ReST application.
>
> +----+-------+
> | id | name |
> +====+=======+
> | 8 | T'est |
> +----+-------+
> | 9 | T*est |
> +----+-------+
> | 10 | T\est |
> +----+-------+

Oops. I was wrong about this one. The backslash needs to be escaped.
It also means expanding the other rows to match so this is the corner
case. In fact, for one or two backslashes you can double it and remove
the space at the start and/or end which is not so bad.

I'm surprised that we don't have a general option to escape special
characters. Perhaps that's the next small enhancement.

darcy=# \pset escape \

For example. The general output filter I suggested previously could
also deal with that, of course.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Fri, Aug 29, 2008 at 8:45 AM, D'Arcy J.M. Cain <darcy(at)druid(dot)net> wrote:
>
> I'm surprised that we don't have a general option to escape special
> characters. Perhaps that's the next small enhancement.
>
> darcy=# \pset escape \
>

this looks like we are trying to make cosmetic magic instead of solve
the real problem...
what about the idea someone propose of having output formats hooks...
seems more reasonable to me

--
regards,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. (593) 87171157

On Fri, 29 Aug 2008 11:23:50 -0500
"Jaime Casanova" <jcasanov(at)systemguards(dot)com(dot)ec> wrote:

> On Fri, Aug 29, 2008 at 8:45 AM, D'Arcy J.M. Cain <darcy(at)druid(dot)net>
> wrote:
> >
> > I'm surprised that we don't have a general option to escape special
> > characters. Perhaps that's the next small enhancement.
> >
> > darcy=# \pset escape \
> >
>
> this looks like we are trying to make cosmetic magic instead of solve
> the real problem...
> what about the idea someone propose of having output formats hooks...
> seems more reasonable to me
>

I am trying to understand why we are having a client do this? If you
want some other type of output, script it.

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
United States PostgreSQL Association: http://www.postgresql.us/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Joshua Drake escribió:

> I am trying to understand why we are having a client do this? If you
> want some other type of output, script it.

Convenience. If we had real rst output, we could just copy'n paste into
Trac or other systems.

Convenience is why we have psql at all. Otherwise, surely everybody
could script the DDL commands that we now use in \d. It's simply a
matter of echo and netcat, after all.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:

> Joshua Drake escribió:
>
>> I am trying to understand why we are having a client do this? If you
>> want some other type of output, script it.
>
> Convenience. If we had real rst output, we could just copy'n paste into
> Trac or other systems.

I'm starting to think D'Arcy's on the right track here.

Keep in mind the use case here is as Alvaro says, just a user convenience
thing. It's not meant for file dumps and loads. If we're going to display an
ascii table we may as well use the same formatting as other tools so it can be
copy/pasted in.

Given that it's just a user convenience thing then I'm not sure the escaping
is necessarily a big deal. If the user happens to have any backslashes in
their data they can always stick a replace() call in their SQL. Perhaps we
should prove a rest_escape() function for that purpose.

I wonder if it's worth keeping two variants at all really. Why not just make
psql's native table formatting exactly ReST? Is there any part of it that we
don't like as much as our existing tables?

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!

On Fri, 29 Aug 2008 18:07:36 +0100
Gregory Stark <stark(at)enterprisedb(dot)com> wrote:
> I'm starting to think D'Arcy's on the right track here.

Is that the train coming? :-)

> Keep in mind the use case here is as Alvaro says, just a user convenience
> thing. It's not meant for file dumps and loads. If we're going to display an
> ascii table we may as well use the same formatting as other tools so it can be
> copy/pasted in.

Well, Tom makes a good point about trying to make it fit one specific
markup language perfectly. The important thing here is that it stand
on its own as a nice display.

> Given that it's just a user convenience thing then I'm not sure the escaping
> is necessarily a big deal. If the user happens to have any backslashes in
> their data they can always stick a replace() call in their SQL. Perhaps we
> should prove a rest_escape() function for that purpose.

I think that a setting is just a lot cleaner. Remember, the use case
here is that someone wants to do an ad-hoc query and drop it into some
other tool. A simple "SELECT * FROM table" should work.

> I wonder if it's worth keeping two variants at all really. Why not just make
> psql's native table formatting exactly ReST? Is there any part of it that we
> don't like as much as our existing tables?

No, Tom is right. This should not be a ReST format. For one thing,
the user may just want the extra lines and any escaping/formatting
would get in their way.

And what do you mean by "native?" Border 0? Border 1? Border 2? I
think that "principle of least surprise" demands that these not change
on a user.
--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

Gregory Stark wrote:
> I wonder if it's worth keeping two variants at all really. Why not just make
> psql's native table formatting exactly ReST? Is there any part of it that we
> don't like as much as our existing tables?

It doubles the number of display lines; a very obvious shortcoming.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

So what have we decided about this suggestion. Should I submit the
patch or just forget about it? So far some people like it and some
people think that it is unneccessary. No one so far has suggested that
it would harm the system or people's use of it.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> So what have we decided about this suggestion. Should I submit the
> patch or just forget about it? So far some people like it and some
> people think that it is unneccessary. No one so far has suggested that
> it would harm the system or people's use of it.

Has there been any consensus on this?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

D'Arcy J.M. Cain wrote:
> So what have we decided about this suggestion. Should I submit the
> patch or just forget about it? So far some people like it and some
> people think that it is unneccessary. No one so far has suggested that
> it would harm the system or people's use of it.

I have gone over the discussion about this issue. I think there is
interest in a ReST output format, but only a 100% ReST-compliant one. I
don't think anyone felt they wanted a ReST-like format just for
appearance sake. For that reason, I have added this TODO entry:

Support the ReST table output format

Details about the ReST format: http://docutils.sourceforge.net/docs/user/rst/quickref.html

* http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php

As I remember, no actual patch was posted for this.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Wed, 7 Jan 2009 17:22:38 -0500 (EST)
Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> D'Arcy J.M. Cain wrote:
> > So what have we decided about this suggestion. Should I submit the
> > patch or just forget about it? So far some people like it and some
> > people think that it is unneccessary. No one so far has suggested that
> > it would harm the system or people's use of it.
>
> I have gone over the discussion about this issue. I think there is
> interest in a ReST output format, but only a 100% ReST-compliant one. I
> don't think anyone felt they wanted a ReST-like format just for
> appearance sake. For that reason, I have added this TODO entry:

Really? I thought that the opposite was true, that the argument
against this change was that it was trying to be ReST. That's why I
made a few posts arguing that while it mostly worked ReST, it was
really just a logical extension of the existing border control.

> As I remember, no actual patch was posted for this.

There was. I am attaching it again in case there were any changes to
original files in the meantime.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> As I remember, no actual patch was posted for this.

> There was. I am attaching it again in case there were any changes to
> original files in the meantime.

I think what Bruce meant to say is that this patch doesn't produce 100%
spec-compliant ReST, and that almost-ReST doesn't seem like a good
feature.

regards, tom lane

D'Arcy J.M. Cain wrote:
> On Wed, 7 Jan 2009 17:22:38 -0500 (EST)
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > D'Arcy J.M. Cain wrote:
> > > So what have we decided about this suggestion. Should I submit the
> > > patch or just forget about it? So far some people like it and some
> > > people think that it is unneccessary. No one so far has suggested that
> > > it would harm the system or people's use of it.
> >
> > I have gone over the discussion about this issue. I think there is
> > interest in a ReST output format, but only a 100% ReST-compliant one. I
> > don't think anyone felt they wanted a ReST-like format just for
> > appearance sake. For that reason, I have added this TODO entry:
>
> Really? I thought that the opposite was true, that the argument
> against this change was that it was trying to be ReST. That's why I
> made a few posts arguing that while it mostly worked ReST, it was
> really just a logical extension of the existing border control.

Well, the discussion kind of went around and around. What I saw was
people wanting ReST, but not wanting the patch to be rejected because it
didn't do 100% ReST, saying they can clean up the output, or don't use
backslashes. Being API-change-phobic, the idea of implementing
something like ReST then adding full ReST later seems bad. Now, I think
it could be implemented with a switch to turn off the ReST escaping, but
in general ReST was the attactiveness of the patch.

> > As I remember, no actual patch was posted for this.
>
> There was. I am attaching it again in case there were any changes to
> original files in the meantime.

I knew I had seen it but could not find it in the archives; I will link
to this version on the TODO list.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Wed, 07 Jan 2009 18:12:58 -0500
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >> As I remember, no actual patch was posted for this.
>
> > There was. I am attaching it again in case there were any changes
> > to original files in the meantime.
>
> I think what Bruce meant to say is that this patch doesn't produce
> 100% spec-compliant ReST, and that almost-ReST doesn't seem like a
> good feature.

It is a great feature for people actually using ReST. However, the
feature is really just a logical extension to the existing border
attribute. The fact that it is useful to people using ReST is just a
happy coincidence. I don't think it should be referred to as the ReST
feature. My patch does not mention ReST in the doc changes.

By "logical extension" I mean this;

0: Underline column names with single line
1: Add lines between columns
2: Add table frame <<== Up to here already exists
3: Underline column names with double line and add line between rows

It might be argued that '3' should be split between '3' and '4' since it
is two changes but that's a minor matter. I can make that change to the
patch if anyone sees any value to that.

3: Add line between rows
4: Use double line between header line and data

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> On Wed, 07 Jan 2009 18:12:58 -0500
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > "D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> > > Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > >> As I remember, no actual patch was posted for this.
> >
> > > There was. I am attaching it again in case there were any changes
> > > to original files in the meantime.
> >
> > I think what Bruce meant to say is that this patch doesn't produce
> > 100% spec-compliant ReST, and that almost-ReST doesn't seem like a
> > good feature.
>
> It is a great feature for people actually using ReST. However, the
> feature is really just a logical extension to the existing border
> attribute. The fact that it is useful to people using ReST is just a
> happy coincidence. I don't think it should be referred to as the ReST
> feature. My patch does not mention ReST in the doc changes.
>
> By "logical extension" I mean this;
>
> 0: Underline column names with single line
> 1: Add lines between columns
> 2: Add table frame <<== Up to here already exists
> 3: Underline column names with double line and add line between rows
>
> It might be argued that '3' should be split between '3' and '4' since it
> is two changes but that's a minor matter. I can make that change to the
> patch if anyone sees any value to that.
>
> 3: Add line between rows
> 4: Use double line between header line and data

Well, did anyone say they actually liked the new format,
appearance-wise, or would use it independently of ReST --- I don't
remember anyone, and we don't want to extend the output format unless
there is user demand.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

D'Arcy J.M. Cain wrote:
> On Wed, 07 Jan 2009 18:12:58 -0500
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> > I think what Bruce meant to say is that this patch doesn't produce
> > 100% spec-compliant ReST, and that almost-ReST doesn't seem like a
> > good feature.
>
> It is a great feature for people actually using ReST. However, the
> feature is really just a logical extension to the existing border
> attribute.

Frankly I don't understand your position. You seem to be saying that
you want the logical extension to the border feature, because it's very
easy to write, but you don't want to go to all the trouble of writing an
actual rst output format -- I guess it's a lot more code. You don't
care that your new border format is not actually rst, because you have
no need for rst.

Can I ask what is this logical extension of the border feature useful
for, keeping in mind that rst is not it?

Some people suggests that this is so close to rst that I should just use
it as if it were, and hand-edit the output for the rare cases where it
doesn't comply. I don't find this very compelling.

Apparently the bottom line is that if it's not actual rst, it will get
rejected.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Well, did anyone say they actually liked the new format,
> appearance-wise, or would use it independently of ReST --- I don't
> remember anyone, and we don't want to extend the output format unless
> there is user demand.

Yeah. If it's not really ReST-compliant then the only argument
for supporting it is that somebody likes it better aesthetically.
I'd want to see more than just one or two people voting for it
if that's the only argument.

If you go the extra mile and make it output 100% valid ReST then
there is some functionality gain not only aesthetics, and then
you'll have a lot easier sell.

regards, tom lane

On Thu, 8 Jan 2009 12:30:52 -0300
Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> > It is a great feature for people actually using ReST. However, the
> > feature is really just a logical extension to the existing border
> > attribute.
>
> Frankly I don't understand your position. You seem to be saying that
> you want the logical extension to the border feature, because it's very
> easy to write, but you don't want to go to all the trouble of writing an
> actual rst output format -- I guess it's a lot more code. You don't
> care that your new border format is not actually rst, because you have
> no need for rst.

In fact I wrote it because I do want it for ReST. When I first
proposed it that was my sell. I received pushback because it was for
too specific a purpose so I stepped back and showed that it was simply
a logical extension that happened to work as ReST input. Now it seems
that unless it is 100% ReST and documented as such it will be rejected.

I'm feeling the ground shift under me.

> Can I ask what is this logical extension of the border feature useful
> for, keeping in mind that rst is not it?

Perhaps some people will like this format better. I don't know. I
want it for ReST.

> Some people suggests that this is so close to rst that I should just use
> it as if it were, and hand-edit the output for the rare cases where it
> doesn't comply. I don't find this very compelling.

The cases are so rare that I can't remember what they were if any.

> Apparently the bottom line is that if it's not actual rst, it will get
> rejected.

OK, it's ReST. If there is a corner case that breaks ReST I am willing
to treat it as a bug and fix it.

Perhaps ReST should be the next level. That way people who just want
the extra lines can get what they want and people who need 100% ReST
can use "border 4" instead. That's if there is any difference of
course. We could document "border 4" as ReST with no change to my code
patch until we find some case where "border 3" breaks ReST.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> On Thu, 8 Jan 2009 12:30:52 -0300
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> > > It is a great feature for people actually using ReST. However, the
> > > feature is really just a logical extension to the existing border
> > > attribute.
> >
> > Frankly I don't understand your position. You seem to be saying that
> > you want the logical extension to the border feature, because it's very
> > easy to write, but you don't want to go to all the trouble of writing an
> > actual rst output format -- I guess it's a lot more code. You don't
> > care that your new border format is not actually rst, because you have
> > no need for rst.
>
> In fact I wrote it because I do want it for ReST. When I first
> proposed it that was my sell. I received pushback because it was for
> too specific a purpose so I stepped back and showed that it was simply
> a logical extension that happened to work as ReST input. Now it seems
> that unless it is 100% ReST and documented as such it will be rejected.
>
> I'm feeling the ground shift under me.

Can you find an email that shows this; I don't remember a shift.

> > Some people suggests that this is so close to rst that I should just use
> > it as if it were, and hand-edit the output for the rare cases where it
> > doesn't comply. I don't find this very compelling.
>
> The cases are so rare that I can't remember what they were if any.

Well, Tom pointed out a few, the most obvious being backslashing markup
characters:

http://docutils.sourceforge.net/docs/user/rst/quickref.html#escaping

and there is a heck of a lot of markup:

http://docutils.sourceforge.net/docs/user/rst/quickref.html#inline-markup

If you want another mode that doesn't do the backslashing, that is fine,
but the ReST-required backslashes are necessary for it to be accepted.

> > Apparently the bottom line is that if it's not actual rst, it will get
> > rejected.
>
> OK, it's ReST. If there is a corner case that breaks ReST I am willing
> to treat it as a bug and fix it.
>
> Perhaps ReST should be the next level. That way people who just want
> the extra lines can get what they want and people who need 100% ReST
> can use "border 4" instead. That's if there is any difference of
> course. We could document "border 4" as ReST with no change to my code
> patch until we find some case where "border 3" breaks ReST.

See above, there are lots of cases, and we aren't going to accept the
patch with partial ReST support and wait for people to complain --- it
is already clear the patch is not 100% ReST compliant.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian a écrit :
> D'Arcy J.M. Cain wrote:
>> On Thu, 8 Jan 2009 12:30:52 -0300
>> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
>>>> It is a great feature for people actually using ReST. However, the
>>>> feature is really just a logical extension to the existing border
>>>> attribute.
>>> Frankly I don't understand your position. You seem to be saying that
>>> you want the logical extension to the border feature, because it's very
>>> easy to write, but you don't want to go to all the trouble of writing an
>>> actual rst output format -- I guess it's a lot more code. You don't
>>> care that your new border format is not actually rst, because you have
>>> no need for rst.
>> In fact I wrote it because I do want it for ReST. When I first
>> proposed it that was my sell. I received pushback because it was for
>> too specific a purpose so I stepped back and showed that it was simply
>> a logical extension that happened to work as ReST input. Now it seems
>> that unless it is 100% ReST and documented as such it will be rejected.
>>
>> I'm feeling the ground shift under me.
>
> Can you find an email that shows this; I don't remember a shift.

I do remember the same : ReSt was rejecting because it was too boring to maintin
or get a 100% compliant ouput (btw are we 100% compliant with any SQL ?)

So the option was to just add some options to handle the border in another fashion.

This kind of patch (improve border,style, .. of output) sound like a good idea
for me.

>
>>> Some people suggests that this is so close to rst that I should just use
>>> it as if it were, and hand-edit the output for the rare cases where it
>>> doesn't comply. I don't find this very compelling.
>> The cases are so rare that I can't remember what they were if any.
>
> Well, Tom pointed out a few, the most obvious being backslashing markup
> characters:
>
> http://docutils.sourceforge.net/docs/user/rst/quickref.html#escaping
>
> and there is a heck of a lot of markup:
>
> http://docutils.sourceforge.net/docs/user/rst/quickref.html#inline-markup
>
> If you want another mode that doesn't do the backslashing, that is fine,
> but the ReST-required backslashes are necessary for it to be accepted.
>
>>> Apparently the bottom line is that if it's not actual rst, it will get
>>> rejected.
>> OK, it's ReST. If there is a corner case that breaks ReST I am willing
>> to treat it as a bug and fix it.
>>
>> Perhaps ReST should be the next level. That way people who just want
>> the extra lines can get what they want and people who need 100% ReST
>> can use "border 4" instead. That's if there is any difference of
>> course. We could document "border 4" as ReST with no change to my code
>> patch until we find some case where "border 3" breaks ReST.
>
> See above, there are lots of cases, and we aren't going to accept the
> patch with partial ReST support and wait for people to complain --- it
> is already clear the patch is not 100% ReST compliant.
>

- --
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAklmKysACgkQo/dppWjpEvz1PQCcDxu4cUeGDZGa+efCtbWe73pa
9dYAoKe2mqV1Mea4+UbtJJIMqNxGXvU9
=b+iU
-----END PGP SIGNATURE-----

Cdric Villemain wrote:
> Bruce Momjian a crit :
> > D'Arcy J.M. Cain wrote:
> >> On Thu, 8 Jan 2009 12:30:52 -0300
> >> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> >>>> It is a great feature for people actually using ReST. However, the
> >>>> feature is really just a logical extension to the existing border
> >>>> attribute.
> >>> Frankly I don't understand your position. You seem to be saying that
> >>> you want the logical extension to the border feature, because it's very
> >>> easy to write, but you don't want to go to all the trouble of writing an
> >>> actual rst output format -- I guess it's a lot more code. You don't
> >>> care that your new border format is not actually rst, because you have
> >>> no need for rst.
> >> In fact I wrote it because I do want it for ReST. When I first
> >> proposed it that was my sell. I received pushback because it was for
> >> too specific a purpose so I stepped back and showed that it was simply
> >> a logical extension that happened to work as ReST input. Now it seems
> >> that unless it is 100% ReST and documented as such it will be rejected.
> >>
> >> I'm feeling the ground shift under me.
> >
> > Can you find an email that shows this; I don't remember a shift.
>
> I do remember the same : ReSt was rejecting because it was too boring to maintin
> or get a 100% compliant ouput (btw are we 100% compliant with any SQL ?)

Where was that said? URL from archives? I am not saying I don't
believe you, it is just that I don't remember anyone saying that, at
least in the thread I read:

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01007.php

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> D'Arcy J.M. Cain wrote:

> > In fact I wrote it because I do want it for ReST. When I first
> > proposed it that was my sell. I received pushback because it was for
> > too specific a purpose so I stepped back and showed that it was simply
> > a logical extension that happened to work as ReST input. Now it seems
> > that unless it is 100% ReST and documented as such it will be rejected.
> >
> > I'm feeling the ground shift under me.
>
> Can you find an email that shows this; I don't remember a shift.

I'm surprised that you find this surprising. It happens all the time.
People change their mind, or they forget the decision they took last
time and take the opposite one later. Tom said that he didn't see a
value in rst:
http://archives.postgresql.org/message-id/27079.1219365411%40sss.pgh.pa.us
Andrew supported this view. However, their arguments were debunked.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > D'Arcy J.M. Cain wrote:
>
> > > In fact I wrote it because I do want it for ReST. When I first
> > > proposed it that was my sell. I received pushback because it was for
> > > too specific a purpose so I stepped back and showed that it was simply
> > > a logical extension that happened to work as ReST input. Now it seems
> > > that unless it is 100% ReST and documented as such it will be rejected.
> > >
> > > I'm feeling the ground shift under me.
> >
> > Can you find an email that shows this; I don't remember a shift.
>
> I'm surprised that you find this surprising. It happens all the time.
> People change their mind, or they forget the decision they took last
> time and take the opposite one later. Tom said that he didn't see a
> value in rst:
> http://archives.postgresql.org/message-id/27079.1219365411%40sss.pgh.pa.us
> Andrew supported this view. However, their arguments were debunked.

Right, so Tom says it isn't 100% ReST:

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php

and that is what I remember. I see no one liking the format for
esthetic sake, but only for ReST.

Here is where escaping backslashes was discussed:

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01323.php

Here is where someone says having it be 100% compliant is a waste of
time (which is almost sure to doom their argument): ;-)

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01311.php

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> People change their mind, or they forget the decision they took last
> time and take the opposite one later. Tom said that he didn't see a
> value in rst:
> http://archives.postgresql.org/message-id/27079.1219365411%40sss.pgh.pa.us

Well, actually, I *still* don't see the value in being able to emit ReST
--- nobody except D'Arcy has stated an interest in the feature. I'd
feel better about it if there were more votes for it. But in any case
I'm quite sure that ReST-except-it-breaks-on-various-special-characters
is not up to our project's quality expectations.

regards, tom lane

On Thu, 8 Jan 2009 12:08:06 -0500 (EST)
Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Right, so Tom says it isn't 100% ReST:
>
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php

Right but did you see the followup?

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01319.php

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> On Thu, 8 Jan 2009 12:08:06 -0500 (EST)
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Right, so Tom says it isn't 100% ReST:
> >
> > http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php
>
> Right but did you see the followup?
>
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01319.php

Yes, I did, and now I see why you said there might be only a few broken
cases. But I did not see any documentation in the standard saying that
was OK:

http://docutils.sourceforge.net/docs/user/rst/quickref.html#escaping

so it might be that your ReST interpreter is broken.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

"D'Arcy J.M. Cain" <darcy(at)druid(dot)net> writes:
> On Thu, 8 Jan 2009 12:08:06 -0500 (EST)
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> Right, so Tom says it isn't 100% ReST:
>>
>> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php

> Right but did you see the followup?
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01319.php

And the followup to that?

regards, tom lane

Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > People change their mind, or they forget the decision they took last
> > time and take the opposite one later. Tom said that he didn't see a
> > value in rst:
> > http://archives.postgresql.org/message-id/27079.1219365411%40sss.pgh.pa.us
>
> Well, actually, I *still* don't see the value in being able to emit ReST
> --- nobody except D'Arcy has stated an interest in the feature. I'd
> feel better about it if there were more votes for it. But in any case

I think I counted 5-6 who wanted it, which seems sufficient from the
hackers list. Besides we didn't actually ask who wanted it but rather
discussed implementation details.

> I'm quite sure that ReST-except-it-breaks-on-various-special-characters
> is not up to our project's quality expectations.

Yep.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Yes, I did, and now I see why you said there might be only a few broken
> cases. But I did not see any documentation in the standard saying that
> was OK:
> http://docutils.sourceforge.net/docs/user/rst/quickref.html#escaping

The example on that page is just bizarre. If the first asterisk
has to be escaped to prevent it being taken as markup, why doesn't the
second have to be? I suppose it's because there's no "closing" asterisk
to match it, but that's not the sort of thing one ought to depend on.

> so it might be that your ReST interpreter is broken.

Looking at this I'd say the ReST standard is broken, or at least expects
unreasonably complicated behavior. But anyway, it's quite clear that
there are *many* cases that need escaping, not only backslashes.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > Yes, I did, and now I see why you said there might be only a few broken
> > cases. But I did not see any documentation in the standard saying that
> > was OK:
> > http://docutils.sourceforge.net/docs/user/rst/quickref.html#escaping
>
> The example on that page is just bizarre. If the first asterisk
> has to be escaped to prevent it being taken as markup, why doesn't the
> second have to be? I suppose it's because there's no "closing" asterisk
> to match it, but that's not the sort of thing one ought to depend on.

So what would this show?

\*escape* \*escape*

Want to bet the second word it italics.

> > so it might be that your ReST interpreter is broken.
>
> Looking at this I'd say the ReST standard is broken, or at least expects
> unreasonably complicated behavior. But anyway, it's quite clear that
> there are *many* cases that need escaping, not only backslashes.

Yea, there doesn't seem to be enough information on that page to form a
standard behavior.

Ah, OK, here is where we should be looking:

http://docutils.sourceforge.net/rst.html#reference-documentation

URL updated on TODO list.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Thu, 8 Jan 2009, D'Arcy J.M. Cain wrote:

>> Some people suggests that this is so close to rst that I should just use
>> it as if it were, and hand-edit the output for the rare cases where it
>> doesn't comply. I don't find this very compelling.
>
> The cases are so rare that I can't remember what they were if any.

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php had
Tom's concerns. Since the ReST Grid Tables you're using is built with the
characters "-", "=", "|", and "+", those would seem the minimum that would
need to be escaped with a \, along with the \ itself, before this would be
likely to work with arbitrary table input.

After spending some time assembling a list of special characters, I had an
ah-ha moment when I realized they are all listed in the "Sections" section
as "section title adornment characters":

! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

Every markup character I found elsewhere shows up in this list, so that
seems like the definitive one: escape any character that appears there
with a \, and I don't see any obvious cases left that will keep this from
being valid ReST output. Shouldn't be a lot of code to add that feature,
and then I think most of the criticism of this patch would go away.

I also note that there are some bullet and arrow inputs it will treat as
special, see "Bullet Lists" in
http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
Seems reasonable to just document that non-ASCII input is somewhat
perilious as a known limitation.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Thu, 8 Jan 2009, Tom Lane wrote:

> Well, actually, I *still* don't see the value in being able to emit ReST
> --- nobody except D'Arcy has stated an interest in the feature.

I suggested interest in it and pointed out the popularity of ReST for
anyone using Trac or Python, and Cedric Villemain threw in his +1 too.
So just on the relatively small -hackers list we're up to three people who
think it's a nice addition.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Thu, 8 Jan 2009, D'Arcy J.M. Cain wrote:

> On Thu, 8 Jan 2009 12:08:06 -0500 (EST)
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> Right, so Tom says it isn't 100% ReST:
>>
>> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01310.php
>
> Right but did you see the followup?
>
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg01319.php

Unfortunately, finding cases where the special markup characters don't
matter isn't the same as proving that they will never matter. The best
example of that I noticed in the spec relates to Enumerated Lists. This:

A. Einstein was a really
smart dude.

Is parsed as two lines of text, while:

A. Einstein was a really smart dude.

Will be treated as a single-item list. That sort of ambiguity is quite a
hindrance to machine-generation of ReST code. As the spec itself is very
loose, barring a deep read of the docutils code to figure out the rules
that exist only via the code implementation it seems to me the only robust
way around it is to just escape every special character.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> After spending some time assembling a list of special characters, I had an
> ah-ha moment when I realized they are all listed in the "Sections" section
> as "section title adornment characters":

> ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~

I'll give you another ah-hah moment: that is exactly the list of all the
non-letter non-digit ASCII characters. (In order, even.) So this seems
to boil down to "if (ispunct(ch)) add backslash".

> I also note that there are some bullet and arrow inputs it will treat as
> special, see "Bullet Lists" in
> http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

Ick.

> Seems reasonable to just document that non-ASCII input is somewhat
> perilious as a known limitation.

Perhaps so, especially in view of all the encoding dependencies that
would arise if we tried to fix that. Still, I'm getting a stronger and
stronger impression that ReST is a good standard to stay away from.
Somewhere along the line its designers lost track of the notions of
simplicity and predictability. It's probably peachy for hand-generated
text, but for machine-generated output there are way too many cute special
cases.

regards, tom lane

On Thu, 2009-01-08 at 13:44 -0500, Greg Smith wrote:
> On Thu, 8 Jan 2009, Tom Lane wrote:
>
> > Well, actually, I *still* don't see the value in being able to emit ReST
> > --- nobody except D'Arcy has stated an interest in the feature.
>
> I suggested interest in it and pointed out the popularity of ReST for
> anyone using Trac or Python, and Cedric Villemain threw in his +1 too.
> So just on the relatively small -hackers list we're up to three people who
> think it's a nice addition.

There is interest in ReST for anyone doing a lot more than Python or
Trac. Although that area is certainly strong with it. It is quickly
becoming one of the more dominant technologies in delivering web
services (now whether or not that is useful here is another argument).

Joshua D. Drake

>
> --
> * Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD
>
--
PostgreSQL
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> There is interest in ReST for anyone doing a lot more than Python or
> Trac. Although that area is certainly strong with it. It is quickly
> becoming one of the more dominant technologies in delivering web
> services (now whether or not that is useful here is another argument).

Based on the number of weird special cases that we've enumerated in this
(very cursory) discussion, it seems like it's great for hand-generated
static text, ie, usages where you can see how it renders before you
publish it. For machine-generated output it seems to be full of
foot-guns, and so I wonder exactly how many people are really truly
using it as a data transmission format.

regards, tom lane

On Jan 8, 2009, at 13:56 , Joshua D. Drake wrote:

> There is interest in ReST for anyone doing a lot more than Python or
> Trac. Although that area is certainly strong with it. It is quickly
> becoming one of the more dominant technologies in delivering web
> services (now whether or not that is useful here is another argument).

I think there may be confusion here betwixt ReST/RST and REST.

REST: http://en.wikipedia.org/wiki/Representational_State_Transfer
ReST/RST: http://en.wikipedia.org/wiki/ReStructuredText

Michael Glaesemann
grzm seespotcode net

On Thu, 8 Jan 2009 14:15:26 -0500
Michael Glaesemann <grzm(at)seespotcode(dot)net> wrote:
> I think there may be confusion here betwixt ReST/RST and REST.
>
> REST: http://en.wikipedia.org/wiki/Representational_State_Transfer
> ReST/RST: http://en.wikipedia.org/wiki/ReStructuredText

Really? I don't think anyone in this conversation has been confused
about what technology we are talking about.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Thu, 2009-01-08 at 14:15 -0500, Michael Glaesemann wrote:
> On Jan 8, 2009, at 13:56 , Joshua D. Drake wrote:
>
> > There is interest in ReST for anyone doing a lot more than Python or
> > Trac. Although that area is certainly strong with it. It is quickly
> > becoming one of the more dominant technologies in delivering web
> > services (now whether or not that is useful here is another argument).
>
> I think there may be confusion here betwixt ReST/RST and REST.
>
> REST: http://en.wikipedia.org/wiki/Representational_State_Transfer
> ReST/RST: http://en.wikipedia.org/wiki/ReStructuredText

DOH!

Yes. Sorry about the noise.

Joshua D. Drake

--
PostgreSQL
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

On Thu, 8 Jan 2009 13:05:03 -0500 (EST)
Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> So what would this show?
>
> \*escape* \*escape*
>
> Want to bet the second word it italics.

Not in the Python implementation anyway. By the way, if you want to
try something, http://www.druid.net/darcy/rest.py.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

On Thu, 8 Jan 2009 13:51:44 -0500 (EST)
Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
> A. Einstein was a really
> smart dude.
>
> Is parsed as two lines of text, while:
>
> A. Einstein was a really smart dude.
>
> Will be treated as a single-item list. That sort of ambiguity is quite a

Yes, this is an issue. I'm not even sure how to fix it.

> hindrance to machine-generation of ReST code. As the spec itself is very
> loose, barring a deep read of the docutils code to figure out the rules
> that exist only via the code implementation it seems to me the only robust
> way around it is to just escape every special character.

Which character in the above example would you escape.

The problem with escaping is that someone may want this output for
non-ReST purposes. They may not be making themselves known now but if
we find a need later it will be hard if not impossible to make it
available in a logical way. I would suggest that if we want actual
ReST-safe output we should create a "border = 4" setting. The code
changes would be minimal. All we need to do is check for a value of 4
in addition to checking whether escaping is necessary.

I would still like to see some sort of plugin system for psql that
would allow filters to be created for output. That would make this
entire discussion moot as each of us could write whatever filter worked
for them. That's a bigger deal though.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> The problem with escaping is that someone may want this output for
> non-ReST purposes. They may not be making themselves known now but if
> we find a need later it will be hard if not impossible to make it
> available in a logical way. I would suggest that if we want actual
> ReST-safe output we should create a "border = 4" setting. The code
> changes would be minimal. All we need to do is check for a value of 4
> in addition to checking whether escaping is necessary.
>
> I would still like to see some sort of plugin system for psql that
> would allow filters to be created for output. That would make this
> entire discussion moot as each of us could write whatever filter worked
> for them. That's a bigger deal though.

Well, the way we do Html and LaTeX now is:

\pset format {html|latex}

so I assume ReST would be:

\pset format {html|latex|rest}

You can implement the user-visible, no-escaping version of ReST by using
border=3.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Thu, 8 Jan 2009, D'Arcy J.M. Cain wrote:

> On Thu, 8 Jan 2009 13:51:44 -0500 (EST)
> Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
>>
>> A. Einstein was a really smart dude.
> Which character in the above example would you escape.

"." is on the long list of characters to be escaped I sent out earlier.
The parser looks for all sorts of enumeration syntaxes--A., I), (IV)--but
they all require some punctuation which makes those characters the ones to
focus on.

> I would suggest that if we want actual ReST-safe output we should create
> a "border = 4" setting. The code changes would be minimal. All we need
> to do is check for a value of 4 in addition to checking whether escaping
> is necessary.

This seems like a reasonable spec to me. If you just want that general
format, you can get that and may very well end up with something that's
useful ReST anyway with the border=3 mode your existing patch implements.
As you demonstrated, there are several situations where a character you
think might do something special turns out to be ignored, with "\" being
the most likely to cause trouble.

If you really want something that will be valid ReST, use border=4, which
adds escaping for everything inside the table that's on that long list of
potential markup characters. That's going to make the table look quite
ugly if you have many characters on the escape list, but I don't see any
way to get around that without a heavy dependency on behavior that's only
documented in the docutils implementation itself. The "spec" itself is
really more of a user guide, and it sure isn't good enough to let you know
every case where special characters may be interpreted as markup. As
demonstrated with the Einstein example, you sometimes need a while line of
lookahead to figure that out anyway, which makes a smarter escaping
solution pretty hard to implement.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Thu, 8 Jan 2009 18:45:43 -0500 (EST)
Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
> >> A. Einstein was a really smart dude.
> > Which character in the above example would you escape.
>
> "." is on the long list of characters to be escaped I sent out earlier.
> The parser looks for all sorts of enumeration syntaxes--A., I), (IV)--but
> they all require some punctuation which makes those characters the ones to
> focus on.

I tried escaping the '.' but it didn't change the behaviour.

> > I would suggest that if we want actual ReST-safe output we should create
> > a "border = 4" setting. The code changes would be minimal. All we need
> > to do is check for a value of 4 in addition to checking whether escaping
> > is necessary.
>
> This seems like a reasonable spec to me. If you just want that general
> format, you can get that and may very well end up with something that's
> useful ReST anyway with the border=3 mode your existing patch implements.
> As you demonstrated, there are several situations where a character you
> think might do something special turns out to be ignored, with "\" being
> the most likely to cause trouble.

Enter the following into my test rig http://www.druid.darcy/rest.py:

+----+--------+
| id | name |
+====+========+
| 8 | T'est |
+----+--------+
| 9 | T*est |
+----+--------+
| 10 | T\est |
+----+--------+
| 11 | T\\est |
+----+--------+
| 12 | T_est |
+----+--------+
| 13 | T`est |
+----+--------+

Notice that only the backslash needs to be escaped. However, if you
just add the backslash it won't work because the table will be
malformed. You need to widen every other field as well.

As Tom has pointed out, ReST has problems beyond our implementation.
People who use it are aware of these warts. Given that I really think
that the cleanest solution is to just give them "border 3", don't
mention ReST in the docs and have it happily work for 95% of the cases
in a ReST processor.

How about my other proposal under the "Output filter for psql"
subject? That would let people create parsers as safe as they need
them. I think that this proposal is still useful for those that need
something quick and dirty though.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

D'Arcy J.M. Cain a écrit :
>
> As Tom has pointed out, ReST has problems beyond our implementation.
> People who use it are aware of these warts. Given that I really think
> that the cleanest solution is to just give them "border 3", don't
> mention ReST in the docs and have it happily work for 95% of the cases
> in a ReST processor.

+1

>
> How about my other proposal under the "Output filter for psql"
> subject? That would let people create parsers as safe as they need
> them. I think that this proposal is still useful for those that need
> something quick and dirty though.
>

Can be interesting, but for my own usage "border=3" will be enough.

- --
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAklneBwACgkQo/dppWjpEvxucQCeIuTatyfoEw/TkqAVLK/hI0wq
WkIAn3mt4tnMz3BAjXIJqtmMlj9d4r4l
=Ykl+
-----END PGP SIGNATURE-----

On Fri, 9 Jan 2009, D'Arcy J.M. Cain wrote:

>> "." is on the long list of characters to be escaped I sent out earlier.
>
> I tried escaping the '.' but it didn't change the behaviour.

I did try that specific exapmle in Trac and it worked fine for me. Using
your test rig (which you gave the wrong URL to:
http://www.druid.net/darcy/rest.py ), I see this:

I. Test -> 1. Test
I\. Test -> I. Test

Which I think suggests escaping the period does what I was
expecting--stops the conversion into an enumeration. Which may not matter
anyway because...

> However, if you just add the backslash it won't work because the table
> will be malformed.

Right, in order for escaping everything to work, you need to escape the
text first, then figure out the table width lest it be malformed. I knew
that but didn't think through that it's no longer just a simple escaping
change at that point.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Fri, 9 Jan 2009 22:46:06 -0500 (EST)
Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
> On Fri, 9 Jan 2009, D'Arcy J.M. Cain wrote:
>
> >> "." is on the long list of characters to be escaped I sent out earlier.
> >
> > I tried escaping the '.' but it didn't change the behaviour.
>
> I did try that specific exapmle in Trac and it worked fine for me. Using
> your test rig (which you gave the wrong URL to:
> http://www.druid.net/darcy/rest.py ), I see this:
>
> I. Test -> 1. Test
> I\. Test -> I. Test

You are right. I can't recall the actual test I did but somehow I
found a situation where escaping the period didn't DTRT. I know that
it was in a table which is all we care about in this discussion but I
just retested and got the same result as you.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

So, what's happening. Is this discussion going into Limbo again for
six months. It feels like the latest round of messages just went
around the same circles as before. Let me summarize the different
possibilities as I see them.

0. Drop this patch
1. Call it Rest and make it 100% compliant
2. Call it Rest-like
3. Call it simply border level 3

I don't think there are many supporting number 0 but... I think
everyone agrees that number 1 is difficult, perhaps impossible, to
achieve so its supporters probably drop into 0 or 2.

Is there any chance we can narrow our choices here in order to focus
discussions? Is it fair to say that our real choices are 0 and 3? Is
there anyone who thinks that number 1 is achievable or that number 2 is
a good precedent for the project?

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> So, what's happening. Is this discussion going into Limbo again for
> six months. It feels like the latest round of messages just went
> around the same circles as before. Let me summarize the different
> possibilities as I see them.
>
> 0. Drop this patch
> 1. Call it Rest and make it 100% compliant
> 2. Call it Rest-like
> 3. Call it simply border level 3
>
> I don't think there are many supporting number 0 but... I think
> everyone agrees that number 1 is difficult, perhaps impossible, to
> achieve so its supporters probably drop into 0 or 2.

My vote goes for 1.

I wonder why you think it's impossible. Is it because you must scan
the whole table before being able to print any of it? (For example to
add extra alignment for the escaping backslashes in a way that doesn't
render it invalid.) Note that psql already does that in aligned mode,
to determine the wide of the columns.

I now consider the rst step much more braindamaged, but this makes it
all the more important that psql gets the output right; otherwise it is
much more painful for the user (it no longer suffices to add one
backslash to the offending row -- it is necessary to edit every single
line of the table to make it rst-compliant).

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

D'Arcy J.M. Cain a écrit :
> So, what's happening. Is this discussion going into Limbo again for
> six months. It feels like the latest round of messages just went
> around the same circles as before. Let me summarize the different
> possibilities as I see them.
>
> 0. Drop this patch
> 1. Call it Rest and make it 100% compliant
> 2. Call it Rest-like
> 3. Call it simply border level 3
>
> I don't think there are many supporting number 0 but... I think
> everyone agrees that number 1 is difficult, perhaps impossible, to
> achieve so its supporters probably drop into 0 or 2.
>
> Is there any chance we can narrow our choices here in order to focus
> discussions? Is it fair to say that our real choices are 0 and 3? Is
> there anyone who thinks that number 1 is achievable or that number 2 is
> a good precedent for the project?
>

I go for 3 (so whithout escaping)

1. Can be interesting *but* if all special char are escaped it will become
useless : one good point with ReST is that you can read it like 'less
README.rest' and it is readable like python code. If it is full of escaped
characters it can turn unreadable by a human. (we, at dalibo, used to write our
docs with ReST and most of the time we don't need to escape special char).

I don't follow 0 or 2..

- --
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAklrQNkACgkQo/dppWjpEvyKeACgsfHLQzrgvbi7unlV15mYushD
YAAAoL1EYgz2bDLBdvEtfn+BlVBQhm/W
=qQls
-----END PGP SIGNATURE-----

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> My vote goes for 1.

> I wonder why you think it's impossible. Is it because you must scan
> the whole table before being able to print any of it? (For example to
> add extra alignment for the escaping backslashes in a way that doesn't
> render it invalid.) Note that psql already does that in aligned mode,
> to determine the wide of the columns.

Hmm. If RST is really that brain-damaged, the problem here is that it's
going to take a very large patch to make it work. It's not going to be
a "border" option but a separate output mode like latex or html. And
then we have to get into the discussion of whether there's really enough
demand for this to justify carrying such a large chunk of code.

In any case, my vote is for either 0 or 1; I'm unimpressed by anything
that emits RST-except-we-skipped-all-the-hard-parts.

regards, tom lane

On Mon, 12 Jan 2009, Cdric Villemain wrote:

> we, at dalibo, used to write our docs with ReST and most of the time we
> don't need to escape special char

I'm interested in this patch for a similar reason, ReST has been working
well for internal documentation at my office. I know I'll run into tables
that require massive escaping on the datasets I have here though. You
might have gotten lucky and had data with little punctuation, I don't
think the user base at large will.

Here's the simplest real-world example I found. This is from an app that
collects PC information. I fudged the second row to pick something that's
still realistic from this problem domain and looks more like markup:

+--------------+-------------------+
| ip | file |
+==============+===================+
| 192.168.0.1 | C:\WINDOWS\SYSTEM |
+--------------+-------------------+
| 192.168.0.1 | *.* |
+--------------+-------------------+

That renders like this:

ip file
192.168.0.1 C:WINDOWSSYSTEM
192.168.0.1 .

Which is no good. Escaping it must transform the text first, recompute
the widths based on that, and then display the table. That gives the
following:

+--------------+---------------------+
| ip | file |
+==============+=====================+
| 192.168.0.1 | C:\\WINDOWS\\SYSTEM |
+--------------+---------------------+
| 192.168.0.2 | \*.\* |
+--------------+---------------------+

As the shortest right thing to do.

I think the problem with the single markup character injection test D'Arcy
ran is that many of these only matter if two appear on a line; to get
italics you need two * characters for example, so showing a single *
doesn't get mangled doesn't prove anything. What I would normally want
next in this sort of situation is some more table input designed to render
badly for testing. I'll see what I can put together there, I am rather
good at breaking code.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Mon, 12 Jan 2009, D'Arcy J.M. Cain wrote:

> 0. Drop this patch
> 1. Call it Rest and make it 100% compliant
> 2. Call it Rest-like
> 3. Call it simply border level 3

Every time I play with this for a minute or two, I'm able to find some
real-world data that completely breaks the ReST compatibility of your
"border level 3" implementation in short order. I've been on the lookout
for them lately and they're not hard to find.

Since ReST compatibility was the only interesting part to me, that knocks
(3) off my list. Obviously I like the idea and don't want (0).

What I would suggest is reasonable is a hybrid of the two remaining ones:

4. Call it Rest-like and make it compliant with all the reasonable cases

If there is anything really hideous in the spec that's extremely difficult
to avoid accidentally tripping over without making the code really
complicated (the non-ASCII bullet stuff I mentioned comes to mind), those
we can just document as known limitations of ReST-like mode and move
along. Anybody who uses ReST regularly knows how easy it is to
accidentally inject things that are interpreted as market, and I don't
think that will be viewed as a PostgreSQL fault as long as there's a good
effort to escape around the most common markup failure situations. It's
not like there's a proper spec to conform against here anyway.

Alvaro suggests the aligned mode code path could be re-used here to find
the widths of the escaped cells, that sounds like a useful way to handle
the "escape all punctuation" pass I proposed.

Cedric expressed some concern that this result would be ugly, and that
basically he'd rather hand-edit it with the minimal escaping required
instead. I'd say turn that around: output an ugly but functional bit of
ReST, and if somebody wants to try removing some escapes to make it
prettier the onus is on them to try that without breaking things.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > My vote goes for 1.
>
> > I wonder why you think it's impossible. Is it because you must scan
> > the whole table before being able to print any of it? (For example to
> > add extra alignment for the escaping backslashes in a way that doesn't
> > render it invalid.) Note that psql already does that in aligned mode,
> > to determine the wide of the columns.
>
> Hmm. If RST is really that brain-damaged, the problem here is that it's
> going to take a very large patch to make it work. It's not going to be
> a "border" option but a separate output mode like latex or html. And
> then we have to get into the discussion of whether there's really enough
> demand for this to justify carrying such a large chunk of code.
>
> In any case, my vote is for either 0 or 1; I'm unimpressed by anything
> that emits RST-except-we-skipped-all-the-hard-parts.

Yep, that is my analysis as well. If you want a pretty ReST-like
output, that can be added later.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Mon, 12 Jan 2009 12:19:51 -0500 (EST)
Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Yep, that is my analysis as well. If you want a pretty ReST-like
> output, that can be added later.

Not if you use "border 3" for full ReST. I see nothing but pushback
later if you try to make "border 4" give less than "border 3."

Anyway, we have gone around the circle again and are no further ahead.
As I see it, the original proposal was a logical extension to the border
settings, it was extremely low impact on the code and some people would
have found it useful. Unfortunately I made the tactical error of
mentioning ReST early on and now it won't be accepted unless it is 100%
ReST safe which it probably can never be and if it came close it would
give butt ugly output in cases where it didn't need to.

I guess my better option now is to push for the output filter for psql
as I posted. It's a much bigger change and my simple filter for my
simple needs would be way more complicated but there doesn't seem to be
any hope down this road.

--
D'Arcy J.M. Cain <darcy(at)druid(dot)net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.

D'Arcy J.M. Cain wrote:
> On Mon, 12 Jan 2009 12:19:51 -0500 (EST)
> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Yep, that is my analysis as well. If you want a pretty ReST-like
> > output, that can be added later.
>
> Not if you use "border 3" for full ReST. I see nothing but pushback
> later if you try to make "border 4" give less than "border 3."
>
> Anyway, we have gone around the circle again and are no further ahead.
> As I see it, the original proposal was a logical extension to the border
> settings, it was extremely low impact on the code and some people would
> have found it useful. Unfortunately I made the tactical error of
> mentioning ReST early on and now it won't be accepted unless it is 100%
> ReST safe which it probably can never be and if it came close it would
> give butt ugly output in cases where it didn't need to.

I don't think it would have been accepted as just a new output format
because few people liked the new display --- they liked it only because
it was like ReST (ah, but then you have to mention ReST). ;-)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +