Re: reparsing query

Hi
I knock once again with this : reparse query to XML ( last knock 5-6 years
before) .
Motivation: more info for front-end developers about query. Users of jdbc
driver
are not able to gather information from meta data about such simple query :
"select x.a,y.b from xxx x, yyy y"
Solution : modification backend/utils/adt/ruleutils.c

I've done it few years ago, but with every new version of potgresql there
is a need to modify library.
I prepared version for 9.4.1, standalone, which could be used as library
loaded by backend. I wish to have such functionality in main distro.
Is anyone interested ?

Andrzej Barszcz

On Wed, Apr 15, 2015 at 1:40 PM, Andrzej Barszcz <abusinf(at)gmail(dot)com> wrote:
> I knock once again with this : reparse query to XML ( last knock 5-6 years
> before) .
>

What exactly "reparse query to XML" does?

Regards,
Qingqing

Qingqing Zhou wrote:
> On Wed, Apr 15, 2015 at 1:40 PM, Andrzej Barszcz <abusinf(at)gmail(dot)com> wrote:
> > I knock once again with this : reparse query to XML ( last knock 5-6 years
> > before) .
> >
>
> What exactly "reparse query to XML" does?

His old posting:
https://www.postgresql.org/message-id/1247323023.16438.35.camel%40ab-desktop

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Wed, Apr 15, 2015 at 2:04 PM, Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com> wrote:
> His old posting:
> https://www.postgresql.org/message-id/1247323023.16438.35.camel%40ab-desktop
>

Is this a proposal to have a better formatted (JSON etc)
debug_print_parse results?

Thanks,
Qingqing

On Wed, Apr 15, 2015 at 2:43 PM, Qingqing Zhou <zhouqq(dot)postgres(at)gmail(dot)com>
wrote:
>
> Is this a proposal to have a better formatted (JSON etc)
> debug_print_parse results?

I've run into the need for this as well for monitoring purposes, my
solution was to compile the needed object files into a library (see [0]).

It'd be interesting to explore if there is some way to make this less
hack-ish, and enable tools to parse queries in a better way. Essentially
what is needed is some way to reliably translate SQL into an AST-like
output, from an outside tool, whilst reusing the current PostgreSQL parser.

I believe the recent work on deparsing DDL statements relates to some of
that (i.e. might be re-usable for this purpose, through an extension), if
I'm understanding correctly.

[0]: https://github.com/pganalyze/pg_query

Best,
Lukas

--
Lukas Fittl

Skype: lfittl
Phone: +1 415 321 0630

On Wed, Apr 15, 2015 at 5:19 PM, Lukas Fittl <lukas(at)fittl(dot)com> wrote:
> It'd be interesting to explore if there is some way to make this less
> hack-ish, and enable tools to parse queries in a better way. Essentially
> what is needed is some way to reliably translate SQL into an AST-like
> output, from an outside tool, whilst reusing the current PostgreSQL parser.
>
It is not difficult to output parsed query in some tool readable
format but it comes with a maintain overhead: once tools rely on it,
we have to conform to some schema continuously, like the xml/xmlns. Do
we want to take this? Depends on how far the tools can go with this
exposed information.

We actually already have explain command in json format and tools
drawing/analyzing query plan rely on it. Will that work for your
scenario?

Regards,
Qingqing

On Wed, Apr 15, 2015 at 8:39 PM, Qingqing Zhou <zhouqq(dot)postgres(at)gmail(dot)com>
wrote:
>
> It is not difficult to output parsed query in some tool readable
> format but it comes with a maintain overhead: once tools rely on it,
> we have to conform to some schema continuously, like the xml/xmlns. Do
> we want to take this? Depends on how far the tools can go with this
> exposed information.
>
> We actually already have explain command in json format and tools
> drawing/analyzing query plan rely on it. Will that work for your
> scenario?
>

Sure, that would work - but as I understand adding an explicit SQL command
(or function) is not something that would ever be merged into Postgres
core. Especially since that command's output could easily change across
versions.

My thought was more along the lines of making something like raw_parser +
nodeToString available through an extension, but with a JSON output format.

Note that an important detail in the monitoring case is that you don't
necessarily have the statement's underlying relations available (since you
might work with statistics data on a different machine).

Best,
Lukas

--
Lukas Fittl

Skype: lfittl
Phone: +1 415 321 0630

> It is not difficult to output parsed query in some tool readable
> format but it comes with a maintain overhead: once tools rely on it,
> we have to conform to some schema continuously, like the xml/xmlns. Do
> we want to take this? Depends on how far the tools can go with this
> exposed information.

I think part of the problems could be resolved by using adequate API
to the parse tree. For example, pgpool-II imports PostgreSQL's raw
parser to do a query rewriting. The parse tree format could be changed
release by release but essential API for it (for example tree_walker)
is very stable. As a result, the workforce for rewriting rarely needs
to be modified.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp