Re: Prelim specs for parser hooks for plpgsql

Here's what I'm thinking of doing to enable plpgsql to resolve variable
references during the main SQL parser processing, instead of its current
hack of replacing references with $n in advance:

1. Add some fields to ParseState to carry hook function pointers as
well as "void *" passthrough arguments for the hook functions.

2. transformColumnRef will need two hooks, a pre-transform hook and
a post-transform hook. The pre-transform hook would be used to
implement "old style" semantics, ie, the plpgsql variable definition
takes precedence over query definition. The post hook would be used
to implement either "oracle style" semantics or throwing error for
conflicts.

3. The pre-transform hook would have a signature like

Node *hook(ParseState *pstate, ColumnRef *cref)

If it returns a node, we immediately return that as the analyzed value
of the ColumnRef. If there's no hook or it returns NULL, we proceed
with the normal analysis of the ColumnRef.

Note: I don't see any big need to pass the "void *passthrough" argument
to the hook explicitly --- if it needs it, it can fetch it out of the
ParseState for itself.

4. The post-transform hook would have a signature like

Node *hook(ParseState *pstate, ColumnRef *cref, Node *var)

"var" is either the analyzed equivalent of the ColumnRef (usually, but
not necessarily, a Var node) or NULL if the system couldn't find a
referent. If "var" is NULL then the hook can return a substitute node
(probably a Param), or it can return NULL to indicate that the normal
no-such-column error should be thrown. If "var" is not null then the
hook should either return "var", or throw error if it wants to complain
that the reference is ambiguous. (We will disallow the third
possibility of returning an override value when var isn't null. This
is because the normal processing may have already made changes to the
parse tree, which the hook won't have enough information to undo.)

It will take a certain amount of code rearrangement to implement the
post-transform hook --- we can't just add a call at the bottom of
transformColumnRef, because we will need to retain enough state to throw
the correct error if neither the core parser nor the hook can resolve
the ColumnRef. This seems reasonably do-able, though the actual hook
call may end up in some weird places like ParseFuncOrColumn.

5. We will also need a hook in transformParamRef() so that plpgsql can
implement old-style $n references to function parameters. In this case
there doesn't seem to be any real need for before/after hooks, since any
given parser caller should only need p_paramtypes or a hook, not both.
In fact, what I'd like to do is see if we can rip out p_paramtypes and
the parser's built-in processing of ParamRefs *entirely*, and
re-implement it as a hook supplied by PREPARE or Bind-message
processing. So the hook would just be

Node *hook(ParseState *pstate, ParamRef *pref)

and the core parser would just throw error if the hook's not there
or returns NULL.

6. Callers will need a way to set the hook pointers in a ParseState
before invoking parsing. I think we can just have them do the
equivalent steps to parse_analyze themselves, ie, call make_parsestate,
set the hooks, call transformStmt, call free_parsestate; all three of
those are exported already anyway. The separate entry point
parse_analyze_varparams should go away entirely since the functionality
it's setting up will move to hooks.

7. plpgsql will have a bit more of a problem since it's currently a
couple of call levels away from the parser --- it goes through SPI.
So we're going to have to modify SPI's API to some extent. I'm a bit
tempted to add an entry point that lets the caller supply the
parsed/analyzed/planned plantree, instead of assuming that that work
must be done internally to SPI. Any thoughts about that?

Any comments or objections to this plan of work?

regards, tom lane

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> 3. The pre-transform hook would have a signature like
> Node *hook(ParseState *pstate, ColumnRef *cref)
>
> 4. The post-transform hook would have a signature like
> Node *hook(ParseState *pstate, ColumnRef *cref, Node *var)

Are there any relationships between the hooks and
Function Call Hook/parser hook submitted by Pavel-san?
https://commitfest.postgresql.org/action/patch_view?id=160

If we can also use them in normal SQL parsing, they are very useful
to implement SYSDATE global variable for porting from Oracle.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center

2009/10/22 Itagaki Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>:
>
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
>> 3. The pre-transform hook would have a signature like
>>       Node *hook(ParseState *pstate, ColumnRef *cref)
>>
>> 4. The post-transform hook would have a signature like
>>       Node *hook(ParseState *pstate, ColumnRef *cref, Node *var)
>
> Are there any relationships between the hooks and
> Function Call Hook/parser hook submitted by Pavel-san?
> https://commitfest.postgresql.org/action/patch_view?id=160

No, "my" hook was little bit more general and global. And I was to
able inject running environment.

>
> If we can also use them in normal SQL parsing, they are very useful
> to implement SYSDATE global variable for porting from Oracle.
>

I thing, so you need a global hook, and it isn't what Tom propose. But
Tom's proposal is correct - we don't need plpgsql behave ouside
plpgsql. Simply, if you like add some special variables like SYSDATE
or others, then you need some add new global hook.

Pavel

Regards
Pavel Stehule

> Regards,
> ---
> ITAGAKI Takahiro
> NTT Open Source Software Center
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>