Re: Custom Scan APIs (Re: Custom Plan node)

On Tue, Apr 15, 2014 at 10:44 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Mon, Apr 14, 2014 at 4:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Yeah. After a fast review of the custom-scan and cache-scan patches, it
>>> seems to me that my original fears are largely confirmed: the custom scan
>>> patch is not going to be sufficient to allow development of any truly new
>>> plan type. Yeah, you can plug in some new execution node types, but
>>> actually doing anything interesting is going to require patching other
>>> parts of the system.
>
>> Without prejudice to the rest of what you said, this argument doesn't
>> hold much water with me. I mean, anything that our extensibility
>> mechanism doesn't support today will require new hooks, but does that
>> mean we're never going to add any more hooks? I sure hope not.
>
> No, that's not what I said. ISTM that the argument for the custom-scan
> API is that it allows interesting new things to be done *without further
> modifying the core code*. But the example application (cache_scan) fails
> to demonstrate that, and indeed seems to be a counterexample. Whether
> we'd accept cache_scan on its own merits is a separate question. The
> problem for me is that custom-scan isn't showing that it can support what
> was claimed without doing serious damage to modularity and maintainability
> of the core code.

I think there's two separate things in there, one of which I agree
with and one of which I disagree with. I agree that we must avoid
damaging the modularity and maintainability of the core code; I don't
agree that custom-scan needs to be able to do interesting things with
zero additional changes to the core code. If we come up with three
interesting applications for custom scan that require 5 new hooks
between them, I'll consider that a major success - assuming those
hooks don't unduly limit future changes we may wish to make in the
core code. I think your concern about exposing APIs that may not be
terribly stable is well-founded, but I don't think that means we
shouldn't expose *anything*.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company