Re: CTE inlining

On 05/04/2017 11:36 AM, Tom Lane wrote:
> Serge Rielau <serge(at)rielau(dot)com> writes:
>>> On May 4, 2017, at 3:02 AM, Gavin Flower <GavinFlower(at)archidevsys(dot)co(dot)nz> wrote:
>>> On 30/04/17 16:28, Tom Lane wrote:
>>>> There's already a pretty large hill to climb here in the way of
>>>> breaking peoples' expectations about CTEs being optimization
>>>> fences. Breaking the documented semantics about CTEs being
>>>> single-evaluation seems to me to be an absolute non-starter.
>> Are you worried about semantics or performance?
>> With proper detection of mutating functions and snapshot isolation I do not see how a user would detect “lack of” single evaluation.
> I do not think a user will have any trouble "detecting" what we did
> if his query runs for two hours, rather than two minutes, because we
> executed some expensive function 100 times rather than the one time
> he expected it to run.
>
> Now you could argue that that's user error because he should have
> marked the expensive function with a sufficiently high cost to
> discourage us from flattening the CTE. But not everyone will have
> done that (and I'm not sure we have any planner smarts that would
> respond to such cases anyway). So what I'm saying is that if you're
> promising there will be no visible bad consequences, you're wrong.
>
> It may be worth breaking some peoples' queries to make other peoples'
> queries faster, but I think we need to tread pretty carefully there.
> And we definitely can't change any case where there would be visible
> semantic differences.
>

Yeah, the idea that this won't cause possibly significant pain is quite wrong. Quite by accident I came across an example just this morning where rewriting as a CTE makes a big improvement.

I wrote this query:

select (json_populate_record(null::mytype, myjson)).*
from mytable;

It turned out that this was an order of magnitude faster:

with r as
(
select json_populate_record(null::mytype, myjson) as x
from mytable
)
select (x).*
from r;

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services