Adding pipelining support to set returning functions

[Hannu Krosing <hannu(at)krosing(dot)net>]

A question to all pg hackers

Is anybody working on adding pipelining to set returning functions.

How much effort would it take ?

Where should I start digging ?

BACKGROUND:

AFAICS , currently set returning functions materialise their results
before returning, as seen by this simple test:

hannu=# select * from generate_series(1,10) limit 2;
 generate_series 
-----------------
               1
               2
(2 rows)

Time: 1.183 ms


hannu=# select * from generate_series(1,10000000) limit 2;
 generate_series 
-----------------
               1
               2
(2 rows)

Time: 3795.032 ms

being able to pipeline (generate results as needed) would enable several
interesting techniques, especially if combined with pl/proxy or any
other functions which stream external data.

Applications and design patterns like http://telegraph.cs.berkeley.edu/
or http://labs.google.com/papers/mapreduce.html would suddenly become
very easy to implement.

-----------------
Hannu