| From: | Stephen Frost <sfrost(at)snowman(dot)net> |
|---|---|
| To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
| Cc: | Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>, Kohei KaiGai <kaigai(at)kaigai(dot)gr(dot)jp>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>, Jim Mlodgenski <jimmy76(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, PgHacker <pgsql-hackers(at)postgresql(dot)org>, Peter Eisentraut <peter_e(at)gmx(dot)net> |
| Subject: | Re: Custom Scan APIs (Re: Custom Plan node) |
| Date: | 2014-03-02 02:04:28 |
| Message-ID: | 20140302020428.GI12995@tamriel.snowman.net |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> I don't see that parallelizing Append is any easier than any other
> problem in this space. There's no parallel I/O facility, so you need
> a background worker per append branch to wait on I/O. And you have
> all the problems of making sure that the workers have the same
> snapshot, making sure they don't self-deadlock, etc. that you have for
> any other case.
Erm, my thought was to use a select() loop which sends out I/O requests
and then loops around waiting to see who finishes it. It doesn't
parallelize the CPU cost of getting the rows back to the caller, but
it'd at least parallelize the I/O, and if what's underneath is actually
a remote FDW running a complex query (because the other side is actually
a view), it would be a massive win to have all the remote FDWs executing
concurrently instead of serially as we have today.
Thanks,
Stephen
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Stephen Frost | 2014-03-02 02:08:19 | Re: Custom Scan APIs (Re: Custom Plan node) |
| Previous Message | Kohei KaiGai | 2014-03-02 02:03:03 | Re: Custom Scan APIs (Re: Custom Plan node) |