Re: how to pass data (tuples) to worker processes?

From: Andrew Tipton <andrew(at)kiwidrew(dot)com>
To: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Cc: Tomas Vondra <tv(at)fuzzy(dot)cz>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: how to pass data (tuples) to worker processes?
Date: 2013-08-03 10:31:34
Message-ID: CA+M2pVUxk5OP3y==LL+v_P7zrr2qAF6ZpTfpQAbN4z7h_CjQvw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Aug 3, 2013 at 5:43 AM, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>wrote:

> Tomas Vondra wrote:
>
> > I'm learning how to use the "background worker processes" commited in
> > 9.3. The usage basics are quite nicely illustrated in the worker_spi
> > extension (kudos to those who designed the feature / extension).
>
> Thanks!
>
> > I'm not quite sure how to pass data between the regular backend and a
> > worker. Implementing the channel (socket/pipe/...) itself is not a big
> > deal, that's IPC 101, but deciding which data to copy (and how) is.
> >
> > [...]
>
> Were you able to figure it out? If so, would you share?

I'm also in the middle of doing some experiments with bgworkers, and for me
it's the IPC part that's proving tricky. I'd love to have a simple socket
that can be used to communicate with the bgworker. But because the
bgworker is launched by the postmaster -- and not the backend which
registers it -- there's no chance for the bgworker to inherit one end of
the socketpair().

Tomas: in the end, what approach did you use for IPC?

Robert: any chance you could share a few more details on the enhancements
you're planning for bgworkers? I seem to recall reading that communicating
with the dynamic bgworkers after they had been launched was next on your
agenda...

Regards,
Andrew Tipton

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dimitri Fontaine 2013-08-03 15:04:40 Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])
Previous Message Satoshi Nagayasu 2013-08-03 10:24:31 Re: inconsistent state after crash recovery