Re: Streaming replication and non-blocking I/O

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> It's going to be a bit more complicated in walsender/walreceiver to work
> with the libpq COPY API. We're going to need a WAL sending/receiving
> protocol on top of it, defined in terms of rows and columns passed
> through the COPY protocol.

AFAIR, libpq knows essentially nothing of the data being passed through
COPY --- it just treats that as a byte stream. I think you can define
any data format you want, it doesn't need to look exactly like a COPY
of a table would. In fact it's probably a lot better if it DOESN'T
look like COPY data once it gets past libpq, so that you can check
that it is WAL and not COPY data.

> One problem is the the standby is supposed to send back acknowledgments
> to the master, telling it how far it has received/replayed the WAL. Is
> there any way to send information back to the server, while a COPY OUT
> is in progress? That's not absolutely necessary with asynchronous
> replication, but will be with synchronous.

Well, a real COPY would of course not stop to look for incoming
messages, but I don't think that's inherent in the protocol. You
would likely need some libpq adjustments so it didn't throw error
when you tried that, but it would be a small and one-time adjustment.

> BTW, something that's been bothering me a bit with this patch is that we
> now have to link the backend with libpq. I don't see an immediate
> problem with that, but I'm not a packager. Does anyone see a problem
> with that?

Yeah, I have a problem with that. What's the backend doing with libpq?
It's not receiving this data, it's sending it.

regards, tom lane