Re: Streaming LIBPQ? Sliding Window in LIBPQ?

[Gregory Stark <stark(at)enterprisedb(dot)com>]

"Martijn van Oosterhout" <kleptog(at)svana(dot)org> writes:

> On Wed, Nov 28, 2007 at 09:11:07AM -0600, Abraham, Danny wrote:
>> Is there a way to break the PGresult array to chuncks  Like Oracle? 
>> 
>> I mean, without changing the text of given queries, 
>> 
>> can we  somehow limit the memory consumption of a client that is using
>> LIBPQ?
>
> Use non-blocking mode to get the data in chunks. You can't easy control
> the number of rows you get each time though...

This doesn't really help. It's nonblocking but you still can't get libpq to
actually give you a result data structure until the entire results have
accumulated.

The only interface like this libpq supports is to use cursors in your SQL and
then FETCH n for each chunk. There's no libpq support for receiving results
incrementally. 

If you're writing a driver implementing the protocol from scratch you could
expose chunks of results to the application but there's no protocol-level
support for it so you can't directly control the rate at which results arrive
or the chunk size or anything like that.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!