Configurable Send/Receive Buffer Sizes

Folks,

please find attached a patch which allows to specify the size of TCP
send/receive buffers (SO_SNDBUF/SO_RECVBUF socket options) in the JDBC driver.

The patch adds two connection parameters to the JDBC driver:

send_buffer_size: Specifes the size of the TCP write buffer used with the
connection

receive_buffer_size: Specifies the size of the TCP read buffer used with the
connection

For example:

Connection conn =
DriverManager.getConnection("jdbc:postgresql://localhost:5432/bernd?loglevel=2&send_buffer_size=262144&receive_buffer_size=1048576",
"bernd", "");

It occured to me several times now where this functionality became useful,
especially when working on migration projects in the past, where the
application couldn't be changed immediately when doing large batch processing.
An interesting example is the small test program from

<http://archives.postgresql.org/pgsql-jdbc/2008-10/msg00045.php>

which could be make working with the appropiate buffer settings.

Maybe this is worth for discussion or even to investigate it further for
inclusion.

Note that on Linux you need to adjust the kernel maximum buffer sizes in
/proc/sys/net/core/wmem_max and /proc/sys/net/core/rmem_max to become any
settings effective. I have reports that this isn't necessary on Windows,
however. This certainly needs adequate documentation.

I didn't invest any time on benchmarks, but maybe there's also some value in it
performance-wise.

--
Thanks

Bernd

Bernd Helmle <mailings(at)oopsware(dot)de> wrote:

> An interesting example is the small test program from
>
> <http://archives.postgresql.org/pgsql-jdbc/2008-10/msg00045.php>
>
> which could be make working with the appropiate buffer settings.

> I didn't invest any time on benchmarks, but maybe there's also
> some value in it performance-wise.

So the primary point is to avoid deadlocks on full network buffers?

-Kevin

--On 21. April 2011 09:49:54 -0500 Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>
wrote:

> Bernd Helmle <mailings(at)oopsware(dot)de> wrote:
>
>> An interesting example is the small test program from
>>
>> <http://archives.postgresql.org/pgsql-jdbc/2008-10/msg00045.php>
>>
>> which could be make working with the appropiate buffer settings.
>
>> I didn't invest any time on benchmarks, but maybe there's also
>> some value in it performance-wise.
>
> So the primary point is to avoid deadlocks on full network buffers?
>

Yeah, that was the primary idea...or, well, to workaround them. You could argue
that its still possible to run into those issue, but less likely. With the
proper settings it didn't occur to me again, though.

--
Thanks

Bernd

I fully support this request! I currently have a overridden PGStream in my Classpath to be able to set it manually to 128k. Else our update scripts hang for the same reason. Making this configurable would be of great help.

Regards,
Daniel Migowski

________________________________________
Von: pgsql-jdbc-owner(at)postgresql(dot)org [pgsql-jdbc-owner(at)postgresql(dot)org]&quot; im Auftrag von &quot;Bernd Helmle [mailings(at)oopsware(dot)de]
Gesendet: Donnerstag, 21. April 2011 17:16
An: Kevin Grittner; pgsql-jdbc(at)postgresql(dot)org
Betreff: Re: [JDBC] Configurable Send/Receive Buffer Sizes

--On 21. April 2011 09:49:54 -0500 Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>
wrote:

> Bernd Helmle <mailings(at)oopsware(dot)de> wrote:
>
>> An interesting example is the small test program from
>>
>> <http://archives.postgresql.org/pgsql-jdbc/2008-10/msg00045.php>
>>
>> which could be make working with the appropiate buffer settings.
>
>> I didn't invest any time on benchmarks, but maybe there's also
>> some value in it performance-wise.
>
> So the primary point is to avoid deadlocks on full network buffers?
>

Yeah, that was the primary idea...or, well, to workaround them. You could argue
that its still possible to run into those issue, but less likely. With the
proper settings it didn't occur to me again, though.

--
Thanks

Bernd

--
Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-jdbc

Is there any objection to this patch ?

Otherwise I will look at committing it .

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

On Thu, Apr 21, 2011 at 10:42 AM, Bernd Helmle <mailings(at)oopsware(dot)de> wrote:
> Folks,
>
> please find attached a patch which allows to specify the size of TCP
> send/receive buffers (SO_SNDBUF/SO_RECVBUF socket options) in the JDBC
> driver.
>
> The patch adds two connection parameters to the JDBC driver:
>
> send_buffer_size: Specifes the size of the TCP write buffer used with the
> connection
>
> receive_buffer_size: Specifies the size of the TCP read buffer used with the
> connection
>
> For example:
>
> Connection conn =
> DriverManager.getConnection("jdbc:postgresql://localhost:5432/bernd?loglevel=2&send_buffer_size=262144&receive_buffer_size=1048576",
> "bernd", "");
>
> It occured to me several times now where this functionality became useful,
> especially when working on migration projects in the past, where the
> application couldn't be changed immediately when doing large batch
> processing. An interesting example is the small test program from
>
> <http://archives.postgresql.org/pgsql-jdbc/2008-10/msg00045.php>
>
> which could be make working with the appropiate buffer settings.
>
> Maybe this is worth for discussion or even to investigate it further for
> inclusion.
>
> Note that on Linux you need to adjust the kernel maximum buffer sizes in
> /proc/sys/net/core/wmem_max and /proc/sys/net/core/rmem_max to become any
> settings effective. I have reports that this isn't necessary on Windows,
> however.  This certainly needs adequate documentation.
>
> I didn't invest any time on benchmarks, but maybe there's also some value in
> it performance-wise.
>
> --
> Thanks
>
>        Bernd
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc
>
>