Pipelining executions to postgresql server

From: Mikko Tiihonen <Mikko(dot)Tiihonen(at)nitorcreations(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, "pgsql-jdbc(at)postgresql(dot)org" <pgsql-jdbc(at)postgresql(dot)org>
Subject: Pipelining executions to postgresql server
Date: 2014-11-01 14:04:05
Message-ID: 1414850643686.34687@nitorcreations.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-jdbc

Hi,

I created a proof of concecpt patch for postgresql JDBC driver that allows the caller to do pipelining of requests within a transaction. The pipelining here means same as for HTTP: the client can send the next execution already before waiting for the response of the previous request to be fully processed.

The goal is to reduce the effects of latency between server and client. The pipelining allowed my test with localhost postgresql and jdbc test that queries a single value from database 200 times to get a more than 20% speed-up. The pipelining version processes the responses every 10 queries. With actual latency between the server and client larger speed-ups are of course possible.

I think pipelining + jsonb support would make postgresql even more competive key/value and document store.

Example use case:
1) insert to shopping cart table, and 3 inserts shopping cart rows table in one pipeline.
- only one round trip penalty instead of 4
2) query shopping cart row and query shopping cart rows in one pipeline
- only one round trip penalty instead of 2

The only alternative way to reduce the round-trip latency is to make every operation in single round-trip and that can only be done with pl functions and by passing in more complex objects, for example the whole shopping cart with rows as json data.

What kind of problems could pipelining cause (assuming we limit it to rather simple use cases only)?

-Mikko

Attachment Content-Type Size
Example-pipelined-jdbc-single-shot-query.patch text/x-patch 15.2 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2014-11-01 14:06:03 Re: Switch pg_basebackup to use -X stream instead of -X fetch by default?
Previous Message Peter Eisentraut 2014-11-01 13:57:33 Re: pg_ctl non-idempotent behavior change

Browse pgsql-jdbc by date

  From Date Subject
Next Message Tom Lane 2014-11-01 16:13:03 Re: Pipelining executions to postgresql server
Previous Message Mikko Tiihonen 2014-11-01 00:34:19 Pull request for better handling of master/slave and load balancing when connecting to multiple hosts