Re: libpq compression

On Mon, Jun 25, 2012 at 2:26 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, Jun 25, 2012 at 4:04 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira <euler(at)timbira(dot)com> wrote:
>>> On 20-06-2012 17:40, Marko Kreen wrote:
>>>> On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug <fgp(at)phlo(dot)org> wrote:
>>>>> I'm starting to think that relying on SSL/TLS for compression of
>>>>> unencrypted connections might not be such a good idea after all. We'd
>>>>> be using the protocol in a way it quite clearly never was intended to
>>>>> be used...
>>>>
>>>> Maybe, but what is the argument that we should avoid
>>>> on encryption+compression at the same time?
>>>>
>>>> AES is quite lightweight compared to compression, so should
>>>> be no problem in situations where you care about compression.
>>>>
>>> If we could solve compression problem without AES that will turn things
>>> easier. Compression-only via encryption is a weird manner to solve the problem
>>> in the user's POV.
>>>
>>>> RSA is noticeable, but only for short connections.
>>>> Thus easily solvable with connection pooling.
>>>>
>>> RSA overhead is not the main problem. SSL/TLS setup is.
>>>
>>>> And for really special compression needs you can always
>>>> create a UDF that does custom compression for you.
>>>>
>>> You have to own the code to modify it; it is not always an option.
>>>
>>>> So what exactly is the situation we need to solve
>>>> with postgres-specific protocol compression?
>>>>
>>> Compression only support. Why do I need to set up SSL/TLS just for compression?
>>>
>>> IMHO SSL/TLS use is no different from relying in another library to handle
>>> compression for the protocol and more it is compression-specific. That way, we
>>> could implement another algorithms in such library without needing to modify
>>> libpq code. Using SSL/TLS you are bounded by what SSL/TLS software products
>>> decide to use as compression algorithms. I'll be happy to maintain the code
>>> iif it is postgres-specific or even as close as possible to core.
>>
>> I guess my feeling on this is that, so far as I can see, supporting
>> compression via OpenSSL involves work and trade-offs, and supporting
>> it without depending on OpenSSL also involves work, and trade-offs.
>
> Nice summary :)
>
>> So it's not real evident to me that we should prefer one to the other
>> on general principle. It seems to me that a lot might come down to
>> performance. If someone can demonstrate that using an external
>> library involves gets significantly better compression, chews up
>> significantly less CPU time, and/or is significantly less code than
>> supporting this via OpenSSL, then maybe we ought to consider it.
>
> I think we should, yes. But as you say, we need to know first. It's
> also a question of if one of these compression schemes are trivial
> enough that we could embed the code rather than rely on it externally
> - I have no idea if that's even remotely possibe, but that would move
> the goalposts a bit too.

A followup on this thread. I was researching something else, and
stumbled across
http://www.openssl.org/docs/ssl/SSL_COMP_add_compression_method.html,
which says:

"
The TLS standard (or SSLv3) allows the integration of compression
methods into the communication. The TLS RFC does however not specify
compression methods or their corresponding identifiers, so there is
currently no compatible way to integrate compression with unknown
peers. It is therefore currently not recommended to integrate
compression into applications. Applications for non-public use may
agree on certain compression methods. Using different compression
methods with the same identifier will lead to connection failure.
"

Which I think is another reason to not go down that path as our
"official way to do compression".

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/