Re: pluggable compression support

On 2013-06-15 14:11:54 +0200, Hannu Krosing wrote:
> > Could you perhaps actually read the the description and the discussion
> > before making wild suggestions? Possibly even the patch.
> > Compressed toast datums now *do* have an identifier of the compression
> > algorithm used.
> > That's how we can discern between pglz and whatever
> > algorithm we come up with.
> Claiming that the algorithm will be one of only two (current and
> "whatever algorithm we come up with ") suggests that it is
> only one bit, which is undoubtedly too little for having a "pluggable"
> compression API :)

No, I am thinking 127 + 2 possibly algorithms for now. If we need more
the space used for the algorithm can be extended transparently at that
point.

> > But those identifiers should be *small* (since they are added to all
> > Datums)

> if there will be any alignment at all between the datums, then
> one byte will be lost in the noise ("remember: nobody will need
> more than 256 compression algorithms")

No. There's no additional alignment involved here.

> OTOH, if you plan to put these format markers in the compressed
> stream and change the compression algorithm while reading it, I am
> lost.

The marker *needs* to be in the compressed stream. When decompressing a
datum we only only get passed the varlena.

> > and they need to be stable, even across pg_upgrade. So I think
> > making this user configurable would be grave error at this point.

> "at this point" in what sense ?

If we add another algorithm with different tradeofs we can have a column
attribute for choosing the algorithm. If there proofs to be a need to
add more configurabily, we can do that at that point.

Greetings,

Andres Freund