Re: [ANNOUNCE] IMCS: In Memory Columnar Store for PostgreSQL

Hi David,

Sorry, but I do not completely understand your suggestions:

1. IMCS really contains single patch file sysv_shmem.patch.
Applying this patch is not mandatory for using IMCS: it just solves the
problem with support of > 256Gb of shared memory.
Right now PostgreSQL is not able to use more than 256Gb shared buffers
at Linux with standard 4kb pages.
I have found proposal for using MAP_HUGETLB flag in commit fest:

http://www.postgresql.org/message-id/20131125032920.GA23793@toroid.org

but unfortunately it was rejected. Hugepages are intensively used by
Oracle and I think that them will be useful for improving performance of
PorstreSQL. So not just IMCS can benefit from this patch. My patch is
much more simple - I specially limited scope of this patch to one file.
Certainly switch huge tlb on/off should be done through postgresql.conf
configuration file.

In any case - IMCS can be used without this patch: you just could not
use more than 256Gb memory, even if your system has more RAM.

2. I do not understand "The add-on is not formatted as an EXTENSION"
IMCS was created as standard extension - I just look at the examples of
other PostgreSQL extensions included in PostgreSQL distribution
(for example pg_stat_statements). It can be added using "create
extension imcs" and removed "drop extension imcs" commands.

If there are some violations of PostgreSQL extensions rules, please let
me know, I will fix them.
But I thought that I have done everything in legal way.

On 01/04/2014 03:21 AM, David Fetter wrote:
> On Thu, Jan 02, 2014 at 08:48:24PM +0400, knizhnik wrote:
>> I want to announce implementation of In-Memory Columnar Store
>> extension for PostgreSQL.
>> Vertical representation of data is stored in PostgreSQL shared memory.
> Thanks for the hard work!
>
> I noticed a couple of things about this that probably need some
> improvement.
>
> 1. There are unexplained patches against other parts of PostgreSQL,
> which means that they may break other parts of PostgreSQL in equally
> inexplicable ways. Please rearrange the patch so it doesn't require
> this. This leads to:
>
> 2. The add-on is not formatted as an EXTENSION, which would allow
> people to add it or remove it cleanly.
>
> Would you be so kind as to fix these?
>
> Cheers,
> David.