Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

On 8 Říjen 2013, 13:52, Atri Sharma wrote:
> On Tue, Oct 8, 2013 at 4:15 PM, Tomas Vondra <tv(at)fuzzy(dot)cz> wrote:
>> On 8 Říjen 2013, 11:42, Atri Sharma wrote:
>>>>
>>>> I've made some significant improvements in the chaining version (in
>>>> the
>>>> master branch), now getting about the memory consumption I've
>>>> estimated.
>>>>
>>> I agree, we can hope to reduce the memory consumption by making changes
>>> in
>>> the current chaining implementation. I would like to look into changing
>>> the data structure used for chaining from singly linked list to maybe
>>> skip
>>> list or something else.
>>
>> Just to be sure - I haven't been messing with the HashAggregate
>> implementation directly, but with a custom aggregate. But feel free to
>> tweak the built-in hash table ;-)
>>
>> Tomas
>>
>
> Heh.
>
> Do you mind if I try it out on the custom agg you built? I assume it
> is on the github repo link you shared?

Not at all, that's why I pushed that into a public repo. The "master"
branch contains the regular chained hash table, the open addressing is in
a separate branch (also in the repo).

Tomas