Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption

From: Atri Sharma <atri(dot)jiit(at)gmail(dot)com>
To: Tomas Vondra <tv(at)fuzzy(dot)cz>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: custom hash-based COUNT(DISTINCT) aggregate - unexpectedly high memory consumption
Date: 2013-10-08 11:52:44
Message-ID: CAOeZVicivJ6TDdczEyXYaCzwrQN9mtX16JDOsGgbMKu+xR2fFQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Oct 8, 2013 at 4:15 PM, Tomas Vondra <tv(at)fuzzy(dot)cz> wrote:
> On 8 Říjen 2013, 11:42, Atri Sharma wrote:
>>>
>>> I've made some significant improvements in the chaining version (in the
>>> master branch), now getting about the memory consumption I've estimated.
>>>
>> I agree, we can hope to reduce the memory consumption by making changes in
>> the current chaining implementation. I would like to look into changing
>> the data structure used for chaining from singly linked list to maybe skip
>> list or something else.
>
> Just to be sure - I haven't been messing with the HashAggregate
> implementation directly, but with a custom aggregate. But feel free to
> tweak the built-in hash table ;-)
>
> Tomas
>

Heh.

Do you mind if I try it out on the custom agg you built? I assume it
is on the github repo link you shared?

Regards,

Atri

--
Regards,

Atri
l'apprenant

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fabien COELHO 2013-10-08 12:15:03 Re: pgbench progress report improvements - split 3 v2 - A
Previous Message Rushabh Lathia 2013-10-08 11:38:17 Re: insert throw error when year field len > 4 for timestamptz datatype