From: | Heikki Linnakangas <hlinnakangas(at)vmware(dot)com> |
---|---|
To: | Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>, PgHacker <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: How to make ResourceOwnerForgetBuffer() O(1), instead of O(N^2) scale |
Date: | 2014-10-03 07:35:42 |
Message-ID: | 542E51CE.2070709@vmware.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 10/03/2014 07:08 AM, Kouhei Kaigai wrote:
> Hello,
>
> I recently got a trouble on development of my extension that utilizes
> the shared buffer when it released each buffer page.
>
> This extension transfers contents of the shared buffers to GPU device
> using DMA feature, then kicks a device kernel code.
Wow, that sounds crazy.
> Once backend/extension calls ReadBuffer(), resowner.c tracks which
> buffer was referenced by the current resource owner, to ensure these
> buffers being released at end of the transaction.
> However, it seems to me implementation of resowner.c didn't assume
> many buffers are referenced by a particular resource owner simultaneously.
> It manages the buffer index using an expandable array, then looks up
> the target buffer by sequential walk but from the tail because recently
> pinned buffer tends to be released first.
> It made a trouble in my case. My extension pinned multiple thousands
> buffers, so owner->buffers[] were enlarged and takes expensive cost
> to walk on.
> In my measurement, ResourceOwnerForgetBuffer() takes 36 seconds in
> total during hash-joining 2M rows; even though hash-joining itself
> takes less than 16 seconds.
>
> What is the best way to solve the problem?
How about creating a separate ResourceOwner for these buffer pins, and
doing a wholesale ResourceOwnerRelease() on it when you're done?
- Heikki
From | Date | Subject | |
---|---|---|---|
Next Message | Simon Riggs | 2014-10-03 08:07:41 | Re: Promise index tuples for UPSERT |
Previous Message | Heikki Linnakangas | 2014-10-03 07:29:35 | Re: Patch to add support of "IF NOT EXISTS" to others "CREATE" statements |