Stream bitmaps

Hi all,

I'd like to see the indexam API changes needed by the bitmap indexam to
be committed soon. Has anyone looked at the proposed API in the latest
patch? Any thoughts?

I'm quite happy with it myself, with a few reservations:

- All the getbitmap implementations except the new bitmap indexam are
just boilerplate. How about making getbitmap-function optional, and
having a generic implementation that fills in a hash bitmap using the
traditional getnext function?

- getbitmap is passed an existing bitmap as argument, and the
implementation needs to OR the existing bitmap with new tuples. How
about AND? An indexam could be smart about ANDing with an existing
bitmap, for example skipping to the first set bit in the existing bitmap
and starting the scan from there.

- I'd like to have support to return candidate matches with both
getbitmap and getnext. A simple flag per page of results would be enough
for getbitmap, I think.

- StreamBitmap and HashBitmap are separate node types, but OpStream is
not. opaque-field in the StreamBitmap struct is not really that opaque,
it needs to be a StreamNode. I drew a UML sketch of what I think the
class-hierarchy is
(http://community.enterprisedb.com/streambitmaps.png). This is
object-oriented programming, we're just implementing classes and
inheritance with structs and function pointers. The current patch mixes
different techniques, and that needs to be cleaned up.

I'd like to see a separate patch that contains just the API changes.
Gavin, could you extract an API-only patch from the bitmap index patch?
I can work on it as well, but I don't want to step on your toes.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki,

On Mon, 5 Mar 2007, Heikki Linnakangas wrote:

> Hi all,
>
> I'd like to see the indexam API changes needed by the bitmap indexam to
> be committed soon. Has anyone looked at the proposed API in the latest
> patch? Any thoughts?

Thanks for looking at it!

>
> I'm quite happy with it myself, with a few reservations:
>
> - All the getbitmap implementations except the new bitmap indexam are
> just boilerplate. How about making getbitmap-function optional, and
> having a generic implementation that fills in a hash bitmap using the
> traditional getnext function?
>
> - getbitmap is passed an existing bitmap as argument, and the
> implementation needs to OR the existing bitmap with new tuples. How
> about AND? An indexam could be smart about ANDing with an existing
> bitmap, for example skipping to the first set bit in the existing bitmap
> and starting the scan from there.
>
> - I'd like to have support to return candidate matches with both
> getbitmap and getnext. A simple flag per page of results would be enough
> for getbitmap, I think.
>
> - StreamBitmap and HashBitmap are separate node types, but OpStream is
> not. opaque-field in the StreamBitmap struct is not really that opaque,
> it needs to be a StreamNode. I drew a UML sketch of what I think the
> class-hierarchy is
> (http://community.enterprisedb.com/streambitmaps.png). This is
> object-oriented programming, we're just implementing classes and
> inheritance with structs and function pointers. The current patch mixes
> different techniques, and that needs to be cleaned up.

All good ideas, I'll look at them more closely in the morning.

> I'd like to see a separate patch that contains just the API changes.
> Gavin, could you extract an API-only patch from the bitmap index patch?
> I can work on it as well, but I don't want to step on your toes.

Okay, I can do that.

Thanks,

Gavin

Hi Gavin,

Any progress?

Gavin Sherry wrote:
> Heikki,
>
> On Mon, 5 Mar 2007, Heikki Linnakangas wrote:
>
>> Hi all,
>>
>> I'd like to see the indexam API changes needed by the bitmap indexam to
>> be committed soon. Has anyone looked at the proposed API in the latest
>> patch? Any thoughts?
>
> Thanks for looking at it!
>
>> I'm quite happy with it myself, with a few reservations:
>>
>> - All the getbitmap implementations except the new bitmap indexam are
>> just boilerplate. How about making getbitmap-function optional, and
>> having a generic implementation that fills in a hash bitmap using the
>> traditional getnext function?
>>
>> - getbitmap is passed an existing bitmap as argument, and the
>> implementation needs to OR the existing bitmap with new tuples. How
>> about AND? An indexam could be smart about ANDing with an existing
>> bitmap, for example skipping to the first set bit in the existing bitmap
>> and starting the scan from there.
>>
>> - I'd like to have support to return candidate matches with both
>> getbitmap and getnext. A simple flag per page of results would be enough
>> for getbitmap, I think.
>>
>> - StreamBitmap and HashBitmap are separate node types, but OpStream is
>> not. opaque-field in the StreamBitmap struct is not really that opaque,
>> it needs to be a StreamNode. I drew a UML sketch of what I think the
>> class-hierarchy is
>> (http://community.enterprisedb.com/streambitmaps.png). This is
>> object-oriented programming, we're just implementing classes and
>> inheritance with structs and function pointers. The current patch mixes
>> different techniques, and that needs to be cleaned up.
>
> All good ideas, I'll look at them more closely in the morning.
>
>> I'd like to see a separate patch that contains just the API changes.
>> Gavin, could you extract an API-only patch from the bitmap index patch?
>> I can work on it as well, but I don't want to step on your toes.
>
> Okay, I can do that.
>
> Thanks,
>
> Gavin

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Thu, 8 Mar 2007, Heikki Linnakangas wrote:

> Hi Gavin,
>
> Any progress?
>

Really busy at the moment, but it's on my TODO list for today.

Thanks,

Gavin

Heikki, are you going to submit this for 8.4?

---------------------------------------------------------------------------

Heikki Linnakangas wrote:
> Hi all,
>
> I'd like to see the indexam API changes needed by the bitmap indexam to
> be committed soon. Has anyone looked at the proposed API in the latest
> patch? Any thoughts?
>
> I'm quite happy with it myself, with a few reservations:
>
> - All the getbitmap implementations except the new bitmap indexam are
> just boilerplate. How about making getbitmap-function optional, and
> having a generic implementation that fills in a hash bitmap using the
> traditional getnext function?
>
> - getbitmap is passed an existing bitmap as argument, and the
> implementation needs to OR the existing bitmap with new tuples. How
> about AND? An indexam could be smart about ANDing with an existing
> bitmap, for example skipping to the first set bit in the existing bitmap
> and starting the scan from there.
>
> - I'd like to have support to return candidate matches with both
> getbitmap and getnext. A simple flag per page of results would be enough
> for getbitmap, I think.
>
> - StreamBitmap and HashBitmap are separate node types, but OpStream is
> not. opaque-field in the StreamBitmap struct is not really that opaque,
> it needs to be a StreamNode. I drew a UML sketch of what I think the
> class-hierarchy is
> (http://community.enterprisedb.com/streambitmaps.png). This is
> object-oriented programming, we're just implementing classes and
> inheritance with structs and function pointers. The current patch mixes
> different techniques, and that needs to be cleaned up.
>
> I'd like to see a separate patch that contains just the API changes.
> Gavin, could you extract an API-only patch from the bitmap index patch?
> I can work on it as well, but I don't want to step on your toes.
>
> --
> Heikki Linnakangas
> EnterpriseDB http://www.enterprisedb.com
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +