Re: Sorted union

On Nov 3, 2005, at 10:21 AM, Merlin Moncure wrote:
> Reading the previous paragraphs I was just about to suggest this.
> This
> is a much more elegant method...you are reaping the benefits of having
> normalized your working set. You were trying to denormalize it
> back to
> what you were used to. Yes, now you can drop your index and simplify
> your queries...normalized data is always more 'natural'.

I'm not sure normalized is the right word. In either case, I'm
storing it in the same form. In either case, my ConcurrencyProcessor
class gets the same form. The only difference is if the database
splits the rows or if my application does so.

But we're essentially agreed. This is the algorithm I'm going to try
implementing, and I think it will work out well. It also means
sending about half as much data from the database to the application.

>> Mind you, I still think PostgreSQL should be able to perform that
>> sorted union fast. Maybe sometime I'll have enough free time to take
>> my first plunge into looking at a database query planner.
>
> I'm not so sure I agree, by using union you were basically pulling two
> independent sets (even if they were from the same table) that
> needed to
> be ordered.

Yes.

> There is zero chance of using the index here for ordering
> because you are ordering a different set than the one being indexed.

I don't think that's true. It just needs to look at the idea of
independently ordering each element of the union and then merging
that, compared to the cost of grabbing the union and then ordering
it. In this case, the former cost is about 0 - it already has
independently ordered them, and the merge algorithm is trivial.
<http://en.wikipedia.org/wiki/Merge_algorithm>

Regards,
Scott

--
Scott Lamb <http://www.slamb.org/>