Re: Configuration Advice

[Steve <cheetah(at)tanabi(dot)org>]

> The thought:
>
> - Load the big chunk of data into a new table
>
> - Generate some minimal set of indices on the new table
>
> - Generate four queries that compare old to new:
>  q1 - See which tuples are unchanged from yesterday to today
>  q2  - See which tuples have been deleted from yesterday to today
>  q3  - See which tuples have been added
>  q4  - See which tuples have been modified
>
> If the "unchanged" set is extremely large, then you might see benefit
> to doing updates based on deleting the rows indicated by q2,
> inserting rows based on q3, and updating based on q4.
>
> In principle, computing and applying those 4 queries might be quicker
> than rebuilding from scratch.
>
> In principle, applying q2, then q4, then vacuuming, then q3, ought to
> be "optimal."


	This looks like an interesting idea, and I'm going to take a look 
at how feasible it'll be to impletement.  I may be able to combine this 
with Mr. Wagner's idea to make a much more efficient system overall.  It's 
going to be a  pretty big programming task, but I've a feeling this 
summarizer thing may just need to be re-written with a smarter system 
like this to get something faster.


Thanks!

Steve