Re: Very slow queries w/ NOT IN preparation (seems like a bug, test case)

From: "Helio Campos Mello de Andrade" <helio(dot)campos(at)gmail(dot)com>
To: "Richard Huxton" <dev(at)archonet(dot)com>
Cc: "Sergey Konoplev" <gray(dot)ru(at)gmail(dot)com>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-general(at)postgresql(dot)org
Subject: Re: Very slow queries w/ NOT IN preparation (seems like a bug, test case)
Date: 2008-11-11 15:00:51
Message-ID: 29e3942f0811110700s1f25d77dqd9188c9575cfd920@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Hi Sergey,

- It's just guess but it could be the range of a SERIAL TYPE that is
generating this behavior.
An example is:
Knowing that "table1_id" is primary a key ( the table will be ordered
by it ) and that a serial range is 2147483647 long.

(a) you use 2000 different numbers of this range the planner will search for
the 2000 numbers in yours 5000 rows in an ordered way and it will stop when
the searched number can't be found any more. It will make, in the worst
case, 2000*log(5000) tests ( aprox: 24.575 ).
(b) If you use "~2000" different numbers and the system understands that you
want that the rest of the range ( 2147481647 numbers ) have to be searched
and it will cost 263.876.368.186 tests.

On Tue, Nov 11, 2008 at 12:20 PM, Richard Huxton <dev(at)archonet(dot)com> wrote:

> Sergey Konoplev wrote:
> >
> > Another thing is that even I set statement_timeout to 20s the query
> > with NOT IN finishes working after 30+ seconds without "canceled by
> > statement timeout" error.
>
> Maybe it's not taking that long to execute the query then.
>
> Maybe something to do with process startup is delaying things - could
> you tweak the test script to send the outputs of the explain somewhere
> other than /dev/null? That way we'd know if there was a big difference
> between query-execution-time and process-execution-time.
>
> --
> Richard Huxton
> Archonet Ltd
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

--
Helio Campos Mello de Andrade

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Joshua D. Drake 2008-11-11 15:10:00 Re: Web Application Engineer - Drupal, PHP, CSS, JavaScript, Postgresql
Previous Message NetGraviton 2008-11-11 14:45:52 Web Application Engineer - Drupal, PHP, CSS, JavaScript, Postgresql

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2008-11-11 15:04:40 Re: libpq with ssl vs psql without
Previous Message Alvaro Herrera 2008-11-11 14:29:34 Re: autovacuum and reloptions