Re: How to opimize the insertion of data

Lists: pgsql-novice
From: Juan Francisco Diaz <j-diaz(at)publicar(dot)com>
To: <pgsql-novice(at)postgresql(dot)org>
Subject: How to opimize the insertion of data
Date: 2003-09-09 21:05:54
Message-ID: BB83AAE2.4D0%j-diaz@publicar.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-novice

Hi, have tried by all means to optimize the insertion of data in my db but
it has been impossible.
Righto now to insert around 300 thou records it takes soemthing like 50 to
65 minutes (too long).
Im using a Mac powerpc g4 533Mhz with 256 RAM.
I would relly appreciate that the insertion process is done in like 30 or 35
minutes TOPS. So far it is impossible.
My db right now has no FK, no indexes, the insertions is being done in batch
(19 thou records).
Is it possible with my current machine to achieve the level of performance
i've metioned?
Any help would be greatly appreciated, by the way the same insertion takes
25 mins in ms sqlserver2000 in a p3 1.4ghz 1gig ram.
Thanks

JuanF


From: Alberto Caso <alberto(dot)caso(at)adaptia(dot)net>
To: Juan Francisco Diaz <j-diaz(at)publicar(dot)com>
Cc: pgsql-novice(at)postgresql(dot)org
Subject: Re: How to opimize the insertion of data
Date: 2003-09-10 06:03:07
Message-ID: 1063173786.2993.35.camel@localhost
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-novice

On Tue, 09-09-2003 at 23:05, Juan Francisco Diaz wrote:
> Hi, have tried by all means to optimize the insertion of data in my db but
> it has been impossible.
> Righto now to insert around 300 thou records it takes soemthing like 50 to
> 65 minutes (too long).

Try enclosing your inserts into transactions of several thousands of
inserts, i.e:

begin;
(several thousands of inserts)
commit;
...

You may also want to have a look at the COPY command:

http://developer.postgresql.org/docs/postgres/sql-copy.html

Have also a look at:

http://www.postgresql.org/docs/7.3/interactive/populate.html

Best regards,

--
Alberto Caso Palomino
Adaptia Soluciones Integrales
http://www.adaptia.net
alberto(dot)caso(at)adaptia(dot)net