Re: Crappy performance even though not swapping

[Richard Broersma Jr <rabroersma(at)yahoo(dot)com>]

> I have a Java program that is trying to import over 8 million 
> rows from a text file.
> 
> When the program starts up, it runs fine, but after a while
> it begins to crawl.
> 
> This is usually symptomatic of a machine that started swapping,
> but take a look at the attached output from top.
> 
> I am using a LOT of memory, but the swap is still very 
> low.  I don't think the machine should be paging to disk.
> 
> The CPU states show that the cpus are spending most of their time 
> in an idle state.  The process list shows the postmaster and java 
> processes are less than 10 percent the CPU cycles.
> 
> Having said that, the load average is still about 2.
> 
> Any ideas what could be going on?

Your question is a good one for the performance list.  From what I've read there, there are a
couple of strategies that you can use to improve performance.

1) if you can roll large numbers of insert statements (say 1K to 10K ins/trans) in to single
transactions, performance should be helped.

2) If this is a one time insert, you may consider TEMPORARILY turning off fsync and greatly
increase your check point segment size for the import.

Regards,
Richard Broersma Jr.