Re: Huge Data sets, simple queries

[Charles Sprickman <spork(at)bway(dot)net>]

On Sun, 29 Jan 2006, Luke Lonergan wrote:

> In fact, in our testing of various host-based SCSI RAID adapters (LSI,
> Dell PERC, Adaptec, HP SmartArray), we find that *all* of them
> underperform, most of them severely.

[snip]

> The important lesson we've learned is to always test the I/O subsystem
> performance - you can do so with a simple test like:
>  time bash -c "dd if=/dev/zero of=bigfile bs=8k count=4000000 && sync"
>  time dd if=bigfile of=/dev/null bs=8k

I'm curious about this since we're shopping around for something new...  I 
do want to get some kind of baseline to compare new products to.  Areca 
sent me stats on their SCSI->SATA controller and it looks like it maxes 
out around 10,000 IOPS.

I'd like to see how our existing stuff compares to this.  I'd especially 
like to see it in graph form such as the docs Areca sent (IOPS on one 
axis, block size on the other, etc.).  Looking at the venerable Bonnie, it 
doesn't really seem to focus so much on the number of read/write 
operations per second, but on big bulky transfers.

What are you folks using to measure your arrays?

I've been considering using some of our data and just basically 
benchmarking postgres on various hardware with that, but I cannot compare 
that to any manufacturer tests.

Sorry to meander a bit off topic, but I've been getting frustrated with 
this little endeavour...

Thanks,

Charles

> - Luke
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faq