Multi-line requests in COPY ... FROM STDIN

From: Mischa <mischa(dot)Sandberg(at)telus(dot)net>
To: pgsql-performance(at)postgresql(dot)org
Subject: Multi-line requests in COPY ... FROM STDIN
Date: 2005-03-10 05:29:47
Message-ID: 1110432587.422fdb4baaffd@webmail.telus.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I'm using a 7.4.6 Perl app that bulk-loads a table,
by executing a "COPY TMP_Message FROM STDIN", then letting
$dbh->func($message_text."\n", "putline")

Speculation made me try catenating Several \n-terminated lines
together, and making a single putline() call with that.
Lo and behold, all the lines went in as separate rows, as I hoped.

I haven't measured the performance difference using this multiline
batching. I'm hoping that there will be as much,since the app is really sucking
on a 500 msg/sec firehose, and the db side needs serious speeding up. Question
is, am I playing with a version-dependent anomaly, or should I expect this
to continue in 8.x (until, eventually, silently, something causes
this to break)?

I'm presuming that this is not a Perl DBI/DBD::Pg question,
but rather, depending on the underlying pq lib and fe protocol.

--
"Dreams come true, not free."

Browse pgsql-performance by date

  From Date Subject
Next Message Karim Nassar 2005-03-10 07:44:44 What's better: Raid 0 or disk for seperate pg_xlog
Previous Message Jim Johannsen 2005-03-10 02:09:20 Re: Help trying to tune query that executes 40x slower