Re: JVM crash when select count(*) on large table through JDBC

Hello,

I have a curious problem, which I would like to know whether somebody
can reproduce this: From a JVM using JDBC, I issue

select count(*) from big_table

where big table has more than 20 million entries (timestamps and
numbers). Upon PreparedStatement.executeQuery(), the VM quits, without
any message.

Java VM: 1.4.2-16 (latest one)
Pg-Jdbc: JDBC3 Postgresql Driver, Version 8.3-603
Server: PostgreSQL 8.1.11 (on Debian etch)

The most surprising thing is the lack of any error message (I had
expected a TimeOutException, or OutOfMemoryError, or the like), as if
someone had called System.exit().

The same commando, when executed through pgAdmin III, works.

As suggested on the website (http://jdbc.postgresql.org/lists.html), I
have searched the mailing lists, to no avail so far.

I would highly appreciate any reproduction of these results, or
suggestions for remedy.

Kind regards,

Kai Ruhl

On 22-Feb-08, at 11:25 AM, Kai Ruhl wrote:

> Hello,
>
> I have a curious problem, which I would like to know whether somebody
> can reproduce this: From a JVM using JDBC, I issue
>
> select count(*) from big_table
>
> where big table has more than 20 million entries (timestamps and
> numbers). Upon PreparedStatement.executeQuery(), the VM quits, without
> any message.
>
It's my understanding that regardless of what a java program is doing
(aside from calling exit, or jni code) , it should never crash.

I would say there is a bug in your jvm.

Just did a quick test with synthetic data (20M rows 3 columns: serial,
timestamp, and text). And it worked ok both on Java (with driver
postgresql-8.1-407.jdbc3.jar) and PgAdmin.

Perhaps you could try to build a test case to expose the problem?

On Fri, Feb 22, 2008 at 1:30 PM, Dave Cramer <pg(at)fastcrypt(dot)com> wrote:
>
> On 22-Feb-08, at 11:25 AM, Kai Ruhl wrote:
>
> > Hello,
> >
> > I have a curious problem, which I would like to know whether somebody
> > can reproduce this: From a JVM using JDBC, I issue
> >
> > select count(*) from big_table
> >
> > where big table has more than 20 million entries (timestamps and
> > numbers). Upon PreparedStatement.executeQuery(), the VM quits, without
> > any message.
> >
> It's my understanding that regardless of what a java program is doing
> (aside from calling exit, or jni code) , it should never crash.
>
> I would say there is a bug in your jvm.
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

On Fri, 22 Feb 2008, Andres Olarte wrote:

> Just did a quick test with synthetic data (20M rows 3 columns: serial,
> timestamp, and text). And it worked ok both on Java (with driver
> postgresql-8.1-407.jdbc3.jar) and PgAdmin.
>
> Perhaps you could try to build a test case to expose the problem?

From a client perspective, select count(*) should be the same no matter
how many rows there are and what types of columns are in the table, so a
test case won't be helpful. If the JVM is just up and dying, that's the
JVM's fault not a pg issue.

Kris Jurka

I know it's the same to count 1 o 20M rows, but it takes much longer
to count 20M. I really don't know much about the internal of the
driver, but maybe some kind of time out?

On Fri, Feb 22, 2008 at 2:33 PM, Kris Jurka <books(at)ejurka(dot)com> wrote:
>
> On Fri, 22 Feb 2008, Andres Olarte wrote:
>
> > Just did a quick test with synthetic data (20M rows 3 columns: serial,
> > timestamp, and text). And it worked ok both on Java (with driver
> > postgresql-8.1-407.jdbc3.jar) and PgAdmin.
> >
> > Perhaps you could try to build a test case to expose the problem?
>
> From a client perspective, select count(*) should be the same no matter
> how many rows there are and what types of columns are in the table, so a
> test case won't be helpful. If the JVM is just up and dying, that's the
> JVM's fault not a pg issue.
>
> Kris Jurka
>

As Kris said and I said. If your jvm crashes it's a jvm problem.

I know people that have queries that take 20 minutes or more...

Dave
On 22-Feb-08, at 3:39 PM, Andres Olarte wrote:

> I know it's the same to count 1 o 20M rows, but it takes much longer
> to count 20M. I really don't know much about the internal of the
> driver, but maybe some kind of time out?
>
> On Fri, Feb 22, 2008 at 2:33 PM, Kris Jurka <books(at)ejurka(dot)com> wrote:
>>
>> On Fri, 22 Feb 2008, Andres Olarte wrote:
>>
>>> Just did a quick test with synthetic data (20M rows 3 columns:
>>> serial,
>>> timestamp, and text). And it worked ok both on Java (with driver
>>> postgresql-8.1-407.jdbc3.jar) and PgAdmin.
>>>
>>> Perhaps you could try to build a test case to expose the problem?
>>
>> From a client perspective, select count(*) should be the same no
>> matter
>> how many rows there are and what types of columns are in the table,
>> so a
>> test case won't be helpful. If the JVM is just up and dying,
>> that's the
>> JVM's fault not a pg issue.
>>
>> Kris Jurka
>>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate