Re: exit code -1073741819

> Whoa ... that is a whole lot more data than I'm used to seeing in
TopMemoryContext. How many stats dump lines are there exactly (from
here to the crash report)?

OK, I didn't know that was a surprise. There are about 600 stats dump lines
in between.

>> The spatial database that the script is using is quite large (about 4
>> GB). So I think making a self-contained test case would be the last
>> resort.

>FWIW, I doubt that the content of the database is the key point here;
you could probably generate a test case with relatively little data,
or maybe a lot of easily-created dummy data. However stripping it down
might require more insight into the nature of the bug than we have at
this point.

I did a test in a small area (which is the street network in a county) before and it worked without crashing the server. In that test there were about 600 records (or addresses) to be processed while in the current case there are about 12K records. Another difference is that the current case uses a larger base relation (the one I mentioned my previous email) that covers a whole state. I'm not sure whether it is the amount of records to be processed or the size of base relation that causes the crash.

_______________________________________________
Join Excite! - http://www.excite.com
The most personalized portal on the Web!

"Shuo Liu" <delphet(at)excite(dot)com> writes:
>> Whoa ... that is a whole lot more data than I'm used to seeing in
>> TopMemoryContext. How many stats dump lines are there exactly (from
>> here to the crash report)?

> OK, I didn't know that was a surprise. There are about 600 stats dump lines
> in between.

Hmm. 600 memory context records are not nearly enough to explain
11MB in TopMemoryContext ... but on the other hand I'm quite sure that
your process crashed before printing all the entries (there's a pretty
standard ordering to them and the last ones that should be there
aren't). So we're still left with more questions than answers.

>> FWIW, I doubt that the content of the database is the key point here;
>> you could probably generate a test case with relatively little data,
>> or maybe a lot of easily-created dummy data. However stripping it down
>> might require more insight into the nature of the bug than we have at
>> this point.

> I did a test in a small area (which is the street network in a county)
> before and it worked without crashing the server.

OK, so maybe it's dependent on the size of the table. Try generating a
test case by loading up just your schema + functions + a lot of dummy
entries generated by script.

Is the data proprietary? If not, maybe you could arrange to send me a
dump off-list. A short test-case script would be better, but let's not
waste huge amounts of person-time to save a few hours of downloading ...

regards, tom lane

"Shuo Liu" <delphet(at)excite(dot)com> writes:
>> Whoa ... that is a whole lot more data than I'm used to seeing in
>> TopMemoryContext. How many stats dump lines are there exactly (from
>> here to the crash report)?

> OK, I didn't know that was a surprise. There are about 600 stats dump lines
> in between.

Well, my suspicion was correct: there were boatloads of memory contexts,
but we weren't seeing all of them. The answer is twofold:

1. Your function is opening a cursor (maybe more than one, I didn't read
too carefully) on each iteration, and not closing it. This results in
a sizable amount of memory being eaten --- it looks like a couple
hundred K per cursor on my machine. Eventually you run out of memory.

2. PostGIS has a bug that causes a guaranteed crash once you run out of
memory, if you've been using transform(), because it sets up a memory
context that has no stats() method. That's why the context stats
printout failed partway through --- it crashed as soon as it came to
one of the PostGIS contexts.

I've complained to the PostGIS folk about the latter, and hopefully
they'll fix it in their next release. Meanwhile, you need to be more
careful about closing cursors when you're done with them.

regards, tom lane