Re: Some 9.5beta2 backend processes not terminating properly?

From: Shay Rojansky <roji(at)roji(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Some 9.5beta2 backend processes not terminating properly?
Date: 2015-12-29 10:41:40
Message-ID: CADT4RqBo79_0Vx=-+y=nFv3zdnm_-CgGzbtSv9LhxrFEoYMVFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>
> > The tests run for a couple minutes, open and close some connection. With
> my
> > pre-9.5 backends, the moment the test runner exits I can see that all
> > backend processes exit immediately, and pg_activity_stat has no rows
> > (except the querying one). With 9.5beta2, however, some backend processes
> > continue to stay alive beyond the test runner, and pg_activity_stat
> > contains extra rows (state idle, waiting false). This situation persists
> > until I restart PostgreSQL.
>
> No idea what's happening, but a couple of questions:
>
> * Are you using SSL connections?
>
> * Can you get stack traces from the seemingly-stuck backends?
>

Most of my tests don't use SSL but some do. Looking at the query field in
pg_stat_activity I can see queries that don't seem to originate from SSL
tests.

Note that the number of backends that stay stuck after the tests is
constant (always 12).

Here's are stack dumps of the same process taken with both VS2015 Community
and Process Explorer, I went over 4 processes and saw the same thing. Let
me know what I else I can provide to help.

From VS2015 Community:

Main Thread
> ntdll.dll!NtWaitForMultipleObjects() Unknown
KernelBase.dll!WaitForMultipleObjectsEx() Unknown
KernelBase.dll!WaitForMultipleObjects() Unknown
postgres.exe!WaitLatchOrSocket(volatile Latch * latch, int wakeEvents,
unsigned __int64 sock, long timeout) Line 202 C
postgres.exe!secure_read(Port * port, void * ptr, unsigned __int64 len)
Line 151 C
postgres.exe!pq_getbyte() Line 926 C
postgres.exe!SocketBackend(StringInfoData * inBuf) Line 345 C
postgres.exe!PostgresMain(int argc, char * * argv, const char * dbname,
const char * username) Line 3984 C
postgres.exe!BackendRun(Port * port) Line 4236 C
postgres.exe!SubPostmasterMain(int argc, char * * argv) Line 4727 C
postgres.exe!main(int argc, char * * argv) Line 211 C
postgres.exe!__tmainCRTStartup() Line 626 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

Worker Thread
> ntdll.dll!NtWaitForWorkViaWorkerFactory() Unknown
ntdll.dll!TppWorkerThread() Unknown
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

Worker Thread
> ntdll.dll!NtFsControlFile() Unknown
KernelBase.dll!ConnectNamedPipe() Unknown
postgres.exe!pg_signal_thread(void * param) Line 279 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

Worker Thread
> ntdll.dll!NtWaitForSingleObject() Unknown
KernelBase.dll!WaitForSingleObjectEx() Unknown
postgres.exe!pg_timer_thread(void * param) Line 49 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

From Process Explorer (slightly different):

ntoskrnl.exe!KeSynchronizeExecution+0x3de6
ntoskrnl.exe!KeWaitForSingleObject+0xc7a
ntoskrnl.exe!KeWaitForSingleObject+0x709
ntoskrnl.exe!KeWaitForSingleObject+0x375
ntoskrnl.exe!IoQueueWorkItem+0x370
ntoskrnl.exe!KeRemoveQueueEx+0x16ba
ntoskrnl.exe!KeWaitForSingleObject+0xe8e
ntoskrnl.exe!KeWaitForSingleObject+0x709
ntoskrnl.exe!KeWaitForMultipleObjects+0x24e
ntoskrnl.exe!ObWaitForMultipleObjects+0x2bd
ntoskrnl.exe!IoWMIRegistrationControl+0x2402
ntoskrnl.exe!setjmpex+0x3943
ntdll.dll!NtWaitForMultipleObjects+0x14
KERNELBASE.dll!WaitForMultipleObjectsEx+0xef
KERNELBASE.dll!WaitForMultipleObjects+0xe
postgres.exe!WaitLatchOrSocket+0x243
postgres.exe!secure_read+0xb0
postgres.exe!pq_getbyte+0xec
postgres.exe!get_stats_option_name+0x392
postgres.exe!PostgresMain+0x537
postgres.exe!ShmemBackendArrayAllocation+0x2a6a
postgres.exe!SubPostmasterMain+0x273
postgres.exe!main+0x480
postgres.exe!pgwin32_popen+0x130b
KERNEL32.DLL!BaseThreadInitThunk+0x22
ntdll.dll!RtlUserThreadStart+0x34

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message amul sul 2015-12-29 11:22:52 commit 5c45d2f need to be back-patch on branch 9.2 & before
Previous Message Amit Kapila 2015-12-29 08:56:20 Re: [PATCH] Refactoring of LWLock tranches