Re: assessing parallel-safety

On 20 March 2015 at 13:55, Thom Brown <thom(at)linux(dot)com> wrote:
> On 20 March 2015 at 13:16, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> wrote:
>> Thom Brown wrote:
>>> On 18 March 2015 at 16:01, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> >> On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch <noah(at)leadboat(dot)com> wrote:
>>> >>> Neither that rule, nor its variant downthread, would hurt operator authors too
>>> >>> much. To make the planner categorically parallel-safe, though, means limiting
>>> >>> evaluate_function() to parallel-safe functions. That would dramatically slow
>>> >>> selected queries. It's enough for the PL scenario if planning a parallel-safe
>>> >>> query is itself parallel-safe. If the planner is parallel-unsafe when
>>> >>> planning a parallel-unsafe query, what would suffer?
>>> >>
>>> >> Good point. So I guess the rule can be that planning a parallel-safe
>>> >> query should be parallel-safe. From there, it follows that estimators
>>> >> for a parallel-safe operator must also be parallel-safe. Which seems
>>> >> fine.
>>> >
>>> > More work is needed here, but for now, here is a rebased patch, per
>>> > Amit's request.
>>>
>>> This no longer applies due to changes in commit
>>> 13dbc7a824b3f905904cab51840d37f31a07a9ef.
>>
>> You should be able to drop the pg_proc.h changes and run the supplied
>> perl program. (I'm not sure that sending the patched pg_proc.h together
>> with this patch is all that useful, really.)
>
> Thanks. All patches applied and building okay.

Okay, breakage experienced, but not sure which thread this belongs on.

createdb pgbench
pgbench -i -s 200 pgbench

CREATE TABLE pgbench_accounts_1 (CHECK (bid = 1)) INHERITS (pgbench_accounts);
...
CREATE TABLE pgbench_accounts_200 (CHECK (bid = 200)) INHERITS
(pgbench_accounts);

WITH del AS (DELETE FROM pgbench_accounts WHERE bid = 1 RETURNING *)
INSERT INTO pgbench_accounts_1 SELECT * FROM del;
...
WITH del AS (DELETE FROM pgbench_accounts WHERE bid = 200 RETURNING *)
INSERT INTO pgbench_accounts_200 SELECT * FROM del;

VACUUM ANALYSE;

# SELECT name, setting FROM pg_settings WHERE name IN
('parallel_seqscan_degree','max_worker_processes','seq_page_cost');
name | setting
-------------------------+---------
max_worker_processes | 20
parallel_seqscan_degree | 8
seq_page_cost | 1000
(3 rows)

# EXPLAIN SELECT DISTINCT bid FROM pgbench_accounts;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

Log file:

2015-03-20 14:19:30 GMT [4285]: [10-1]
user=thom,db=pgbench,client=[local] DEBUG: StartTransactionCommand
2015-03-20 14:19:30 GMT [4285]: [11-1]
user=thom,db=pgbench,client=[local] DEBUG: StartTransaction
2015-03-20 14:19:30 GMT [4285]: [12-1]
user=thom,db=pgbench,client=[local] DEBUG: name: unnamed; blockState:
DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0, nestl
vl: 1, children:
2015-03-20 14:19:30 GMT [4285]: [13-1]
user=thom,db=pgbench,client=[local] DEBUG: ProcessUtility
2015-03-20 14:19:30 GMT [4285]: [14-1]
user=thom,db=pgbench,client=[local] DEBUG: rehashing catalog cache id
45 for pg_class; 257 tups, 128 buckets
2015-03-20 14:19:30 GMT [4285]: [15-1]
user=thom,db=pgbench,client=[local] DEBUG: rehashing catalog cache id
47 for pg_statistic; 257 tups, 128 buckets
2015-03-20 14:19:30 GMT [4285]: [16-1]
user=thom,db=pgbench,client=[local] DEBUG: rehashing catalog cache id
47 for pg_statistic; 513 tups, 256 buckets
2015-03-20 14:19:30 GMT [4285]: [17-1]
user=thom,db=pgbench,client=[local] DEBUG: rehashing catalog cache id
47 for pg_statistic; 1025 tups, 512 buckets
2015-03-20 14:19:31 GMT [4273]: [76-1] user=,db=,client= DEBUG:
forked new backend, pid=4286 socket=10
2015-03-20 14:19:31 GMT [4286]: [1-1]
user=thom,db=pgbench,client=[local] DEBUG: postgres child[4286]:
starting with (
2015-03-20 14:19:31 GMT [4273]: [77-1] user=,db=,client= DEBUG:
reaping dead processes
2015-03-20 14:19:31 GMT [4273]: [78-1] user=,db=,client= DEBUG:
server process (PID 4285) was terminated by signal 11: Segmentation
fault
2015-03-20 14:19:31 GMT [4273]: [79-1] user=,db=,client= DETAIL:
Failed process was running: EXPLAIN SELECT DISTINCT bid FROM
pgbench_accounts;
2015-03-20 14:19:31 GMT [4273]: [80-1] user=,db=,client= LOG: server
process (PID 4285) was terminated by signal 11: Segmentation fault
2015-03-20 14:19:31 GMT [4273]: [81-1] user=,db=,client= DETAIL:
Failed process was running: EXPLAIN SELECT DISTINCT bid FROM
pgbench_accounts;
2015-03-20 14:19:31 GMT [4273]: [82-1] user=,db=,client= LOG:
terminating any other active server processes
2015-03-20 14:19:31 GMT [4273]: [83-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4286
2015-03-20 14:19:31 GMT [4273]: [84-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4279
2015-03-20 14:19:31 GMT [4286]: [2-1]
user=thom,db=pgbench,client=[local] DEBUG: postgres
2015-03-20 14:19:31 GMT [4273]: [85-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4277
2015-03-20 14:19:31 GMT [4286]: [3-1]
user=thom,db=pgbench,client=[local] DEBUG: )
2015-03-20 14:19:31 GMT [4273]: [86-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4280
2015-03-20 14:19:31 GMT [4273]: [87-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4281
2015-03-20 14:19:31 GMT [4273]: [88-1] user=,db=,client= DEBUG:
sending SIGQUIT to process 4282
2015-03-20 14:19:31 GMT [4286]: [4-1]
user=thom,db=pgbench,client=[local] WARNING: terminating connection
because of crash of another server process
2015-03-20 14:19:31 GMT [4286]: [5-1]
user=thom,db=pgbench,client=[local] DETAIL: The postmaster has
commanded this server process to roll back the current transaction
and exit, because another server process exited abnormally and
possibly corrupted shared memory.
2015-03-20 14:19:31 GMT [4286]: [6-1]
user=thom,db=pgbench,client=[local] HINT: In a moment you should be
able to reconnect to the database and repeat your command.
2015-03-20 14:19:31 GMT [4277]: [2-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4277]: [3-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4277]: [4-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4279]: [1-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4279]: [2-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4279]: [3-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4286]: [7-1]
user=thom,db=pgbench,client=[local] DEBUG: shmem_exit(-1): 0
before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4286]: [8-1]
user=thom,db=pgbench,client=[local] DEBUG: shmem_exit(-1): 0
on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4286]: [9-1]
user=thom,db=pgbench,client=[local] DEBUG: proc_exit(-1): 0 callbacks
to make
2015-03-20 14:19:31 GMT [4280]: [1-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4280]: [2-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4280]: [3-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4281]: [12-1] user=,db=,client= WARNING:
terminating connection because of crash of another server process
2015-03-20 14:19:31 GMT [4281]: [13-1] user=,db=,client= DETAIL: The
postmaster has commanded this server process to roll back the current
transaction and exit, because another server process exited abnormally
and possibly corrupted shared memory.
2015-03-20 14:19:31 GMT [4281]: [14-1] user=,db=,client= HINT: In a
moment you should be able to reconnect to the database and repeat your
command.
2015-03-20 14:19:31 GMT [4281]: [15-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4281]: [16-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4281]: [17-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4282]: [4-1] user=,db=,client= DEBUG:
writing stats file "pg_stat/global.stat"
2015-03-20 14:19:31 GMT [4282]: [5-1] user=,db=,client= DEBUG:
writing stats file "pg_stat/db_16384.stat"
2015-03-20 14:19:31 GMT [4282]: [6-1] user=,db=,client= DEBUG:
removing temporary stats file "pg_stat_tmp/db_16384.stat"
2015-03-20 14:19:31 GMT [4282]: [7-1] user=,db=,client= DEBUG:
writing stats file "pg_stat/db_0.stat"
2015-03-20 14:19:31 GMT [4282]: [8-1] user=,db=,client= DEBUG:
removing temporary stats file "pg_stat_tmp/db_0.stat"
2015-03-20 14:19:31 GMT [4282]: [9-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4282]: [10-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4282]: [11-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4273]: [89-1] user=,db=,client= DEBUG:
reaping dead processes
2015-03-20 14:19:31 GMT [4273]: [90-1] user=,db=,client= DEBUG:
server process (PID 4286) exited with exit code 2
2015-03-20 14:19:31 GMT [4273]: [91-1] user=,db=,client= DEBUG:
reaping dead processes
2015-03-20 14:19:31 GMT [4273]: [92-1] user=,db=,client= LOG: all
server processes terminated; reinitializing
2015-03-20 14:19:31 GMT [4273]: [93-1] user=,db=,client= DEBUG:
shmem_exit(1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4273]: [94-1] user=,db=,client= DEBUG:
shmem_exit(1): 4 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4273]: [95-1] user=,db=,client= DEBUG:
cleaning up dynamic shared memory control segment with ID 1804289383
2015-03-20 14:19:31 GMT [4273]: [96-1] user=,db=,client= DEBUG:
invoking IpcMemoryCreate(size=4211515392)
2015-03-20 14:19:31 GMT [4273]: [97-1] user=,db=,client= DEBUG: mmap
with MAP_HUGETLB failed, huge pages disabled: Cannot allocate memory
2015-03-20 14:19:31 GMT [4273]: [98-1] user=,db=,client= DEBUG:
SlruScanDirectory invoking callback on pg_notify/0000
2015-03-20 14:19:31 GMT [4273]: [99-1] user=,db=,client= DEBUG:
removing file "pg_notify/0000"
2015-03-20 14:19:31 GMT [4273]: [100-1] user=,db=,client= DEBUG:
dynamic shared memory system will support 312 segments
2015-03-20 14:19:31 GMT [4273]: [101-1] user=,db=,client= DEBUG:
created dynamic shared memory control segment 1478456815 (2508 bytes)
2015-03-20 14:19:31 GMT [4287]: [1-1] user=,db=,client= LOG: database
system was interrupted; last known up at 2015-03-20 14:19:19 GMT
2015-03-20 14:19:31 GMT [4287]: [2-1] user=,db=,client= DEBUG:
checkpoint record is at 1/C4354BA0
2015-03-20 14:19:31 GMT [4287]: [3-1] user=,db=,client= DEBUG: redo
record is at 1/C4354BA0; shutdown TRUE
2015-03-20 14:19:31 GMT [4287]: [4-1] user=,db=,client= DEBUG: next
transaction ID: 0/2046; next OID: 24576
2015-03-20 14:19:31 GMT [4287]: [5-1] user=,db=,client= DEBUG: next
MultiXactId: 1; next MultiXactOffset: 0
2015-03-20 14:19:31 GMT [4287]: [6-1] user=,db=,client= DEBUG: oldest
unfrozen transaction ID: 676, in database 1
2015-03-20 14:19:31 GMT [4287]: [7-1] user=,db=,client= DEBUG: oldest
MultiXactId: 1, in database 1
2015-03-20 14:19:31 GMT [4287]: [8-1] user=,db=,client= DEBUG: commit
timestamp Xid oldest/newest: 0/0
2015-03-20 14:19:31 GMT [4287]: [9-1] user=,db=,client= DEBUG:
transaction ID wrap limit is 2147484323, limited by database with OID
1
2015-03-20 14:19:31 GMT [4287]: [10-1] user=,db=,client= DEBUG:
MultiXactId wrap limit is 2147483648, limited by database with OID 1
2015-03-20 14:19:31 GMT [4287]: [11-1] user=,db=,client= DEBUG:
starting up replication slots
2015-03-20 14:19:31 GMT [4287]: [12-1] user=,db=,client= LOG:
database system was not properly shut down; automatic recovery in
progress
2015-03-20 14:19:31 GMT [4287]: [13-1] user=,db=,client= DEBUG:
resetting unlogged relations: cleanup 1 init 0
2015-03-20 14:19:31 GMT [4287]: [14-1] user=,db=,client= LOG: invalid
record length at 1/C4354C10
2015-03-20 14:19:31 GMT [4287]: [15-1] user=,db=,client= LOG: redo is
not required
2015-03-20 14:19:31 GMT [4287]: [16-1] user=,db=,client= DEBUG:
resetting unlogged relations: cleanup 0 init 1
2015-03-20 14:19:31 GMT [4287]: [17-1] user=,db=,client= DEBUG:
performing replication slot checkpoint
2015-03-20 14:19:31 GMT [4287]: [18-1] user=,db=,client= DEBUG:
attempting to remove WAL segments older than log file
0000000000000001000000C3
2015-03-20 14:19:31 GMT [4287]: [19-1] user=,db=,client= DEBUG:
SlruScanDirectory invoking callback on pg_multixact/offsets/0000
2015-03-20 14:19:31 GMT [4287]: [20-1] user=,db=,client= DEBUG:
SlruScanDirectory invoking callback on pg_multixact/members/0000
2015-03-20 14:19:31 GMT [4287]: [21-1] user=,db=,client= DEBUG:
SlruScanDirectory invoking callback on pg_multixact/offsets/0000
2015-03-20 14:19:31 GMT [4287]: [22-1] user=,db=,client= DEBUG:
shmem_exit(0): 1 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4287]: [23-1] user=,db=,client= DEBUG:
shmem_exit(0): 3 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4287]: [24-1] user=,db=,client= DEBUG:
proc_exit(0): 2 callbacks to make
2015-03-20 14:19:31 GMT [4287]: [25-1] user=,db=,client= DEBUG: exit(0)
2015-03-20 14:19:31 GMT [4287]: [26-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 before_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4287]: [27-1] user=,db=,client= DEBUG:
shmem_exit(-1): 0 on_shmem_exit callbacks to make
2015-03-20 14:19:31 GMT [4287]: [28-1] user=,db=,client= DEBUG:
proc_exit(-1): 0 callbacks to make
2015-03-20 14:19:31 GMT [4273]: [102-1] user=,db=,client= DEBUG:
reaping dead processes
2015-03-20 14:19:31 GMT [4288]: [1-1] user=,db=,client= DEBUG:
checkpointer updated shared memory configuration values
2015-03-20 14:19:31 GMT [4273]: [103-1] user=,db=,client= LOG:
database system is ready to accept connections

--
Thom