Re: out of memory

Hello,

i have a statement which ends with
ERROR: out of memory
DETAIL: Failed on request of size 639.

i tried it several times, the number after size changes but not the outcome.

could someone tell me what information you need to tell me whats wrong?

what i have so fare is:
PostgreSQL version: 8.1.4
Linux version 2.6.16.20-0.12-bigsmp (geeko(at)buildhost) (gcc version
4.1.0 (SUSE Linux)) #1 SMP Mon Jun 19 22:25:45 UTC 2006

the statement:
SELECT catalog_prefix, supplier_aid
, stage_1.article_data_md5(
COALESCE(src.description_short ,'')
,
COALESCE(src.description_long ,'')
, COALESCE(src.ean
,'')
,
COALESCE(m2.manufacturer_name , src.manufacturer_name)
,
COALESCE(m1.manufacturer_name_old, src.manufacturer_name, '')
,
COALESCE(src.manufacturer_aid ,'')
) AS md5
, COALESCE(m2.manufacturer_name, src.manufacturer_name)
AS manufacturer_name
, COALESCE(m1.manufacturer_name_old,
src.manufacturer_name) AS manufacturer_name_old
, description_short, description_long, ean, manufacturer_aid
FROM quellen.article
src
LEFT JOIN ( SELECT SUBSTRING(file FROM
'^([a-zA-Z0-9]*)[.].*') AS catalog_prefix
, sku AS supplier_aid
, manufacturer_name_old
FROM quellen.manufacturer_names_old
) m1 USING (catalog_prefix, supplier_aid)
LEFT JOIN data.manufacturer_mapping
m2 ON (
UPPER(m2.manufacturer_name_old)=UPPER(COALESCE(m1.manufacturer_name_old,src.manufacturer_name)))
WHERE EXISTS ( SELECT 1
FROM stage_1.article_category_a
WHERE isactive IS TRUE
AND root_code=(SELECT
col_root_category_id FROM quellen.cataloginfo WHERE catalog_prefix =
'root')
AND catalog_prefix=src.catalog_prefix
AND supplier_aid=src.supplier_aid
)

and the explain plan:
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
Merge Right Join (cost=17894565.16..18024911.70 rows=5792735 width=496)
Merge Cond: ("outer"."?column3?" = "inner"."?column9?")
-> Sort (cost=62.33..64.83 rows=1000 width=64)
Sort Key: upper(version.manufacturer_name_old)
-> Function Scan on dblink version (cost=0.00..12.50
rows=1000 width=64)
-> Sort (cost=17894502.83..17897399.20 rows=1158547 width=464)
Sort Key:
upper(COALESCE(manufacturer_names_old.manufacturer_name_old,
src.manufacturer_name))
-> Merge Left Join (cost=17752829.62..17777814.64
rows=1158547 width=464)
Merge Cond: (("outer".supplier_aid = "inner".sku) AND
("outer".catalog_prefix = "inner"."?column4?"))
-> Sort (cost=17587009.86..17589906.23 rows=1158547 width=448)
Sort Key: src.supplier_aid, src.catalog_prefix
-> Seq Scan on article src
(cost=0.00..17470321.68 rows=1158547 width=448)
Filter: (subplan)
SubPlan
-> Index Scan using
article_category_a_pkey on article_category_a (cost=3.44..7.46 rows=1
width=0)
Index Cond: ((catalog_prefix = $1)
AND (supplier_aid = $2) AND (root_code = $0))
Filter: (isactive IS TRUE)
InitPlan
-> Index Scan using
cataloginfo_pkey on cataloginfo (cost=0.00..3.44 rows=1 width=5)
Index Cond: (catalog_prefix
= 'root'::text)
-> Sort (cost=165819.76..169160.88 rows=1336447 width=63)
Sort Key: manufacturer_names_old.sku,
"substring"(manufacturer_names_old.file, '^([a-zA-Z0-9]*)[.].*'::text)
-> Seq Scan on manufacturer_names_old
(cost=0.00..29836.47 rows=1336447 width=63)

greetings, anita

"Anita Lederer" <anita(dot)lederer(at)googlemail(dot)com> writes:
> i have a statement which ends with
> ERROR: out of memory
> DETAIL: Failed on request of size 639.

Do you have work_mem set to a large value? Your query plan contains
several sorts so would potentially try to use several times work_mem
... if that's more than the kernel will allow one process to have,
you would get a failure like this.

The other theory that comes to mind is that you've discovered an
intraquery memory-leak bug, but there's not nearly enough information
here to find it if so.

The out-of-memory error should result in dumping a list of memory
context statistics into the postmaster's log file --- if you can post
that information it would be helpful.

regards, tom lane

hi,

thx for the hint. i decreased work_mem. its working now.

the dump written in the logfile is:

TopMemoryContext: 40960 total in 5 blocks; 11064 free (37 chunks); 29896 used
TopTransactionContext: 8192 total in 1 blocks; 7856 free (0 chunks); 336 used
Remote Con hash: 8192 total in 1 blocks; 5968 free (0 chunks); 2224 used
Record information cache: 8192 total in 1 blocks; 1864 free (0
chunks); 6328 used
Type information cache: 8192 total in 1 blocks; 1864 free (0 chunks); 6328 used
CFuncHash: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used
MessageContext: 253952 total in 5 blocks; 115160 free (1199 chunks); 138792 used
Operator class cache: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used
smgr relation table: 8192 total in 1 blocks; 1840 free (0 chunks); 6352 used
Portal hash: 8192 total in 1 blocks; 3912 free (0 chunks); 4280 used
PortalMemory: 8192 total in 1 blocks; 8040 free (0 chunks); 152 used
PortalHeapMemory: 1024 total in 1 blocks; 856 free (0 chunks); 168 used
ExecutorState: -1211899784 total in 386 blocks; 40648 free (537
chunks); -1211940432 used
ExprContext: 8192 total in 1 blocks; 8176 free (0 chunks); 16 used
ExecutorState: 8192 total in 1 blocks; 2088 free (5 chunks); 6104 used
ExecutorState: 8192 total in 1 blocks; 5088 free (0 chunks); 3104 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 8192 total in 1 blocks; 8160 free (0 chunks); 32 used
ExprContext: 8192 total in 1 blocks; 8176 free (0 chunks); 16 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
ExprContext: 0 total in 0 blocks; 0 free (0 chunks); 0 used
Relcache by OID: 8192 total in 1 blocks; 3376 free (0 chunks); 4816 used
CacheMemoryContext: 1040384 total in 7 blocks; 552584 free (207
chunks); 487800 used
article_manufacturer_name_lower: 1024 total in 1 blocks; 392 free (0
chunks); 632 used
article_description_short_lower: 1024 total in 1 blocks; 392 free (0
chunks); 632 used
article_pkey: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
pg_toast_2619_index: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
article_data_a_ean: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
article_data_a_pkey: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
article_data_a_article_id_key: 1024 total in 1 blocks; 392 free (0
chunks); 632 used
article_category_a_pkey: 1024 total in 1 blocks; 192 free (0 chunks); 832 used
cataloginfo_pkey: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
catalog_prefix: 15360 total in 4 blocks; 7016 free (0 chunks); 8344 used
pg_constraint_conrelid_index: 1024 total in 1 blocks; 392 free (0
chunks); 632 used
pg_attrdef_adrelid_adnum_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
manufacturer_mapping: 7168 total in 3 blocks; 3912 free (0 chunks); 3256 used
pg_database_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_index_indrelid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_type_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_type_typname_nsp_index: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
pg_statistic_relid_att_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_rewrite_rel_rulename_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_class_relname_nsp_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_proc_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_proc_proname_args_nsp_index: 1024 total in 1 blocks; 256 free (0
chunks); 768 used
pg_operator_oprname_l_r_n_index: 1024 total in 1 blocks; 192 free (0
chunks); 832 used
pg_namespace_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_namespace_nspname_index: 1024 total in 1 blocks; 392 free (0
chunks); 632 used
pg_language_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_language_name_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_inherits_relid_seqno_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_conversion_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_conversion_name_nsp_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_conversion_default_index: 1024 total in 1 blocks; 192 free (0
chunks); 832 used
pg_opclass_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_opclass_am_name_nsp_index: 1024 total in 1 blocks; 256 free (0
chunks); 768 used
pg_cast_source_target_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_authid_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_auth_members_role_member_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_auth_members_member_role_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_attribute_relid_attnam_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_amop_opr_opc_index: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
pg_aggregate_fnoid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_database_datname_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_authid_rolname_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_trigger_tgrelid_tgname_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_operator_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_amproc_opc_proc_index: 1024 total in 1 blocks; 256 free (0 chunks); 768 used
pg_amop_opc_strat_index: 1024 total in 1 blocks; 256 free (0 chunks); 768 used
pg_index_indexrelid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_attribute_relid_attnum_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_class_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
MdSmgr: 8192 total in 1 blocks; 6448 free (0 chunks); 1744 used
LockTable (locallock hash): 8192 total in 1 blocks; 3912 free (0
chunks); 4280 used
Timezones: 47592 total in 2 blocks; 5968 free (0 chunks); 41624 used
ErrorContext: 16384 total in 2 blocks; 16352 free (10 chunks); 32 used

regards, anita