Re: Memory management question

It's probably a pretty basic question explained in some document I haven't seen
but...if I do something like a CreateTupleDescCopy() how do I know my memory
context owns everything allocated without following the code all the way
through until it returns to me?

--
Nigel J. Andrews

On Tue, Sep 03, 2002 at 12:28:37PM +0100, Nigel J. Andrews wrote:
>
>
> It's probably a pretty basic question explained in some document I haven't seen
> but...if I do something like a CreateTupleDescCopy() how do I know my memory
> context owns everything allocated without following the code all the way
> through until it returns to me?

If some code doesn't call MemoryContextSwitchTo() all is allocated in
current memory context. You can check if CurrentMemoryContext is same
before and after call that is important for you - but this check say
nothing, bacuse some code can switch to other context and after usage
switch back to your context. IMHO is not common way how check it.
(Ok, maybe check all contexts size before/after call...)

Suggestion: add to memory managment counter that handle number
of MemoryContextSwitchTo() calls. IMHO it can be compile
only if MEMORY_CONTEXT_CHECKING is define.

But I think there is not to much places which switching between
contexts and all are good commented (I hope, I wish :-)

Karel

--
Karel Zak <zakkr(at)zf(dot)jcu(dot)cz>
http://home.zf.jcu.cz/~zakkr/

C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz

On Tue, 3 Sep 2002, Nigel J. Andrews wrote:

>
>
> It's probably a pretty basic question explained in some document I haven't seen
> but...if I do something like a CreateTupleDescCopy() how do I know my memory
> context owns everything allocated without following the code all the way
> through until it returns to me?

Umm.. how else could you *really* know unless you read the
source? Basically, all convenience routines off this nature allow memory
in the current memory context.

As for CreateTupleDescCopy() you don't have to look too far to see what it
does:

--

CreateTupleDescCopy(TupleDesc tupdesc)
{
TupleDesc desc;
int i,
size;

desc = (TupleDesc) palloc(sizeof(struct tupleDesc));

--

Gavin

On Tue, 3 Sep 2002, Karel Zak wrote:

> On Tue, Sep 03, 2002 at 12:28:37PM +0100, Nigel J. Andrews wrote:
> >
> >
> > It's probably a pretty basic question explained in some document I haven't seen
> > but...if I do something like a CreateTupleDescCopy() how do I know my memory
> > context owns everything allocated without following the code all the way
> > through until it returns to me?
>
> If some code doesn't call MemoryContextSwitchTo() all is allocated in
> current memory context. You can check if CurrentMemoryContext is same
> before and after call that is important for you - but this check say
> nothing, bacuse some code can switch to other context and after usage
> switch back to your context. IMHO is not common way how check it.
> (Ok, maybe check all contexts size before/after call...)
>
> Suggestion: add to memory managment counter that handle number
> of MemoryContextSwitchTo() calls. IMHO it can be compile
> only if MEMORY_CONTEXT_CHECKING is define.

I quite like that idea. Only thing is it doesn't full address the issue of
identifying if my context owns memory allocated by other functions I've
used. For example:

A called procedure could be doing (psuedo code obviously):

SwitchContext()
mem=palloc(anumber)
/* use mem */
pfree(mem)
SwitchContectBack()
retmem=palloc(anothersize)

There, net effect is that I do own retmem but the test on context switch
counters would indicate that I may not.

I think the problem is that I don't fully understand why [and when] is context
switch is or should be done.

> But I think there is not to much places which switching between
> contexts and all are good commented (I hope, I wish :-)

As someone pointed out my example wasn't very complex so checking the source
wasn't onerous. Checking something like heap_modifytuple() is more time
consuming.

I was hoping there was some sort of 'rule of thumb'. In general I can't see how
it could be sensibly known without such a rule and without tracing through the
source.

--
Nigel J. Andrews

"Nigel J. Andrews" <nandrews(at)investsystems(dot)co(dot)uk> writes:
> It's probably a pretty basic question explained in some document I
> haven't seen but...if I do something like a CreateTupleDescCopy() how
> do I know my memory context owns everything allocated without
> following the code all the way through until it returns to me?

If it doesn't, then it's broken. A general rule of the system is that
structures being allocated for return to a routine's caller must be
allocated in the caller's CurrentMemoryContext. The only exceptions are
for cases where the routine in question is taking responsibility for the
long-term management of the object (for example, a syscache) --- in
which case, it isn't your problem.

regards, tom lane

Maybe when this thread is over, some parts of it can be
added to the dev. FAQ?

-s

On Tue, 3 Sep 2002, Karel Zak wrote:

> Date: Tue, 3 Sep 2002 13:52:09 +0200
> From: Karel Zak <zakkr(at)zf(dot)jcu(dot)cz>
> To: Nigel J. Andrews <nandrews(at)investsystems(dot)co(dot)uk>
> Cc: pgsql-hackers(at)postgresql(dot)org
> Subject: Re: [HACKERS] Memory management question
>
> On Tue, Sep 03, 2002 at 12:28:37PM +0100, Nigel J. Andrews wrote:
> >
> >
> > It's probably a pretty basic question explained in some document I haven't seen
> > but...if I do something like a CreateTupleDescCopy() how do I know my memory
> > context owns everything allocated without following the code all the way
> > through until it returns to me?
>
> If some code doesn't call MemoryContextSwitchTo() all is allocated in
> current memory context. You can check if CurrentMemoryContext is same
> before and after call that is important for you - but this check say
> nothing, bacuse some code can switch to other context and after usage
> switch back to your context. IMHO is not common way how check it.
> (Ok, maybe check all contexts size before/after call...)
>
> Suggestion: add to memory managment counter that handle number
> of MemoryContextSwitchTo() calls. IMHO it can be compile
> only if MEMORY_CONTEXT_CHECKING is define.
>
> But I think there is not to much places which switching between
> contexts and all are good commented (I hope, I wish :-)
>
> Karel

Nigel J. Andrews wrote:
>
> It's probably a pretty basic question explained in some document I haven't seen
> but...if I do something like a CreateTupleDescCopy() how do I know my memory
> context owns everything allocated without following the code all the way
> through until it returns to me?

I asked a related question recently. Here it is with Tom's response:

Tom Lane wrote:
> Joe Conway wrote:
>>Does a good primer on proper backend memory-context handling exist?
>
> The original design document is in src/backend/utils/mmgr/README;
> somebody needs to recast that into present tense and put it into the
> Developer's Guide SGML docs.
>
> If you read that and feel you understand it, next read
> executor/nodeAgg.c and see if you follow the memory management
> there...
> AFAIR that's the most complex use of short-term contexts in the
> system.

You might want to read through those to get a better understanding.

HTH,

Joe