Re: Regarding identifying a foreign scan

Hi,

I am trying to identify foreign scans uniquely.I am trying to do that
by struct ForeignScanState,but I am confused as to how I can identify
the scan.

Is there a member of ForeignScanState that can be used for this purpose?

Atri

--
Regards,

Atri
l'apprenant

Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
> I am trying to identify foreign scans uniquely.

What do you mean by "identify"? What are you trying to accomplish,
and in what context?

regards, tom lane

On Sat, Oct 6, 2012 at 3:45 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>> I am trying to identify foreign scans uniquely.
>
> What do you mean by "identify"? What are you trying to accomplish,
> and in what context?
>
> regards, tom lane

Hi Tom,

I am trying to identify the situation where a query has multiple
foreign scans.In that case,I need to check whether the current scan is
the same as a previous scan or not.If not,then I think it means that
multiple scans are in progress on the same backend.

Atri

--
Regards,

Atri
l'apprenant

Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
> On Sat, Oct 6, 2012 at 3:45 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>>> I am trying to identify foreign scans uniquely.

>> What do you mean by "identify"? What are you trying to accomplish,
>> and in what context?

> I am trying to identify the situation where a query has multiple
> foreign scans.In that case,I need to check whether the current scan is
> the same as a previous scan or not.If not,then I think it means that
> multiple scans are in progress on the same backend.

Well, if you search the plan tree and find more than one ForeignScan
node, it means there's more than one foreign scan. It doesn't seem to
me to be very complicated. Now, if you're wondering whether they
reference the same server or not, that's a bit harder. I guess you
could look at the RTEs, fetch the foreign-table data for each FT
relation OID, and see if the same server is referenced.

regards, tom lane

On Sat, Oct 6, 2012 at 4:05 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>> On Sat, Oct 6, 2012 at 3:45 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>>>> I am trying to identify foreign scans uniquely.
>
>>> What do you mean by "identify"? What are you trying to accomplish,
>>> and in what context?
>
>> I am trying to identify the situation where a query has multiple
>> foreign scans.In that case,I need to check whether the current scan is
>> the same as a previous scan or not.If not,then I think it means that
>> multiple scans are in progress on the same backend.
>
> Well, if you search the plan tree and find more than one ForeignScan
> node, it means there's more than one foreign scan. It doesn't seem to
> me to be very complicated. Now, if you're wondering whether they
> reference the same server or not, that's a bit harder. I guess you
> could look at the RTEs, fetch the foreign-table data for each FT
> relation OID, and see if the same server is referenced.
>
> regards, tom lane

Hi Tom,

Thanks for the extensive reply.

The issue I am trying to resolve is that if two scans are taking place
on the same backend(due to the same query),then,the server is
crashing.

e.g. foreign_table is a foreign table

SELECT * FROM foreign_table UNION SELECT * FROM foreign_table; results
in a crash of the server.

I think it is because I am not saving the state of the scan,so,if
multiple scans a re running on the same backend,then,it is causing the
crash.

Any hints on how I can detect this condition please?

Atri

--
Regards,

Atri
l'apprenant

Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
> The issue I am trying to resolve is that if two scans are taking place
> on the same backend(due to the same query),then,the server is
> crashing.

That sounds like an FDW bug ... which FDW are we talking about?

> I think it is because I am not saving the state of the scan,so,if
> multiple scans a re running on the same backend,then,it is causing the
> crash.

The FDW should certainly not assume that only one scan can happen at a
time. I would not think this would be a problem as long as you're using
reasonable coding techniques, like keeping scan-local state in
scan-local storage and not globally.

> Any hints on how I can detect this condition please?

If you are imagining that you'd do something differently depending on
whether the current query contains multiple ForeignScan nodes for your
FDW, that's still doomed to lose, because of nested queries. You really
need to manage storage in a way that doesn't make assumptions of this
sort.

regards, tom lane

On Sun, Oct 7, 2012 at 3:17 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>> The issue I am trying to resolve is that if two scans are taking place
>> on the same backend(due to the same query),then,the server is
>> crashing.
>
> That sounds like an FDW bug ... which FDW are we talking about?
>
>> I think it is because I am not saving the state of the scan,so,if
>> multiple scans a re running on the same backend,then,it is causing the
>> crash.
>
> The FDW should certainly not assume that only one scan can happen at a
> time. I would not think this would be a problem as long as you're using
> reasonable coding techniques, like keeping scan-local state in
> scan-local storage and not globally.
>
>> Any hints on how I can detect this condition please?
>
> If you are imagining that you'd do something differently depending on
> whether the current query contains multiple ForeignScan nodes for your
> FDW, that's still doomed to lose, because of nested queries. You really
> need to manage storage in a way that doesn't make assumptions of this
> sort.
>
> regards, tom lane

Hi Tom,

Thanks for the reply.

Does that mean that using (some) global storage is the cause of the problem?

Atri

--
Regards,

Atri
l'apprenant

Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
> Does that mean that using (some) global storage is the cause of the problem?

If you're using global storage for state that needs to be replicated
per-scan, yes, probably. But it's hard to be sure when you're being
so vague about what you're doing.

regards, tom lane

On Sun, Oct 7, 2012 at 10:29 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Atri Sharma <atri(dot)jiit(at)gmail(dot)com> writes:
>> Does that mean that using (some) global storage is the cause of the problem?
>
> If you're using global storage for state that needs to be replicated
> per-scan, yes, probably. But it's hard to be sure when you're being
> so vague about what you're doing.

yeah -- the problem here is that Atri is not using
ForeignScanState->fdw_state properly for storage (this in jdbc-fdw).
I think he knows what to do -- he's going to fix it up.

merlin