Re: ECPG support for struct in INTO list

Hi,

one of our clients wants to port their application suite
from Informix to PostgreSQL, they use constructs like

SELECT * INTO :tablerec FROM table ...

where "tablerec" mirrors the table fields in a C struct.

Currently ECPG dumps core on this, more exactly aborts on it
in ecpg_type_name().

Patch is attached that solves it by introducing add_struct_to_head()
called from rule "coutputvariable" that also catches C unions now,
emitting an error because unions cannot be unambiguously unrolled.
It tries to handle NULL indicator, expecting a struct with at least
the same amount of members.

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Fri, Jul 17, 2009 at 12:27:49PM +0200, Boszormenyi Zoltan wrote:
> one of our clients wants to port their application suite
> from Informix to PostgreSQL, they use constructs like
>
> SELECT * INTO :tablerec FROM table ...
>
> where "tablerec" mirrors the table fields in a C struct.

Well, this was supposed to work.

> Currently ECPG dumps core on this, more exactly aborts on it
> in ecpg_type_name().

Could you please send an example where it dumps core?

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Fri, Jul 17, 2009 at 12:27:49PM +0200, Boszormenyi Zoltan wrote:
>
>> one of our clients wants to port their application suite
>> from Informix to PostgreSQL, they use constructs like
>>
>> SELECT * INTO :tablerec FROM table ...
>>
>> where "tablerec" mirrors the table fields in a C struct.
>>
>
> Well, this was supposed to work.
>
>
>> Currently ECPG dumps core on this, more exactly aborts on it
>> in ecpg_type_name().
>>
>
> Could you please send an example where it dumps core?
>

Attached is the short example I can reproduce with.
The version I used was final PostgreSQL 8.4.0, without our
extensions posted already. I added an indication to ecpg_type_name():

[zozo(at)db00 ecpg-test]$ ecpg -C INFORMIX test28.pgc
ecpg_type_name: unhandled type 22
Félbeszakítva (core dumped)

Type 22 is exactly ECPGt_struct. gdb cannot get the passed value:

[zozo(at)db00 ecpg-test]$ gdb ~/pgc84pre/bin/ecpg core.850
...
#0 0x0000003994032215 in raise (sig=<value optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt
#0 0x0000003994032215 in raise (sig=<value optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x0000003994033d83 in abort () at abort.c:88
#2 0x0000000000423d65 in ecpg_type_name (typ=<value optimized out>) at
typename.c:65
#3 0x0000000000402742 in adjust_informix (list=0x1e74560) at preproc.y:272
#4 0x0000000000406ca7 in base_yyparse () at preproc.y:6581
#5 0x0000000000422b22 in main (argc=4, argv=0x7fff24b40aa8) at ecpg.c:456

test28.pgc contains this, ECPG aborts:

EXEC SQL DECLARE mycur CURSOR FOR SELECT * INTO :myvar FROM a1 WHERE id = 1;
EXEC SQL FETCH FROM mycur;

But you are right about the "supposed to work" part,
if I modify it the way below, it works:

EXEC SQL DECLARE mycur CURSOR FOR SELECT * FROM a1 WHERE id = 1;
EXEC SQL FETCH FROM mycur INTO :myvar;

Thanks,
Zoltán Böszörményi

> Michael
>

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Fri, Jul 17, 2009 at 03:58:21PM +0200, Boszormenyi Zoltan wrote:
> Attached is the short example I can reproduce with.
> The version I used was final PostgreSQL 8.4.0, without our
> extensions posted already. I added an indication to ecpg_type_name():
>
> [zozo(at)db00 ecpg-test]$ ecpg -C INFORMIX test28.pgc
> ...
> But you are right about the "supposed to work" part,
> if I modify it the way below, it works:

Thanks for spotting this. It appears to be a bug in Informix compatibility
mode. Without -C INFORMIX it works nicely. Thus I guess we have to find and fix
the bug. Your patch probably/hopefully is not needed.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Fri, Jul 17, 2009 at 03:58:21PM +0200, Boszormenyi Zoltan wrote:
>
>> Attached is the short example I can reproduce with.
>> The version I used was final PostgreSQL 8.4.0, without our
>> extensions posted already. I added an indication to ecpg_type_name():
>>
>> [zozo(at)db00 ecpg-test]$ ecpg -C INFORMIX test28.pgc
>> ...
>> But you are right about the "supposed to work" part,
>> if I modify it the way below, it works:
>>
>
> Thanks for spotting this. It appears to be a bug in Informix compatibility
> mode. Without -C INFORMIX it works nicely. Thus I guess we have to find and fix
> the bug. Your patch probably/hopefully is not needed.
>
> Michael
>

My previous patch on this broke "make check" in ecpg,
so ignore that. Your comment that it's a bug in Informix-mode
made me look around more. Find the attached patch I came up with.
Now my previous test code works and produces similar C code
as without "-C INFORMIX". Can it be this simple?
Can you see anything wrong with this approach?

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Fri, Jul 31, 2009 at 11:42:33AM +0200, Boszormenyi Zoltan wrote:
> made me look around more. Find the attached patch I came up with.
> Now my previous test code works and produces similar C code
> as without "-C INFORMIX". Can it be this simple?

Unfortunately it is not.

> Can you see anything wrong with this approach?

Yes, please look at the slightly changed test version that is attached to this email.

If you use e.g. int instead of MYTYPE, it works nicely, but not with MYTYPE.
Please see the comments in adjust_informix to see what this function is
supposed to do.

Michael

--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Fri, Jul 31, 2009 at 11:42:33AM +0200, Boszormenyi Zoltan wrote:
>
>> made me look around more. Find the attached patch I came up with.
>> Now my previous test code works and produces similar C code
>> as without "-C INFORMIX". Can it be this simple?
>>
>
> Unfortunately it is not.
>
>
>> Can you see anything wrong with this approach?
>>
>
> Yes, please look at the slightly changed test version that is attached to this email.
>

I have looked at it. The code seems to be invalid.

get_var(void)
{
EXEC SQL BEGIN DECLARE SECTION;
MYTYPE myvar;
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT * INTO :myvar FROM a1 WHERE id = 1;
}

"myvar" is lost as soon as the function returns
and is not visible to calling functions.

I tried to compile your code (with my previous "fix"
in place, so at least :myvar is processed and C code is output):

$ make test28
ecpg -C INFORMIX test28.pgc
cc -g -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv -g -I/home/zozo/pgc84pre/include
-I/home/zozo/pgc84pre/include/postgresql/internal -c -o test28.o test28.c
test28.pgc:15: warning: return type defaults to ‘int’
test28.pgc:14: warning: no previous prototype for ‘get_var’
test28.pgc: In function ‘get_var’:
test28.pgc:17: warning: unused variable ‘myvar’
test28.pgc: In function ‘main’:
test28.pgc:45: error: ‘myvar’ undeclared (first use in this function)
test28.pgc:45: error: (Each undeclared identifier is reported only once
test28.pgc:45: error: for each function it appears in.)
make: *** [test28.o] Error 1
rm test28.c

Line 45 in the modified code sent by you is:
printf("c = '%s'\n", myvar.c);
and the compiler correctly complains.

> If you use e.g. int instead of MYTYPE, it works nicely, but not with MYTYPE.
>

I modified getvar this way:

get_var(void)
{
EXEC SQL BEGIN DECLARE SECTION;
int myid;
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myid FROM a1 WHERE id
= 1;
}

And the preprocessed code via "ecpg -C INFORMIX" is:

get_var(void)
{
/* exec sql begin declare section */

#line 17 "test28.pgc"
int myid ;
/* exec sql end declare section */
#line 18 "test28.pgc"

ECPG_informix_set_var( 0, &( myid ), __LINE__);\
/* declare mycur cursor for select id from a1 where id = 1 */
#line 20 "test28.pgc"

}

Some systems (stack-protector extensions to GCC, etc)
make the code segfault immediately as soon as the first
FETCH statement tries to touch the lost memory area.
Just because ECPG does some tricks with ECPG_informix_set_var()
and ECPG_informix_get_var() converting variable reference
to runtime pointer values, the code wouldn't get magically valid.

> Please see the comments in adjust_informix to see what this function is
> supposed to do.
>

I did and I don't understand. I think it's just a bug
in ESQL/C to accept such constructs.

> Michael
>

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Wed, Aug 05, 2009 at 11:08:26AM +0200, Boszormenyi Zoltan wrote:
> I have looked at it. The code seems to be invalid.

Yes, it is, I was too lazy to make it valid. If you just allocate the memory
for the variable in get_var() it becomes valid.

> I tried to compile your code (with my previous "fix"
> in place, so at least :myvar is processed and C code is output):

Yes, but incorrect one.

> Some systems (stack-protector extensions to GCC, etc)
> make the code segfault immediately as soon as the first
> FETCH statement tries to touch the lost memory area.
> Just because ECPG does some tricks with ECPG_informix_set_var()
> and ECPG_informix_get_var() converting variable reference
> to runtime pointer values, the code wouldn't get magically valid.

Again, this doesn't matter in this case as we try to get the preprocessor to
work with a test case as small as possible.

> I did and I don't understand. I think it's just a bug
> in ESQL/C to accept such constructs.

I do not like this "feature" either, but it's there and therefore should work
in our compatibility mode. And it does with non-struct variables. Just look at
test/compat_informix/test_informix.pgc for a real and working example.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Wed, Aug 05, 2009 at 11:08:26AM +0200, Boszormenyi Zoltan wrote:
>
>> I have looked at it. The code seems to be invalid.
>>
>
> Yes, it is, I was too lazy to make it valid. If you just allocate the memory
> for the variable in get_var() it becomes valid.
>

With allocated memory, yes, the code would be valid.

This means that what I did in my first patch for this
problem in "add_struct_to_head()" (unrolling members
of the struct) has to be done in adjust_informix(),
turning it into a recursive function.
I think this would be a good solution. What do you think?

>
>> I tried to compile your code (with my previous "fix"
>> in place, so at least :myvar is processed and C code is output):
>>
>
> Yes, but incorrect one.
>
>
>> Some systems (stack-protector extensions to GCC, etc)
>> make the code segfault immediately as soon as the first
>> FETCH statement tries to touch the lost memory area.
>> Just because ECPG does some tricks with ECPG_informix_set_var()
>> and ECPG_informix_get_var() converting variable reference
>> to runtime pointer values, the code wouldn't get magically valid.
>>
>
> Again, this doesn't matter in this case as we try to get the preprocessor to
> work with a test case as small as possible.
>
>
>> I did and I don't understand. I think it's just a bug
>> in ESQL/C to accept such constructs.
>>
>
> I do not like this "feature" either, but it's there and therefore should work
> in our compatibility mode. And it does with non-struct variables. Just look at
> test/compat_informix/test_informix.pgc for a real and working example.
>
> Michael
>

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Wed, Aug 05, 2009 at 11:52:57AM +0200, Boszormenyi Zoltan wrote:
> This means that what I did in my first patch for this
> problem in "add_struct_to_head()" (unrolling members
> of the struct) has to be done in adjust_informix(),
> turning it into a recursive function.
> I think this would be a good solution. What do you think?

No, this doesn't seem right. There should be no need to unroll a struct.
Instead I would think struct support should be added to the get_var/set_var
functions. At least that's my guess without really digging into it.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Wed, Aug 05, 2009 at 11:52:57AM +0200, Boszormenyi Zoltan wrote:
>
>> This means that what I did in my first patch for this
>> problem in "add_struct_to_head()" (unrolling members
>> of the struct) has to be done in adjust_informix(),
>> turning it into a recursive function.
>> I think this would be a good solution. What do you think?
>>
>
> No, this doesn't seem right. There should be no need to unroll a struct.
> Instead I would think struct support should be added to the get_var/set_var
> functions. At least that's my guess without really digging into it.
>
> Michael
>

My question is: why not unroll the struct in the preprocessor?
This adjust_informix() issue aside, if I do this in the same function:

EXEC SQL BEGIN DECLARE SECTION;
MYTYPE myvar;
MYNULLTYPE mynullvar;
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT * INTO :myvar
:mynullvar FROM a1;
EXEC SQL OPEN mycur;

EXEC SQL WHENEVER NOT FOUND DO BREAK;
while (1) {
EXEC SQL FETCH FROM mycur;
...
}

then the preprocessed code for DECLARE and FETCH look like below:

{ ECPGdo(__LINE__, 1, 1, NULL, 0, ECPGst_normal, "declare mycur
cursor for select * from a1 where id = 1", ECPGt_EOIT,
ECPGt_int,&(myvar.id),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_char,&(myvar.t),(long)64,(long)1,(64)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_double,&(myvar.d1),(long)1,(long)1,sizeof(double),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_double,&(myvar.d2),(long)1,(long)1,sizeof(double),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_char,&(myvar.c),(long)30,(long)1,(30)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
#line 39 "test28.pgc"

...

{ ECPGdo(__LINE__, 1, 1, NULL, 0, ECPGst_normal, "fetch from
mycur", ECPGt_EOIT,
ECPGt_int,&(myvar.id),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_char,&(myvar.t),(long)64,(long)1,(64)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_double,&(myvar.d1),(long)1,(long)1,sizeof(double),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_double,&(myvar.d2),(long)1,(long)1,sizeof(double),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_char,&(myvar.c),(long)30,(long)1,(30)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
#line 43 "test28.pgc"

These are done by ECPGdump_a_struct(), when the struct's members
and their type, size, etc are known. It's unrolled by the ecpg preprocessor.
Your idea about pushing struct support into set_var/get_var seems
not appropriate. With the current set_var scheme, you pass one variable's
properties. Why not keep this simplicity and unroll the struct in the
preprocessor into multiple set_var() calls for simple types?

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Wed, Aug 05, 2009 at 03:04:00PM +0200, Boszormenyi Zoltan wrote:
> My question is: why not unroll the struct in the preprocessor?

The problem is not that the struct is unrolled in the preprocessor. I just
don't like the idea of having two places where structs are unrolled when one
could be sufficient.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Wed, Aug 05, 2009 at 11:08:26AM +0200, Boszormenyi Zoltan wrote:
>
>> I have looked at it. The code seems to be invalid.
>>
>
> Yes, it is, I was too lazy to make it valid. If you just allocate the memory
> for the variable in get_var() it becomes valid.
>

If you meant like this below, then ECPG segfaults on this too:

int *
get_var(void)
{
EXEC SQL BEGIN DECLARE SECTION;
int *myvar;
EXEC SQL END DECLARE SECTION;

myvar = malloc(sizeof(int));
EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1
WHERE id = 1;
return myvar;
}

ecpg_type_name() aborts, ECPGt_array is unhandled
besides struct and union, it's called at the same place
in adjust_informix() as ECPGt_struct.

Attached is my modified test28.pgc. Compiling it
*without* -C INFORMIX makes it unusable, the variable
or the address where the data should be fetched into
doesn't even gets emitted in neither the DECLARE/OPEN
nor the FETCH callsites. I think this code should be valid
even in non-Informix-compatible mode.

> ... Just look at
> test/compat_informix/test_informix.pgc for a real and working example.
>

The example there is the other way around.
The variable, the DECLARE and FETCH commands
are in the outer main() function, and it calls a function called
openit() where the OPEN command is emitted, so that
example doesn't help here too much.

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

Michael Meskes írta:
> On Wed, Aug 05, 2009 at 03:04:00PM +0200, Boszormenyi Zoltan wrote:
>
>> My question is: why not unroll the struct in the preprocessor?
>>
>
> The problem is not that the struct is unrolled in the preprocessor. I just
> don't like the idea of having two places where structs are unrolled when one
> could be sufficient.
>

Yes, one place should be sufficient. Read my other mail about
the modified get_var() function. I start to get convinced that
adjust_informix() and ECPG_informix_{get|set}_var() should not
be Informix compat specific at all.

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

Boszormenyi Zoltan írta:
> Michael Meskes írta:
>
>> On Wed, Aug 05, 2009 at 11:08:26AM +0200, Boszormenyi Zoltan wrote:
>>
>>
>>> I have looked at it. The code seems to be invalid.
>>>
>>>
>> Yes, it is, I was too lazy to make it valid. If you just allocate the memory
>> for the variable in get_var() it becomes valid.
>>
>>
>
> If you meant like this below, then ECPG segfaults on this too:
>
> int *
> get_var(void)
> {
> EXEC SQL BEGIN DECLARE SECTION;
> int *myvar;
> EXEC SQL END DECLARE SECTION;
>
> myvar = malloc(sizeof(int));
> EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1
> WHERE id = 1;
> return myvar;
> }
>

And another problem that we have run into already.
ECPG is a one-stage preprocessor, instead of a two-stage one.
If the above function is located later in the source file than
the "OPEN mycur" or "FETCH mycur", then ECPG complains
about an unknown cursor. Not a big annoyance, but ESQL/C
supports that.

> ecpg_type_name() aborts, ECPGt_array is unhandled
> besides struct and union, it's called at the same place
> in adjust_informix() as ECPGt_struct.
>
> Attached is my modified test28.pgc. Compiling it
> *without* -C INFORMIX makes it unusable, the variable
> or the address where the data should be fetched into
> doesn't even gets emitted in neither the DECLARE/OPEN
> nor the FETCH callsites. I think this code should be valid
> even in non-Informix-compatible mode.
>
>
>> ... Just look at
>> test/compat_informix/test_informix.pgc for a real and working example.
>>
>>
>
> The example there is the other way around.
> The variable, the DECLARE and FETCH commands
> are in the outer main() function, and it calls a function called
> openit() where the OPEN command is emitted, so that
> example doesn't help here too much.
>
> Best regards,
> Zoltán Böszörményi
>
>
> ------------------------------------------------------------------------
>
>

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Wed, Aug 05, 2009 at 04:22:53PM +0200, Boszormenyi Zoltan wrote:
> If you meant like this below, then ECPG segfaults on this too:

Right, because arrays as as well unimplemented as are structs. I meant
something like this;

int *
get_var(void)
{
EXEC SQL BEGIN DECLARE SECTION;
static int myvar;
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1 WHERE id = 1;
return (&myvar);
}

> Attached is my modified test28.pgc. Compiling it
> *without* -C INFORMIX makes it unusable, the variable
> or the address where the data should be fetched into
> doesn't even gets emitted in neither the DECLARE/OPEN
> nor the FETCH callsites. I think this code should be valid
> even in non-Informix-compatible mode.

I don't think i buy into this. The variable is out of scope at the time open is
called. Why do you think this should work?

> > ... Just look at
> > test/compat_informix/test_informix.pgc for a real and working example.
> >
>
> The example there is the other way around.
> The variable, the DECLARE and FETCH commands
> are in the outer main() function, and it calls a function called
> openit() where the OPEN command is emitted, so that
> example doesn't help here too much.

Eh, why not? We're talking about a bug/missing feature in the precompiler
itself. And the precompiler doesn't see this difference. I just pointed you to
one working example. Anyway I attached a modified test28.pgc that should work
in compatibility mode.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Wed, Aug 05, 2009 at 04:22:53PM +0200, Boszormenyi Zoltan wrote:
>
>> If you meant like this below, then ECPG segfaults on this too:
>>
>
> Right, because arrays as as well unimplemented as are structs. I meant
> something like this;
>
> int *
> get_var(void)
> {
> EXEC SQL BEGIN DECLARE SECTION;
> static int myvar;
> EXEC SQL END DECLARE SECTION;
>
> EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1 WHERE id = 1;
> return (&myvar);
> }
>

Which isn't exactly a good programming habit.
It can break subtly in multi-threaded code.
And you were talking about allocated variables, which,
in my book (without explicitely mentioning "statically")
boils down to malloc().

>> Attached is my modified test28.pgc. Compiling it
>> *without* -C INFORMIX makes it unusable, the variable
>> or the address where the data should be fetched into
>> doesn't even gets emitted in neither the DECLARE/OPEN
>> nor the FETCH callsites. I think this code should be valid
>> even in non-Informix-compatible mode.
>>
>
> I don't think i buy into this. The variable is out of scope at the time open is
> called. Why do you think this should work?
>

Because the allocated area, the pointer to it that's returned
from the function _is_ in scope. Possibly under a new name,
but that's why ECPG employs ECPG_informix_set_var() and
_get_var(), to make OPEN and FETCH independent of the
out of scope variables, no?

This code is not much different:

int *
get_var(void)
{
static int myvar;
return (&myvar);
}

another_func(...)
{
EXEC SQL BEGIN DECLARE SECTION;
int *myvar = get_var();
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1 WHERE id = 1;

EXEC SQL OPEN mycur;

...
}

But it also breaks in compatibility mode, exactly because
adjust_informix() is called on it and even in-scope variables
are replaced with ECPG_informix_set_var() and _get_var() calls
and adjust_informix() is not prepared to certain types (array, struct).

And if you just consider this which doesn't break:

another_func(...)
{
EXEC SQL BEGIN DECLARE SECTION;
int myvar;
EXEC SQL END DECLARE SECTION;

EXEC SQL DECLARE mycur CURSOR FOR SELECT id INTO :myvar FROM a1 WHERE id = 1;

EXEC SQL OPEN mycur;

...
}

In the above code local, in-scope variables are also replaced
with ECPG_informix_set_var() and _get_var() calls.
Totally unnecessary, or totally necessary even in non-compatible
mode, depending on which leg I stand on. If you move the
struct/union unrolling into adjust_informix() and handle arrays,
ECPGdump_a_struct() won't be needed then.

>>> ... Just look at
>>> test/compat_informix/test_informix.pgc for a real and working example.
>>>
>>>
>> The example there is the other way around.
>> The variable, the DECLARE and FETCH commands
>> are in the outer main() function, and it calls a function called
>> openit() where the OPEN command is emitted, so that
>> example doesn't help here too much.
>>
>
> Eh, why not?

Because that example contradicts all sensible programming habits.
(Well, what is "sensible" is different between people, so don't take it
personal.)

> We're talking about a bug/missing feature in the precompiler
> itself. And the precompiler doesn't see this difference. I just pointed you to
> one working example. Anyway I attached a modified test28.pgc that should work
> in compatibility mode.
>
> Michael
>

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

On Fri, Aug 07, 2009 at 11:48:33AM +0200, Boszormenyi Zoltan wrote:
> Which isn't exactly a good programming habit.

I couldn't agree more.

> In the above code local, in-scope variables are also replaced
> with ECPG_informix_set_var() and _get_var() calls.
> Totally unnecessary, or totally necessary even in non-compatible
> mode, depending on which leg I stand on. If you move the
> struct/union unrolling into adjust_informix() and handle arrays,
> ECPGdump_a_struct() won't be needed then.

Yeah, right, and you also add this hack to all applications. No.

> Because that example contradicts all sensible programming habits.
> (Well, what is "sensible" is different between people, so don't take it
> personal.)

Hey, don't blame me! Informix uses this feature to some extend, this is why it
got implemented. If you look into the source code you will see this:

* This breaks standard and leads to some very dangerous programming.
* Since they do, we have to work around and accept their syntax as well.
* But we will do so ONLY in Informix mode.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: meskes(at)jabber(dot)org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes írta:
> On Fri, Aug 07, 2009 at 11:48:33AM +0200, Boszormenyi Zoltan wrote:
>
>> Which isn't exactly a good programming habit.
>>
>
> I couldn't agree more.
>

:-)

>> In the above code local, in-scope variables are also replaced
>> with ECPG_informix_set_var() and _get_var() calls.
>> Totally unnecessary, or totally necessary even in non-compatible
>> mode, depending on which leg I stand on. If you move the
>> struct/union unrolling into adjust_informix() and handle arrays,
>> ECPGdump_a_struct() won't be needed then.
>>
>
> Yeah, right, and you also add this hack to all applications. No.
>

What do you mean?

In the meantime, I recalled my original idea, the patch
you have already seen, where an unrolling was introduced
in "ecpg_into:" rule with a new add_struct_to_head() function.
"ecpg_into:" is a good central place where this unrolling
can be done, so:
- ECPGdump_a_struct() won't be needed, and
- adjust_informix() doesn't encounter structs or unions.
So, struct unrolling would still be done at only one place
and one problem goes away.
About the arrays, I have to think about more.

>> Because that example contradicts all sensible programming habits.
>> (Well, what is "sensible" is different between people, so don't take it
>> personal.)
>>
>
> Hey, don't blame me! Informix uses this feature to some extend, this is why it
> got implemented. If you look into the source code you will see this:
>
> * This breaks standard and leads to some very dangerous programming.
> * Since they do, we have to work around and accept their syntax as well.
> * But we will do so ONLY in Informix mode.
>

I didn't want to blame you, I just wanted to say that
from my experience, it's more common that these are
the separation blocks:
1. memory allocation
1a. DECLARE
2. OPEN/FETCH
2a. CLOSE
3. memory freeing

Point "1a" can be grouped with either "1" or "2", similarly
point "2a" can be grouped with either "2" or "3".
"Grouped" means that they are in the same function, at
the same visibility level.

It was very alien to me seeing that only OPEN was
moved out to another function in the mentioned
regression test example.

I think that the DECLARE doing this adjust_informix()
call (which should be called adjust_out_of_scope_vars() ;-) )
shouldn't be Informix-specific at all. The above separation
is natural (again, a very subjective POV), and should be
supported in native mode, too.

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/

Hi,

I got around to fix the struct problem, when ECPG aborted
upon finding the following statement:

DECLARE mycur
CURSOR FOR
SELECT *
INTO :mystruct [:myindicatorstruct] ...

As I found out earlier, ECPG also aborted when the variable
is a pointer, like:
struct mytype *myvar;

I fixed both cases, and added the quirk to the lexer to keep
the last identifier as a function name before any ( or { is
encountered and this allowed me to detect and handle
out of scope DECLARE/OPEN/FETCH.
You know the ecpg lexer/grammar better than me,
you may find a better way to do this.

Anyway, the end result is that now in compat mode
ECPGinformix_get_var() type vars are not always used,
only when OPEN and FETCH are out of scope, i.e. they
are in a function different from where the cursor was
DECLAREd.

I added two regression tests, one tests only the struct
usability in compat mode, which didn't work before and
started this little "crusade". :-) The other tests struct pointers
and out of scope DECLARE/OPEN/FETCH.

Tell me that this isn't a useful feature in non-compat mode.
I will leave the rewrite to you if you find it so, I left a comment
for you in the code. ;-)

Best regards,
Zoltán Böszörményi

--
Bible has answers for everything. Proof:
"But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil." (Matthew 5:37) - basics of digital technology.
"May your kingdom come" - superficial description of plate tectonics

----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
http://www.postgresql.at/