Re: using arrays within structure in ECPG

Lists: pgsql-hackers
From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: using arrays within structure in ECPG
Date: 2014-03-24 06:22:30
Message-ID: CAFjFpRdwCZ+0tiJg1iRXG4d=ehVDHPTyooM6Ra6aFff-VADpaA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi,
I tried using integer array within a structure array in ECPG code. But it
resulted in some garbage values being printed from the table. Here are the
details,

The ECPG program is attached (array_test.pgc). It tries to read the
contents of table emp, whose structure and contents are as follows
postgres=# \d+ emp
Table "public.emp"
Column | Type | Modifiers | Storage | Stats target |
Description
---------+-------------------+-----------+----------+--------------+-------------
empno | numeric(4,0) | | main | |
ename | character varying | | extended | |
job | character varying | | extended | |
arr_col | integer[] | | extended | |
Has OIDs: no

postgres=# select * from emp;
empno | ename | job | arr_col
-------+--------+---------+------------
7900 | JAMES | CLERK | {1,2,7900}
7902 | FORD | ANALYST | {1,2,7902}
7934 | MILLER | CLERK | {1,2,7934}
(3 rows)

You will notice that the last element of the arr_col array is same as the
empno of that row.

The ECPG program tries to read the rows using FETCH in a structure emp
defined as
15 struct employee {
16 int empno;
17 char ename[11];
18 char job[15];
19 int arr_col[3];
20 };

and then print the read contents as
39 /* Print members of the structure. */
40 for ( i = 0 ;i < 3; i++){
41 printf("empno=%d, ename=%s, job=%s, arr_col[2]=%d\n",
emp[i].empno, emp[i].ename, emp[i].job, emp[i].arr_col[2]);
42
43 }

But garbage values are printed
[ashutosh(at)ubuntu repro]./array_test

+++++++++++++++++++++++++++++++++++++++++++++++
empno=7900, ename=JAMES, job=CLERK, arr_col[2]=1
empno=2, ename=�, job=ANALYST, arr_col[2]=32767
empno=7934, ename=MILLER, job=CLERK, arr_col[2]=1719202336

Here are steps I have used to compile the ECPG program
[ashutosh(at)ubuntu repro]make array_test
ecpg -c -I/work/pg_head/build/include array_test.pgc
cc -I/work/pg_head/build/include -g -c -o array_test.o array_test.c
cc -g array_test.o -L/work/pg_head/build/lib -lecpg -lpq -o array_test
rm array_test.o array_test.c

where /work/pg_head/build is the directory containing the postgresql build
(essentially argument to the --prefix option to configure).

The programs compiles and links fine.

Without the arr_col member, the program works fine. So, it seems to be a
problem with array within structure array.

In array_test.c I see that the ECPGdo statement corresponding to the FETCH
command is as follows
87 /* Fetch multiple columns into one structure. */
88 { ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "fetch 3 from
cur1", ECPGt_EOIT,
89 ECPGt_int,&(emp->empno),(long)1,(long)14,sizeof( struct employee ),
90 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
91 ECPGt_char,&(emp->ename),(long)11,(long)14,sizeof( struct employee
),
92 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
93 ECPGt_char,&(emp->job),(long)15,(long)14,sizeof( struct employee ),
94 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
95 ECPGt_int,(emp->arr_col),(long)1,(long)3,sizeof(int),
96 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);

For all the members of struct employee, except arr_col, the size of array
is set to 14 and next member offset is set of sizeof (struct employee). But
for arr_col they are set to 3 and sizeof(int) resp. So, for the next row
onwards, the calculated offset of arr_col member would not coincide with
the real arr_col member's address.

Am I missing something here?
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Attachment Content-Type Size
array_test.pgc application/octet-stream 953 bytes

From: Boszormenyi Zoltan <zboszor(at)pr(dot)hu>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-03-24 10:10:25
Message-ID: 53300491.5070603@pr.hu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

2014-03-24 07:22 keltezéssel, Ashutosh Bapat írta:
> Hi,
> I tried using integer array within a structure array in ECPG code. But it resulted in
> some garbage values being printed from the table. Here are the details,
>
> The ECPG program is attached (array_test.pgc). It tries to read the contents of table
> emp, whose structure and contents are as follows
> postgres=# \d+ emp
> Table "public.emp"
> Column | Type | Modifiers | Storage | Stats target | Description
> ---------+-------------------+-----------+----------+--------------+-------------
> empno | numeric(4,0) | | main | |
> ename | character varying | | extended | |
> job | character varying | | extended | |
> arr_col | integer[] | | extended | |
> Has OIDs: no
>
> postgres=# select * from emp;
> empno | ename | job | arr_col
> -------+--------+---------+------------
> 7900 | JAMES | CLERK | {1,2,7900}
> 7902 | FORD | ANALYST | {1,2,7902}
> 7934 | MILLER | CLERK | {1,2,7934}
> (3 rows)
>
> You will notice that the last element of the arr_col array is same as the empno of that
> row.
>
> The ECPG program tries to read the rows using FETCH in a structure emp defined as
> 15 struct employee {
> 16 int empno;
> 17 char ename[11];
> 18 char job[15];
> 19 int arr_col[3];
> 20 };
>
> and then print the read contents as
> 39 /* Print members of the structure. */
> 40 for ( i = 0 ;i < 3; i++){
> 41 printf("empno=%d, ename=%s, job=%s, arr_col[2]=%d\n", emp[i].empno,
> emp[i].ename, emp[i].job, emp[i].arr_col[2]);
> 42
> 43 }
>
> But garbage values are printed
> [ashutosh(at)ubuntu repro]./array_test
>
> +++++++++++++++++++++++++++++++++++++++++++++++
> empno=7900, ename=JAMES, job=CLERK, arr_col[2]=1
> empno=2, ename=�, job=ANALYST, arr_col[2]=32767
> empno=7934, ename=MILLER, job=CLERK, arr_col[2]=1719202336
>
> Here are steps I have used to compile the ECPG program
> [ashutosh(at)ubuntu repro]make array_test
> ecpg -c -I/work/pg_head/build/include array_test.pgc
> cc -I/work/pg_head/build/include -g -c -o array_test.o array_test.c
> cc -g array_test.o -L/work/pg_head/build/lib -lecpg -lpq -o array_test
> rm array_test.o array_test.c
>
> where /work/pg_head/build is the directory containing the postgresql build (essentially
> argument to the --prefix option to configure).
>
> The programs compiles and links fine.
>
> Without the arr_col member, the program works fine. So, it seems to be a problem with
> array within structure array.
>
> In array_test.c I see that the ECPGdo statement corresponding to the FETCH command is as
> follows
> 87 /* Fetch multiple columns into one structure. */
> 88 { ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "fetch 3 from cur1", ECPGt_EOIT,
> 89 ECPGt_int,&(emp->empno),(long)1,(long)14,sizeof( struct employee ),
> 90 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 91 ECPGt_char,&(emp->ename),(long)11,(long)14,sizeof( struct employee ),
> 92 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 93 ECPGt_char,&(emp->job),(long)15,(long)14,sizeof( struct employee ),
> 94 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 95 ECPGt_int,(emp->arr_col),(long)1,(long)3,sizeof(int),
> 96 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
>
> For all the members of struct employee, except arr_col, the size of array is set to 14
> and next member offset is set of sizeof (struct employee). But for arr_col they are set
> to 3 and sizeof(int) resp. So, for the next row onwards, the calculated offset of
> arr_col member would not coincide with the real arr_col member's address.
>
> Am I missing something here?

ECPG (I think intentionally) doesn't interpret or parse array fields.
You need to pass a character array and parse the contents by yourself.

Best regards,
Zoltán Böszörményi

> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>
>


From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: Boszormenyi Zoltan <zboszor(at)pr(dot)hu>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-03-25 05:05:05
Message-ID: CAFjFpRdeSi=5oN1Z7J+Pdha_ovLk2Mb26WAHmA+XWmTEPmda_Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Mar 24, 2014 at 3:40 PM, Boszormenyi Zoltan <zboszor(at)pr(dot)hu> wrote:

> 2014-03-24 07:22 keltezéssel, Ashutosh Bapat írta:
>
> Hi,
> I tried using integer array within a structure array in ECPG code. But
> it resulted in some garbage values being printed from the table. Here are
> the details,
>
> The ECPG program is attached (array_test.pgc). It tries to read the
> contents of table emp, whose structure and contents are as follows
> postgres=# \d+ emp
> Table "public.emp"
> Column | Type | Modifiers | Storage | Stats target |
> Description
>
> ---------+-------------------+-----------+----------+--------------+-------------
> empno | numeric(4,0) | | main | |
> ename | character varying | | extended | |
> job | character varying | | extended | |
> arr_col | integer[] | | extended | |
> Has OIDs: no
>
> postgres=# select * from emp;
> empno | ename | job | arr_col
> -------+--------+---------+------------
> 7900 | JAMES | CLERK | {1,2,7900}
> 7902 | FORD | ANALYST | {1,2,7902}
> 7934 | MILLER | CLERK | {1,2,7934}
> (3 rows)
>
> You will notice that the last element of the arr_col array is same as
> the empno of that row.
>
> The ECPG program tries to read the rows using FETCH in a structure emp
> defined as
> 15 struct employee {
> 16 int empno;
> 17 char ename[11];
> 18 char job[15];
> 19 int arr_col[3];
> 20 };
>
> and then print the read contents as
> 39 /* Print members of the structure. */
> 40 for ( i = 0 ;i < 3; i++){
> 41 printf("empno=%d, ename=%s, job=%s, arr_col[2]=%d\n",
> emp[i].empno, emp[i].ename, emp[i].job, emp[i].arr_col[2]);
> 42
> 43 }
>
> But garbage values are printed
> [ashutosh(at)ubuntu repro]./array_test
>
> +++++++++++++++++++++++++++++++++++++++++++++++
> empno=7900, ename=JAMES, job=CLERK, arr_col[2]=1
> empno=2, ename=� , job=ANALYST, arr_col[2]=32767
> empno=7934, ename=MILLER, job=CLERK, arr_col[2]=1719202336
>
> Here are steps I have used to compile the ECPG program
> [ashutosh(at)ubuntu repro]make array_test
> ecpg -c -I/work/pg_head/build/include array_test.pgc
> cc -I/work/pg_head/build/include -g -c -o array_test.o array_test.c
> cc -g array_test.o -L/work/pg_head/build/lib -lecpg -lpq -o array_test
> rm array_test.o array_test.c
>
> where /work/pg_head/build is the directory containing the postgresql
> build (essentially argument to the --prefix option to configure).
>
> The programs compiles and links fine.
>
> Without the arr_col member, the program works fine. So, it seems to be a
> problem with array within structure array.
>
> In array_test.c I see that the ECPGdo statement corresponding to the
> FETCH command is as follows
> 87 /* Fetch multiple columns into one structure. */
> 88 { ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "fetch 3 from
> cur1", ECPGt_EOIT,
> 89 ECPGt_int,&(emp->empno),(long)1,(long)14,sizeof( struct employee ),
> 90 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 91 ECPGt_char,&(emp->ename),(long)11,(long)14,sizeof( struct employee
> ),
> 92 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 93 ECPGt_char,&(emp->job),(long)15,(long)14,sizeof( struct employee ),
> 94 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
> 95 ECPGt_int,(emp->arr_col),(long)1,(long)3,sizeof(int),
> 96 ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
>
> For all the members of struct employee, except arr_col, the size of
> array is set to 14 and next member offset is set of sizeof (struct
> employee). But for arr_col they are set to 3 and sizeof(int) resp. So, for
> the next row onwards, the calculated offset of arr_col member would not
> coincide with the real arr_col member's address.
>
> Am I missing something here?
>
>
> ECPG (I think intentionally) doesn't interpret or parse array fields.
> You need to pass a character array and parse the contents by yourself.
>
>
That doesn't seem to be the case with bare arrays (not included in anything
else). I added following lines to the program attached in my first mail,
and they worked perfectly fine.

47 /* Test only arrays */
48 EXEC SQL DECLARE cur2 CURSOR FOR select arr_col from emp;
49
50 EXEC SQL OPEN cur2;
51
52 EXEC SQL FETCH 1 FROM cur2 into :emp[0].arr_col;
53
54 printf("\n+++++++++++++++++++++++++++++++++++++++++++++++\n");
55
56 /* Print members of the array fetched. */
57 for ( i = 0 ;i < 3; i++)
58 {
59 printf("arr_col[%d] = %d\n", i, emp[0].arr_col[i]);
60 }
61 EXEC SQL CLOSE cur2;

Anyway, if that's a restriction, shouldn't there be an error during
compilation?

> Best regards,
> Zoltán Böszörményi
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>
>
>
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


From: Michael Meskes <meskes(at)postgresql(dot)org>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-03-27 17:35:26
Message-ID: 20140327173526.GA19440@feivel.credativ.lan
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Mar 24, 2014 at 11:52:30AM +0530, Ashutosh Bapat wrote:
> For all the members of struct employee, except arr_col, the size of array
> is set to 14 and next member offset is set of sizeof (struct employee). But
> for arr_col they are set to 3 and sizeof(int) resp. So, for the next row
> onwards, the calculated offset of arr_col member would not coincide with
> the real arr_col member's address.
>
> Am I missing something here?

No, this looks like a bug to me. I haven't had time to look into the source codebut the offset definitely is off.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL


From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-01 11:04:03
Message-ID: CAFjFpRfDJSuLxEQgYszdKsKPoJRu60-2rZPRxFNKNiAjODujXA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi MIchael,
I tried to fix the offset problem. PFA the patch. It does solve the problem
of setting wrong offset in ECPGdo() call.

But then there is problem of interpreting the result from server as an
array within array of structure. The problem is there is in
ecpg_get_data(). This function can not understand that the "field" is an
array of integers (or for that matter array of anything) and store all the
values in contiguous memory at the given address.

On Thu, Mar 27, 2014 at 11:05 PM, Michael Meskes <meskes(at)postgresql(dot)org>wrote:

> On Mon, Mar 24, 2014 at 11:52:30AM +0530, Ashutosh Bapat wrote:
> > For all the members of struct employee, except arr_col, the size of array
> > is set to 14 and next member offset is set of sizeof (struct employee).
> But
> > for arr_col they are set to 3 and sizeof(int) resp. So, for the next row
> > onwards, the calculated offset of arr_col member would not coincide with
> > the real arr_col member's address.
> >
> > Am I missing something here?
>
> No, this looks like a bug to me. I haven't had time to look into the
> source codebut the offset definitely is off.
>
> Michael
> --
> Michael Meskes
> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
> Jabber: michael.meskes at gmail dot com
> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Attachment Content-Type Size
pg_ecpg_offset.patch text/x-patch 1.2 KB

From: Michael Meskes <meskes(at)postgresql(dot)org>
To: pgsql-hackers(at)postgresql(dot)org
Cc: ashutosh(dot)bapat(at)enterprisedb(dot)com
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-01 16:50:36
Message-ID: 533AEE5C.9030307@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi Ashutosh,

> I tried to fix the offset problem. PFA the patch. It does solve the
> problem of setting wrong offset in ECPGdo() call.

Thanks, looks correct to me.

> But then there is problem of interpreting the result from server as an
> array within array of structure. The problem is there is in
> ecpg_get_data(). This function can not understand that the "field" is an
> array of integers (or for that matter array of anything) and store all
> the values in contiguous memory at the given address.

I guess I know where that comes from, without actually looking at the
code, though. Nested arrays are not supported by ecpg and the
precompiler spits out an error message, just check preproc/type.c.
However, in your example you have the struct essantially sandwiched
between the arrays and the (too) simple check in that file doesn't
notice, but because the implementation is nevertheless lacking.

I'm sorry, but this sounds like a missing feature bug.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL


From: Merlin Moncure <mmoncure(at)gmail(dot)com>
To: Michael Meskes <meskes(at)postgresql(dot)org>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, ashutosh(dot)bapat(at)enterprisedb(dot)com
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-01 23:28:43
Message-ID: CAHyXU0zUQeHtgVCWjJzYzhpN8u-96_8-cw-4eK+fwB_RiJRfog@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Apr 1, 2014 at 11:50 AM, Michael Meskes <meskes(at)postgresql(dot)org> wrote:
> Hi Ashutosh,
>
>> I tried to fix the offset problem. PFA the patch. It does solve the
>> problem of setting wrong offset in ECPGdo() call.
>
> Thanks, looks correct to me.
>
>> But then there is problem of interpreting the result from server as an
>> array within array of structure. The problem is there is in
>> ecpg_get_data(). This function can not understand that the "field" is an
>> array of integers (or for that matter array of anything) and store all
>> the values in contiguous memory at the given address.
>
> I guess I know where that comes from, without actually looking at the
> code, though. Nested arrays are not supported by ecpg and the
> precompiler spits out an error message, just check preproc/type.c.
> However, in your example you have the struct essantially sandwiched
> between the arrays and the (too) simple check in that file doesn't
> notice, but because the implementation is nevertheless lacking.
>
> I'm sorry, but this sounds like a missing feature bug.

Small aside:

I've often wondered if the right long term approach is to abstract
backend type code into a shared library that both the server and
(optionally) the client would link with. That would make extending
support to exotic types in ecpg much easier.

merlin


From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: Michael Meskes <meskes(at)postgresql(dot)org>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-02 04:03:15
Message-ID: CAFjFpRe6dxuCr3i4BqMuPj-AW8qPNp0sa5nsU4-pOfRNxt3VHg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

So, you are saying that we should try to catch such errors and report
during pre-compile time. That's better than silently corrupting the data.

On Tue, Apr 1, 2014 at 10:20 PM, Michael Meskes <meskes(at)postgresql(dot)org>wrote:

> Hi Ashutosh,
>
> > I tried to fix the offset problem. PFA the patch. It does solve the
> > problem of setting wrong offset in ECPGdo() call.
>
> Thanks, looks correct to me.
>
> > But then there is problem of interpreting the result from server as an
> > array within array of structure. The problem is there is in
> > ecpg_get_data(). This function can not understand that the "field" is an
> > array of integers (or for that matter array of anything) and store all
> > the values in contiguous memory at the given address.
>
> I guess I know where that comes from, without actually looking at the
> code, though. Nested arrays are not supported by ecpg and the
> precompiler spits out an error message, just check preproc/type.c.
> However, in your example you have the struct essantially sandwiched
> between the arrays and the (too) simple check in that file doesn't
> notice, but because the implementation is nevertheless lacking.
>
> I'm sorry, but this sounds like a missing feature bug.
>
> Michael
> --
> Michael Meskes
> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
> Jabber: michael.meskes at gmail dot com
> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


From: Michael Meskes <meskes(at)postgresql(dot)org>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
Cc: Michael Meskes <meskes(at)postgresql(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-02 09:40:05
Message-ID: 20140402094005.GA14552@feivel.credativ.lan
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Wed, Apr 02, 2014 at 09:33:15AM +0530, Ashutosh Bapat wrote:
> So, you are saying that we should try to catch such errors and report
> during pre-compile time. That's better than silently corrupting the data.

Well, I think this goes without saying.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL


From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, Michael Meskes <meskes(at)postgresql(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-02 12:19:03
Message-ID: CAFjFpReujo8H_KeHF3k1Zbk7LL7AX2cQ2d_r_8R6fnDStpD1kA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

I have one more doubt, regarding offsets.

In ECPGdump_a_simple() we have code
if (siz == NULL || strlen(siz) == 0 || strcmp(arrsize, "0") == 0 ||
strcmp(arrsize, "1") == 0)
fprintf(o, "\n\t%s,%s,(long)%s,(long)%s,%s, ", get_type(type),
variable, varcharsize, arrsize, offset);
else
fprintf(o, "\n\t%s,%s,(long)%s,(long)%s,%s, ", get_type(type),
variable, varcharsize, arrsize, siz);

If the caller has passed siz, it means that this variable is part of the a
structure. Remember in dump_variables(), this function is called with
struct_sizeof = NULL. So, once we know that siz != NULL and strlen(siz) !=
0, it's evident that the simple variable we are dumping is part of a
structure and hence we should be using "siz" instead of "offset". Why then
we still check arrsize?

In a case, where we are dumping a pointer to a structure, this code dumps
each member with offset = size of that member, thus again corrupting
memory, if there are more than one rows being saved through pointer.

On Wed, Apr 2, 2014 at 3:10 PM, Michael Meskes <meskes(at)postgresql(dot)org>wrote:

> On Wed, Apr 02, 2014 at 09:33:15AM +0530, Ashutosh Bapat wrote:
> > So, you are saying that we should try to catch such errors and report
> > during pre-compile time. That's better than silently corrupting the data.
>
> Well, I think this goes without saying.
>
> Michael
> --
> Michael Meskes
> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
> Jabber: michael.meskes at gmail dot com
> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


From: Michael Meskes <meskes(at)postgresql(dot)org>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
Cc: Michael Meskes <meskes(at)postgresql(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-02 14:59:34
Message-ID: 20140402145934.GA9724@feivel.credativ.lan
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Wed, Apr 02, 2014 at 05:49:03PM +0530, Ashutosh Bapat wrote:
> I have one more doubt, regarding offsets.
> ...

This is actually a very good question. Parts of this code are older than my
work on ecpg, meaning they were already in version 0.1. It could very well be
that with some changes over the years this test isn't needed anymore. The
regression suite works without just nicely.

To be honest, the whole type.c code needs a real rewrite to be better legible.

Michael
--
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
Jabber: michael.meskes at gmail dot com
VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL


From: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: using arrays within structure in ECPG
Date: 2014-04-03 07:08:21
Message-ID: CAFjFpRfSXMtLnym6w4MKJEvYS65SAHZfrYohv1XcKZ_Et6WR_w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi Michael,
The problem of offsets seems to be universal. If there is a structure
within structure. The offset to the members of inner structure should be
the size of the outer structure and not size of inner structure. Applying
this rule recursively, offset to the member of any nested structure, at
whatever level of nesting it is, should be same as the size of the
outermost structure. But the code as of now, is using the size of the
immediate parent.

None of these problems are caught in the regression because, whatever tests
I have seen are not fetching more than one tuple into such complex
structure.

On Tue, Apr 1, 2014 at 4:34 PM, Ashutosh Bapat <
ashutosh(dot)bapat(at)enterprisedb(dot)com> wrote:

> Hi MIchael,
> I tried to fix the offset problem. PFA the patch. It does solve the
> problem of setting wrong offset in ECPGdo() call.
>
> But then there is problem of interpreting the result from server as an
> array within array of structure. The problem is there is in
> ecpg_get_data(). This function can not understand that the "field" is an
> array of integers (or for that matter array of anything) and store all the
> values in contiguous memory at the given address.
>
>
>
> On Thu, Mar 27, 2014 at 11:05 PM, Michael Meskes <meskes(at)postgresql(dot)org>wrote:
>
>> On Mon, Mar 24, 2014 at 11:52:30AM +0530, Ashutosh Bapat wrote:
>> > For all the members of struct employee, except arr_col, the size of
>> array
>> > is set to 14 and next member offset is set of sizeof (struct employee).
>> But
>> > for arr_col they are set to 3 and sizeof(int) resp. So, for the next row
>> > onwards, the calculated offset of arr_col member would not coincide with
>> > the real arr_col member's address.
>> >
>> > Am I missing something here?
>>
>> No, this looks like a bug to me. I haven't had time to look into the
>> source codebut the offset definitely is off.
>>
>> Michael
>> --
>> Michael Meskes
>> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
>> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
>> Jabber: michael.meskes at gmail dot com
>> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
>>
>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company