Re: BUG #7730: intarray representation of empty arrays

The following bug has been logged on the website:

Bug reference: 7730
Logged by: elein
Email address: elein(at)varlena(dot)com
PostgreSQL version: 9.2.1
Operating system: Linux
Description:

select NULLIF('{1,2,3}'::integer[] - '{3,2,1}'::integer[], '{}'::integer[]);
This returns an empty array. It should return NULL.

Per RhodiumToad: the core code represents '{}' as an array with 0
dimensions, whereas intarray represents it as an array with 1 dimension but
0 elements

intarray should use the same standards as the core code if possible. I
peered at the code and don't see anything untoward but did not have time to
spend on it.

On Wed, Dec 5, 2012 at 12:44:39AM +0000, elein(at)varlena(dot)com wrote:
> The following bug has been logged on the website:
>
> Bug reference: 7730
> Logged by: elein
> Email address: elein(at)varlena(dot)com
> PostgreSQL version: 9.2.1
> Operating system: Linux
> Description:
>
> select NULLIF('{1,2,3}'::integer[] - '{3,2,1}'::integer[], '{}'::integer[]);
> This returns an empty array. It should return NULL.
>
> Per RhodiumToad: the core code represents '{}' as an array with 0
> dimensions, whereas intarray represents it as an array with 1 dimension but
> 0 elements
>
> intarray should use the same standards as the core code if possible. I
> peered at the code and don't see anything untoward but did not have time to
> spend on it.

I just got time to look at this, and it is certainly easier to see when
you use array_dims():

SELECT '{1,2,3}'::integer[] - '{3,2,1}'::integer[];
?column?
----------
{}

SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
array_dims
------------
[1:0]

SELECT array_dims('{}'::integer[]);
array_dims
------------
(null)

This is part of the larger TODO item of how to handle empty
>=1-dimensional empty arrays vs. zero-dimensional empty arrays, which is
discussed here:

https://wiki.postgresql.org/wiki/Todo#Arrays
Improve handling of empty arrays

In that thread, no one could find a way to create a 1-dimensional empty
array at the SQL level, but thanks to intarray, you found a way. It is
natural that intarray, being mostly used for one-dimensional arrays,
would return a 1-dimensional empty array. However, besides being
inconsistent, as you mentioned, there is also no way to dump/restore
one-dimensional empty arrays, which is a larger concern.

I have developed the attached patch to force empty intarray results to
be zero-dimensional empty arrays, rather than 1-dimensional empty
arrays. With this patch, a zero-dimensional empty array is returned:

SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
array_dims
------------
(null)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Applied.

---------------------------------------------------------------------------

On Wed, Sep 4, 2013 at 03:01:50PM -0400, Bruce Momjian wrote:
> On Wed, Dec 5, 2012 at 12:44:39AM +0000, elein(at)varlena(dot)com wrote:
> > The following bug has been logged on the website:
> >
> > Bug reference: 7730
> > Logged by: elein
> > Email address: elein(at)varlena(dot)com
> > PostgreSQL version: 9.2.1
> > Operating system: Linux
> > Description:
> >
> > select NULLIF('{1,2,3}'::integer[] - '{3,2,1}'::integer[], '{}'::integer[]);
> > This returns an empty array. It should return NULL.
> >
> > Per RhodiumToad: the core code represents '{}' as an array with 0
> > dimensions, whereas intarray represents it as an array with 1 dimension but
> > 0 elements
> >
> > intarray should use the same standards as the core code if possible. I
> > peered at the code and don't see anything untoward but did not have time to
> > spend on it.
>
> I just got time to look at this, and it is certainly easier to see when
> you use array_dims():
>
> SELECT '{1,2,3}'::integer[] - '{3,2,1}'::integer[];
> ?column?
> ----------
> {}
>
> SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
> array_dims
> ------------
> [1:0]
>
> SELECT array_dims('{}'::integer[]);
> array_dims
> ------------
> (null)
>
> This is part of the larger TODO item of how to handle empty
> >=1-dimensional empty arrays vs. zero-dimensional empty arrays, which is
> discussed here:
>
> https://wiki.postgresql.org/wiki/Todo#Arrays
> Improve handling of empty arrays
>
> In that thread, no one could find a way to create a 1-dimensional empty
> array at the SQL level, but thanks to intarray, you found a way. It is
> natural that intarray, being mostly used for one-dimensional arrays,
> would return a 1-dimensional empty array. However, besides being
> inconsistent, as you mentioned, there is also no way to dump/restore
> one-dimensional empty arrays, which is a larger concern.
>
> I have developed the attached patch to force empty intarray results to
> be zero-dimensional empty arrays, rather than 1-dimensional empty
> arrays. With this patch, a zero-dimensional empty array is returned:
>
> SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
> array_dims
> ------------
> (null)
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + It's impossible for everything to be true. +

> diff --git a/contrib/intarray/_int_tool.c b/contrib/intarray/_int_tool.c
> new file mode 100644
> index 8635576..fc462b2
> *** a/contrib/intarray/_int_tool.c
> --- b/contrib/intarray/_int_tool.c
> *************** resize_intArrayType(ArrayType *a, int nu
> *** 246,251 ****
> --- 246,258 ----
> int nbytes = ARR_DATA_OFFSET(a) + sizeof(int) * num;
> int i;
>
> + /* if no elements, return a zero-dimensional array */
> + if (num == 0)
> + {
> + ARR_NDIM(a) = 0;
> + return a;
> + }
> +
> if (num == ARRNELEMS(a))
> return a;
>

>
> --
> Sent via pgsql-bugs mailing list (pgsql-bugs(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-bugs

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Thanks for looking into this. Finally I'll get to test existing columns filled with a variety of array[0], '{}' and array[0]-array[0] or Null. Sorry it took so long.

What is the target production release?

A. Elein Mustain
Elein(at)varlena(dot)com
510-637-9106

> On Sep 7, 2013, at 8:44 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
>
> Applied.
>
> ---------------------------------------------------------------------------
>
>> On Wed, Sep 4, 2013 at 03:01:50PM -0400, Bruce Momjian wrote:
>>> On Wed, Dec 5, 2012 at 12:44:39AM +0000, elein(at)varlena(dot)com wrote:
>>> The following bug has been logged on the website:
>>>
>>> Bug reference: 7730
>>> Logged by: elein
>>> Email address: elein(at)varlena(dot)com
>>> PostgreSQL version: 9.2.1
>>> Operating system: Linux
>>> Description:
>>>
>>> select NULLIF('{1,2,3}'::integer[] - '{3,2,1}'::integer[], '{}'::integer[]);
>>> This returns an empty array. It should return NULL.
>>>
>>> Per RhodiumToad: the core code represents '{}' as an array with 0
>>> dimensions, whereas intarray represents it as an array with 1 dimension but
>>> 0 elements
>>>
>>> intarray should use the same standards as the core code if possible. I
>>> peered at the code and don't see anything untoward but did not have time to
>>> spend on it.
>>
>> I just got time to look at this, and it is certainly easier to see when
>> you use array_dims():
>>
>> SELECT '{1,2,3}'::integer[] - '{3,2,1}'::integer[];
>> ?column?
>> ----------
>> {}
>>
>> SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
>> array_dims
>> ------------
>> [1:0]
>>
>> SELECT array_dims('{}'::integer[]);
>> array_dims
>> ------------
>> (null)
>>
>> This is part of the larger TODO item of how to handle empty
>>> =1-dimensional empty arrays vs. zero-dimensional empty arrays, which is
>> discussed here:
>>
>> https://wiki.postgresql.org/wiki/Todo#Arrays
>> Improve handling of empty arrays
>>
>> In that thread, no one could find a way to create a 1-dimensional empty
>> array at the SQL level, but thanks to intarray, you found a way. It is
>> natural that intarray, being mostly used for one-dimensional arrays,
>> would return a 1-dimensional empty array. However, besides being
>> inconsistent, as you mentioned, there is also no way to dump/restore
>> one-dimensional empty arrays, which is a larger concern.
>>
>> I have developed the attached patch to force empty intarray results to
>> be zero-dimensional empty arrays, rather than 1-dimensional empty
>> arrays. With this patch, a zero-dimensional empty array is returned:
>>
>> SELECT array_dims('{1,2,3}'::integer[] - '{3,2,1}'::integer[]);
>> array_dims
>> ------------
>> (null)
>>
>> --
>> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
>> EnterpriseDB http://enterprisedb.com
>>
>> + It's impossible for everything to be true. +
>
>> diff --git a/contrib/intarray/_int_tool.c b/contrib/intarray/_int_tool.c
>> new file mode 100644
>> index 8635576..fc462b2
>> *** a/contrib/intarray/_int_tool.c
>> --- b/contrib/intarray/_int_tool.c
>> *************** resize_intArrayType(ArrayType *a, int nu
>> *** 246,251 ****
>> --- 246,258 ----
>> int nbytes = ARR_DATA_OFFSET(a) + sizeof(int) * num;
>> int i;
>>
>> + /* if no elements, return a zero-dimensional array */
>> + if (num == 0)
>> + {
>> + ARR_NDIM(a) = 0;
>> + return a;
>> + }
>> +
>> if (num == ARRNELEMS(a))
>> return a;
>
>>
>> --
>> Sent via pgsql-bugs mailing list (pgsql-bugs(at)postgresql(dot)org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-bugs
>
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + It's impossible for everything to be true. +
>

On Sun, Jan 19, 2014 at 04:21:42PM -0800, Elein wrote:
> Thanks for looking into this. Finally I'll get to test existing columns filled with a variety of array[0], '{}' and array[0]-array[0] or Null. Sorry it took so long.
>
> What is the target production release?

Well, I think we are going to close patching for 9.4 beta in about 1-2
months.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On Sun, Jan 19, 2014 at 04:21:42PM -0800, Elein wrote:
> Thanks for looking into this. Finally I'll get to test existing columns filled with a variety of array[0], '{}' and array[0]-array[0] or Null. Sorry it took so long.
>
> What is the target production release?

FYI, it has been committed; I just need testing help:

commit c155f654b4f755b4111bd9adb60559fe22526a10
Author: Bruce Momjian <bruce(at)momjian(dot)us>
Date: Sat Sep 7 11:44:33 2013 -0400

intarray: return empty zero-dimensional array for an empty array

Previously a one-dimensional empty array was returned, but its text
representation matched a zero-dimensional array, and there is no way to
dump/reload a one-dimensional empty array.

BACKWARD INCOMPATIBILITY

Per report from elein

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +