regexp_matches and regexp_split are inconsistent

I noticed the following behavior in CVS HEAD, using a pattern that is
capable of matching no characters:

regression=# SELECT foo FROM regexp_matches('ab cde', $re$\s*$re$, 'g') AS foo;
foo
-------
{""}
{""}
{" "}
{""}
{""}
{""}
{""}
(7 rows)

regression=# SELECT foo FROM regexp_split_to_table('ab cde', $re$\s*$re$) AS foo;
foo
-----
a
b
c
d
e
(5 rows)

If you count carefully, you will see that regexp_matches() reports a
match of the pattern at the start of the string and at the end of the
string, and also just before 'c' (after the match to the single space).
However, regexp_split() disregards these "degenerate" matches of the
same pattern.

Is this what we want? Arguably regexp_split is doing the most
reasonable thing for its intended usage, but the strict definition of
regexp matching seems to require what regexp_matches does. I think
we need to either change one function to match the other, or else
document the inconsistency.

Thoughts?

regards, tom lane

>
> If you count carefully, you will see that regexp_matches() reports a
> match of the pattern at the start of the string and at the end of the
> string, and also just before 'c' (after the match to the single space).
> However, regexp_split() disregards these "degenerate" matches of the
> same pattern.
>
> Is this what we want? Arguably regexp_split is doing the most
> reasonable thing for its intended usage, but the strict definition of
> regexp matching seems to require what regexp_matches does. I think
> we need to either change one function to match the other, or else
> document the inconsistency.
>

Regexp_matches behave is correct, but less usable. I thing space from
virtual begin to first char and from last char to virtual end can be
eliminated.

Regards
Pavel Stehule

On Fri, 10 Aug 2007, Tom Lane wrote:

> I noticed the following behavior in CVS HEAD, using a pattern that is
> capable of matching no characters:
>
> regression=# SELECT foo FROM regexp_matches('ab cde', $re$\s*$re$, 'g') AS foo;
> foo
> -------
> {""}
> {""}
> {" "}
> {""}
> {""}
> {""}
> {""}
> (7 rows)
>
> regression=# SELECT foo FROM regexp_split_to_table('ab cde', $re$\s*$re$) AS foo;
> foo
> -----
> a
> b
> c
> d
> e
> (5 rows)
>
> If you count carefully, you will see that regexp_matches() reports a
> match of the pattern at the start of the string and at the end of the
> string, and also just before 'c' (after the match to the single space).
> However, regexp_split() disregards these "degenerate" matches of the
> same pattern.
>
> Is this what we want? Arguably regexp_split is doing the most
> reasonable thing for its intended usage, but the strict definition of
> regexp matching seems to require what regexp_matches does. I think
> we need to either change one function to match the other, or else
> document the inconsistency.
>
> Thoughts?

I'm not sure how many languages do this, but at least perl seems to work
similarly, which makes me guess that it's probably similar in a bunch of
languages. If it is, then we should probably just document the
inconsistency.

Perl seems to document the split behavior with "Empty leading (or
trailing) fields are produced when there are positive width matches at the
beginning (or end) of the string; a zero-width match at the beginning (or
end) of the string does not produce an empty field."

Stephan Szabo <sszabo(at)megazone(dot)bigpanda(dot)com> writes:
> On Fri, 10 Aug 2007, Tom Lane wrote:
>> Is this what we want? Arguably regexp_split is doing the most
>> reasonable thing for its intended usage, but the strict definition of
>> regexp matching seems to require what regexp_matches does. I think
>> we need to either change one function to match the other, or else
>> document the inconsistency.

> I'm not sure how many languages do this, but at least perl seems to work
> similarly, which makes me guess that it's probably similar in a bunch of
> languages. If it is, then we should probably just document the
> inconsistency.

The Perl precedent is good enough for me. Documented...

regards, tom lane