Use of backslash in tsearch2

Bruce Momjian wrote:
>
> I backed out the patch, attached, and it has fixed the regression
> problem. What has me confused is that is looks like it is checking for
> ', then putting \, which doesn't make a lot of sense, but the regression
> output is corrected, so I just don't get it. Here is an example:
>
> test=> SELECT E'''1 \\''2''';
> ?column?
> ----------
> '1 \'2'
>
> My only guess is that the output is somehow a single-quoted string
> itself, and in fact \' should become ''. Is that right? Basically they
> are doing \' in their output, and it should be doing '', but then the
> query above would be wrong and shouldn't be using \'.

As part of the move to support standard-conforming strings and treat
backslash literally, I reviewed the tsearch2 code and found two place
that seemed to use \' rather than '', and generated the attached patch.
('' is standards conforming.) However, when I fixed the code, the
regression tests failed.

Teodor, are the new attached regression results correct? If so, I will
apply the patch and update the expected file.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> As part of the move to support standard-conforming strings and treat
> backslash literally, I reviewed the tsearch2 code and found two place
> that seemed to use \' rather than '', and generated the attached patch.

I thought we had decided that that code should not be changed. It has
nothing to do with SQL literals, and changing it will create unnecessary
backwards-compatibility problems.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > As part of the move to support standard-conforming strings and treat
> > backslash literally, I reviewed the tsearch2 code and found two place
> > that seemed to use \' rather than '', and generated the attached patch.
>
> I thought we had decided that that code should not be changed. It has
> nothing to do with SQL literals, and changing it will create unnecessary
> backwards-compatibility problems.

I don't remember any comment regarding that. I think it does relate to
SQL literals because it is creating a literal inside a literal. Also, at
the time this was a core-only discussion and I am hoping from a comment
from the tsearch2 folks.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Bruce Momjian wrote:
> >
> > I backed out the patch, attached, and it has fixed the regression
> > problem. What has me confused is that is looks like it is checking for
> > ', then putting \, which doesn't make a lot of sense, but the regression
> > output is corrected, so I just don't get it. Here is an example:
> >
> > test=> SELECT E'''1 \\''2''';
> > ?column?
> > ----------
> > '1 \'2'
> >
> > My only guess is that the output is somehow a single-quoted string
> > itself, and in fact \' should become ''. Is that right? Basically they
> > are doing \' in their output, and it should be doing '', but then the
> > query above would be wrong and shouldn't be using \'.
>
> As part of the move to support standard-conforming strings and treat
> backslash literally, I reviewed the tsearch2 code and found two place
> that seemed to use \' rather than '', and generated the attached patch.
> ('' is standards conforming.) However, when I fixed the code, the
> regression tests failed.
>
> Teodor, are the new attached regression results correct? If so, I will
> apply the patch and update the expected file.

Updated patch attached. The previous one was reversed.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

>> Teodor, are the new attached regression results correct? If so, I will
>> apply the patch and update the expected file.

Patch isn't full, simple test (values are took from regression.diffs):
# create table tt (tv tsvector, tq tsquery);
# insert into tt values (E'''1 \\''2''', NULL);
# insert into tt values (E'''1 \\''2''3', NULL);
# insert into tt values ( E'''1 \\''2'' 3', NULL);
# insert into tt values ( E'''1 \\''2'' '' 3'' 4 ', NULL);
# insert into tt values ( NULL, E'''1 \\''2''');
# insert into tt values ( NULL, E'''1 \\''2''');
# insert into tt values ( NULL, E'1&(''2''&('' 4''&(\\|5 | ''6 \\'' !|&'')))');
# insert into tt values ( NULL, E'1&(''2''&('' 4''&(\\|5 | ''6 \\'' !|&'')))');

and try dump table and restore:
ERROR: syntax error
CONTEXT: COPY tt, line 5, column tq: "'1 ''2'"

PS I'm not subscribed to -patches, so I post to -hackers

--
Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
WWW: http://www.sigaev.ru/

> Patch isn't full, simple test (values are took from regression.diffs):
> and try dump table and restore:
> ERROR: syntax error
> CONTEXT: COPY tt, line 5, column tq: "'1 ''2'"
>

Attached cumulative patch fixes problem, but I have some doubts, is it really
needed?

--
Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
WWW: http://www.sigaev.ru/

Thanks. Yes, it is need for two reasons. In 8.2 you can set
standard_conforming_strings to on, meaning \' is really treated as \ and
', and because some encodings now can't support \' for security reasons,
though I don't think tsearch2 supports those multibyte encodings.
Anyway, applied to 8.2 only, not backpatched. Thanks.

---------------------------------------------------------------------------

Teodor Sigaev wrote:
> > Patch isn't full, simple test (values are took from regression.diffs):
> > and try dump table and restore:
> > ERROR: syntax error
> > CONTEXT: COPY tt, line 5, column tq: "'1 ''2'"
> >
>
> Attached cumulative patch fixes problem, but I have some doubts, is it really
> needed?
>
>
> --
> Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
> WWW: http://www.sigaev.ru/

[ application/x-tar is not supported, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +