Re: Implementing full UTF-8 support (aka supporting 0x00)

On 03/08/16 17:23, Tom Lane wrote:
> =?UTF-8?Q?=c3=81lvaro_Hern=c3=a1ndez_Tortosa?= <aht(at)8kdata(dot)com> writes:
>> As has been previously discussed (see
>> https://www.postgresql.org/message-id/BAY7-F17FFE0E324AB3B642C547E96890%40phx.gbl
>> for instance) varlena fields cannot accept the literal 0x00 value.
> Yup.
>
>> What would it take to support it?
> One key reason why that's hard is that datatype input and output
> functions use nul-terminated C strings as the representation of the
> text form of any datatype. We can't readily change that API without
> breaking huge amounts of code, much of it not under the core project's
> control.
>
> There may be other places where nul-terminated strings would be a hazard
> (mumble fgets mumble), but offhand that API seems like the major problem
> so far as the backend is concerned.
>
> There would be a slew of client-side problems as well. For example this
> would assuredly break psql and pg_dump, along with every other client that
> supposes that it can treat PQgetvalue() as returning a nul-terminated
> string. This end of it would possibly be even worse than fixing the
> backend, because so little of the affected code is under our control.
>
> In short, the problem is not with having an embedded nul in a stored
> text value. The problem is the reams of code that suppose that the
> text representation of any data value is a nul-terminated C string.
>
> regards, tom lane

Wow. That seems like a daunting task.

I guess, then, than even implementing a new datatype based on bytea
but that would use the text IO functions to show up as text (not
send/recv) would neither work, right?

Thanks for the input,

Álvaro

--

Álvaro Hernández Tortosa

-----------
8Kdata