Re: Practical error logging for very large COPY

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Martijn van Oosterhout <kleptog(at)svana(dot)org> writes:
> > On Tue, Nov 22, 2005 at 09:58:44AM -0500, Tom Lane wrote:
> >> The general problem that needs to be solved is "trap any error that
> >> occurs during attempted insertion of a COPY row, and instead of aborting
> >> the copy, record the data and the error message someplace else".
>
> > Actually, there are really only a few errors people want to trap I
> > imagine:
>
> You've forgotten bad data, eg "foo" in an integer field, or an
> untranslatable multibyte character. The bad-data problem is what lets
> out trigger-based solutions, or indeed anything that presumes that the
> bad data can be forced into a particular representation.

I think that's precisely the point here though. There are basically two
categories of errors:

1) Data that can be parsed and loaded but generates some sort of constraint
violation such as a UNIQUE violation, foreign key violation, or other
constraint violation.

2) Data that can't be parsed as the correct data type at all.

It would be nice to be able to have the former loaded into an actual table
where it can be queried and perhaps fixed and reloaded.

The latter clearly cannot. I would say it should just generate a log entry.

--
greg