After an upgrade we are experiencing error 12112 on one of the databases
"Unable to delete old values for the index of a word-indexed field (index TOYART9 of table LB-TOYART). The index may be damaged. (12112)"
Our cpinternal was changed to utf-8, our main database is utf-8, the error is on a db that wasn't converted.
proutil idxfix (option 3 with fix) and proutil idxcheck report no errors
I found several PKB entries describing similar issues when cpinternal isn't capable of dealing with the database characters. But here I'm confronted with the reverse, utf-8 can deal with all characters in iso8859-1.
I know this will be gone if we convert the db to utf-8 but I'm looking for alternatives.
The proutil convchar conversion takes too long : 12 hours on a slightly less powerful test server, this is without the required(?) index rebuild.
Thanks for all advice upfront.
PS: I created a Tech support case for this issue as well
I'm currently working around the issue by handling the stop condition caused by this.
do on error undo, throw,
on stop undo, retry: if retry then undo, throw new progress.lang.apperror("stop", 0). /* failing code comes here */ end.
Not ideal but as a work-around it's enough for now. It affects 30 out of 120000 items and we know which ones are affected.
Out of interest is code page 1252 an option to try? I've found that has fixed a similar issue in the past.
Hi James,
thanks for the suggestion.
Is 1252 an option ?
In the past we had various issues with cpinternal iso8859-1, mainly utf-8 xml file processing that failed .
In my experience processing xml files requires -cpinternal utf-8
I'll have to check the code for XML processing.
I can reproduce the issue on the sports2000 db. I already passed this to tech support, included here in case someone wants to reproduce it . I tested on linux in 11.5.1, 11.6 and 11.6.2
# steps to reproduce
prodb sports2000 sports2000 proutil sports2000 -C convchar convert iso8859-1 pro sports2000 -cpinternal iso8859-1 -p add_feedback.p -b | tee pro sports2000 -cpinternal utf-8 -p del_feedback.p -b | tee
/* add_feedback.p */ create feedback. feedback.comment = "360" + chr(176, "iso8859-1") + " (degrees) issue".
/* del_feedback.p */ find feedback exclusive-lock where comments begins "360". delete feedback.
Result
Unable to delete old values for the index of a word-indexed field (index Comments of table Feedback). The index may be damaged. (12112)
I'm currently working around the issue by handling the stop condition caused by this.
do on error undo, throw,
on stop undo, retry: if retry then undo, throw new progress.lang.apperror("stop", 0). /* failing code comes here */ end.
Not ideal but as a work-around it's enough for now. It affects 30 out of 120000 items and we know which ones are affected.
you are correct, utf-8 can deal with all characters in iso8859-1. but the encodings are not the same.
iso 8859-1 is an 8 bit character set. all characters are one byte.
in utf-8, 7 bit characters are one byte. all others are multiple bytes. so only some single byte 8859-1 characters map directly to utf-8 single byte characters.
take the degree symbol:
in 8859-1 the degree symbol is encoded as a single byte that has the value 0XB0 (176 decimal)
in utf-8 the degree symbol is encoded as two bytes: 0XC2 0XB0.
all the iso 8859-1 characters above 0XFF are encoded as two bytes in utf-8.
in your example, you have
> feedback.comment = "360" + chr(176, "iso8859-1") + " (degrees) issue".
This is invalid for cpinternal utf-8 because it is not a valid utf-8 encoding for the degree character.
You have to convert the existing data to the proper character set.