Problems with Byte Order Mark in SQL*Plus

[Bernhard S]

When you add a character in a Command Window that is only covered in Unicode, i.e. "≤" (smaller or equal sign), PL/SQL Developer (PSD) implicitly marks the beginning of the file with Byte Order Mark for UTF-8 (hex: EF BB BF). In PSD such a file still works fine. However when using such a file then in SQL*Plus it causes problems! You get errors like this then:
SP2-0042: unknown command "

 

[Marco Kalter]

We'll consider an option to omit the BOM.

 

[KaiAIM]

In Version 11 you can disable BOM in file format settings, thats fine.
But if I open a pck file which contains at least one unicode character ('Δ') and encoding is UTF-8 without BOM (in notepad++, visual studio I can open it and see the correct character), the Program editor of PL/SQL Developer opens it as ANSI and not as UTF-8. The unicode characters are converted to corresponding ASCII characters ('?'), but then my code is wrong, because PL/SQL Developer is saving as UTF-8 without BOM, but with the wrong characters.
When I execute the correct PCK file with SQLPlus to create the package in the database the correct unicode character is in the database. When I now open the source code from the Database the program editor shows the right character.
The problem is only, if I open a file which is encoded UTF-8 without BOM, because then the unicode characters are translated to ASCII without any notice to the user.

Regards
Kai

 

[Bernhard S]

Just came across this problem again with using the default Encoding preferences with selected options:
- "Autodetect (ANSI or UTF8)"
- "Save with BOM (unicode Byte Order Mark)"

I assume whith this setting new and therfore still empty files get started as ANSI. Then when PL/SQL Developer detects unicode characters within the files it switches to UTF8 and adds the BOM, causing the SP2-0734 error when executed in SQL*Plus.
It would be great if the user could get informed when this happens. I would also like to be able to find those unicode characters within the file easily. Then I could use alternative methods to i.e. get such characters inserted into a table without actually using unicode in the file by using CHR function for example.
Might be also helpful to show an indicator which encoding format (ANSI or UTF8, with or without BOM) is used in the status bar of each file window.