Bug: case conversion DOTTED CAPITAL I and DOTLESS SMALL I

[postmaster]

Hi,

[Textpad v5.2.0 Trial]

Textpad fails to correctly perform when asked to lowecase (Ctrl + L) DOTTED CAPITAL I.

Same goes for uppercasing DOTLESS SMALL I

This is how it should have been:

Lowercase(DOTTED CAPITAL I) --> i
Uppercase(DOTLESS SMALL I) --> I

Could you please look into this.

[bveldkamp]

Could you tell us how to get a dotted capital i (U+0130) in TextPad in the first place? IOW, which codepage contains that character (Win1252 doesn't for sure)
When I try copy/paste from CharMap, it converts İ to I; Opening a file containing that character, TP warns about a conversion to the Windows 1252 codepage.

Berend

[ben_josephs]

ISO 8859-9; CP 1254; Turkish:
Configure | Preferences | Document Classes | <Class> | Font | Script: Turkish
or
View | Document Properties | Font | Script: Turkish.

Code: Select all

           CP 1252     CP 1254
           ISO 8859-1  ISO 8859-9
hex   dec  Western     Turkish
----  ---  ----------  ----------------
0xDD  221  �           DOTTED CAPITAL I
0xFD  253  ý           DOTLESS SMALL I

TextPad does behave as the OP says.

[bveldkamp]

Yes, I see it now. Just to make sure, which language settings did you set in the Control Panel's 'Regional Settings' dialog?

Berend

[postmaster]

It really does not matter what the language settings in the Control Panel's 'Regional Settings' pane is.

My 'Regional Settings' is English; but, this does not mean much, on which I'll elaborate a little later.

'DOTTED CAPITAL I' and 'DOTLESS SMALL I' are only peculiar to Turkish/Turkic languages so, whenever you encounter them, there is only one way to do proper case conversion.

i.e.

Lowercase(DOTTED CAPITAL I) --> DOTTED SMALL I

Uppercase(DOTLESS SMALL I) --> DOTLESS CAPITAL I

This is always valid (and simpler to implement) for all texts.

Yet, only if you do know that that particular (piece of) text is in Turkish, can you assume the following to be valid:

Lowercase(DOTLESS CAPITAL I) --> DOTLESS SMALL I

In order to handle this 'quirk', it will require TextPad to have a 'per selection' flag that the user needs to set before properly lovercasing the selected text.

And, no, TextPad cannot rely on the global 'Regional Settings'; neither does having a TextPad-global setting help, nor is a 'per document' setting is useful. It has to be 'per selection'.

And the reason is simple: Since we are now all dealing with Unicode, the whole text could a mixture of languages with just a portion of it in Turkish.

Actually, this sort of thing (i.e., language-dependent case conversion) is not peculiar to Turkish. There a quite a few others (Spanish, Dutch, even German to name a few) which TextPad needs to cater for.

Who said Unicoding a text editor would simply be letting fonts from different locales display nicely

[bveldkamp]

I think it actually does matter which language settings you have for Windows. Here's my guess (of course I don't have TP's source code, but it might explain what you're seeing):

I think TP uses the CharUpperBuff() and CharLowerBuff() Win32Api calls to convert case. These methods don't take a codepage as a parameter, but rather rely on the regional settings for Windows. What's more, if you look at the documentation, it says

Note that CharUpperBuff always maps lowercase I ("i") to uppercase I, even when the current language is Turkish or Azeri.

and

Note that CharLowerBuff always maps uppercase I to lowercase I ("i"), even when the current language is Turkish or Azeri

http://msdn.microsoft.com/en-us/library ... 85%29.aspx
http://msdn.microsoft.com/en-us/library ... 85%29.aspx

Your Unicode comments do not apply to TextPad, since TP unfortunately only deals with single byte character sets, hence the 'Script' setting in the document's properties.

Berend

[postmaster]

Shame MS doesn't seem to say anything about lowercasing 'dotted capital I'..

And, about Unicoding.. I would have expected a product that has reached to v5.x to be able to be properly unicodified..