Hyphenation and Ligatures

Scrivener Scrivener for macOS
1

(First off, I realize that the typography is handled by OS X. I was however unable to replicate this in TextEdit, and it also results in an application freeze. See below.)

I am using Scrivener 2.1 (14118) and OS X Lion 10.7.1. I just encountered a very strange bug where seemingly random sections of text became corrupted.

Here is an example I was able to generate:

example.png

I was eventually able to reliably replicate this by causing automatic hyphenation to try to split a ligature such as the “ff” in “different”. In this case I am using the font Garamond Premiere Pro, but I was able to replicate it with other fonts as long as they include the ligature.

I was not able to get the same thing to happen in TextEdit. It seems that TextEdit avoids this issue by refusing to hyphenate between the “ff” ligature even when it would normally do so. Obviously the better behavior would be to change from the ligature to two seperate "f"s and insert the hyphen, but I was unable to get TextEdit to hyphenate between them.

After continuing to type the example I inserted above, the application became unresponsive for a time and then later recovered. In the console I found a bunch of entries like this:

9/14/11 8:42:49.653 PM Scrivener: An uncaught exception was raised 9/14/11 8:42:49.653 PM Scrivener: *** NSRunStorage, _NSBlockNumberForIndex(): index (478) beyond array bounds (477) 9/14/11 8:42:49.657 PM Scrivener: ( 0 CoreFoundation 0x95d1de77 __raiseError + 231 1 libobjc.A.dylib 0x97397149 objc_exception_throw + 155 2 CoreFoundation 0x95c85e89 +[NSException raise:format:arguments:] + 137 3 CoreFoundation 0x95c85df9 +[NSException raise:format:] + 57 4 AppKit 0x997606e0 _NSBlockNumberForIndex + 83 5 AppKit 0x9975fe64 -[NSLayoutManager(NSPrivate) _invalidateGlyphsForExtendedCharacterRange:changeInLength:includeBlocks:] + 531 6 AppKit 0x99768457 -[NSLayoutManager(NSPrivate) _fillLayoutHoleForCharacterRange:desiredNumberOfLines:isSoft:] + 751 7 AppKit 0x99775b8d _NSFastFillAllLayoutHolesForGlyphRange + 1121 8 AppKit 0x99766461 -[NSLayoutManager textContainerForGlyphAtIndex:effectiveRange:] + 187 9 AppKit 0x99d66532 -[NSTextView(NSPrivate) _ensureLayoutCompleteToEndOfCharacterRange:] + 129 10 AppKit 0x99d72448 -[NSTextView(NSSharing) didChangeText] + 178 11 Scrivener 0x00149e58 -[KMBTextView didChangeText] + 46 12 AppKit 0x99d3ae0a -[NSTextView insertText:replacementRange:] + 2218 13 AppKit 0x99f19223 -[NSTextInputContext insertText:replacementRange:] + 430 14 AppKit 0x99f168fd -[NSTextInputContext handleTSMEvent:] + 2654 15 AppKit 0x99f15bca _NSTSMEventHandler + 214 16 HIToolbox 0x98e07e54 _Z22_InvokeEventHandlerUPPP25OpaqueEventHandlerCallRefP14OpaqueEventRefPvPFlS0_S2_S3_E + 36 17 HIToolbox 0x98c8382b _ZL23DispatchEventToHandlersP14EventTargetRecP14OpaqueEventRefP14HandlerCallRec + 1602 18 HIToolbox 0x98c82ca8 _ZL30SendEventToEventTargetInternalP14OpaqueEventRefP20OpaqueEventTargetRefP14HandlerCallRec + 482 19 HIToolbox 0x98c97ac9 SendEventToEventTarget + 76 20 HIToolbox 0x98f07382 SendTSMEvent + 74 21 HIToolbox 0x98f07c7d SendUnicodeTextAEToUnicodeDoc + 753 22 HIToolbox 0x98f06b9a TSMKeyEvent + 1007 23 HIToolbox 0x98cac85b TSMProcessRawKeyEvent + 2795 24 AppKit 0x99f19a7a -[NSTextInputContext handleEvent:] + 1257 25 AppKit 0x99dc0ccc -[NSView interpretKeyEvents:] + 220 26 AppKit 0x99d2dabe -[NSTextView keyDown:] + 676 27 AppKit 0x9978c53c -[NSWindow sendEvent:] + 10892 28 AppKit 0x99723b23 -[NSApplication sendEvent:] + 4936 29 Scrivener 0x00041075 -[SCRApplication sendEvent:] + 412 30 AppKit 0x996b552b -[NSApplication run] + 1000 31 AppKit 0x99948b54 NSApplicationMain + 1054 32 Scrivener 0x00002b41 _start + 208 33 Scrivener 0x00002a70 start + 40 )

I will email you trimmed logs.

I was able to verify that this does not occur on OS 10.6 with Scrivener 2.1.

Because of the involvement of the text rendering system, I don’t know if you will be able to fix this or if Apple will have to do something, but for now the only work around seems to be to turn off either automatic hyphenation or ligatures.

Thanks,

Michael

2

I found the following post about a bug that seems to be the same. (I’d searched for similar bugs, but focused on hyphenation and ligatures, so didn’t find it even though it was right there below. Oh well.)

https://forum.literatureandlatte.com/t/strange-character-substitutions-in-full-screen-mode/14207/1

I tried the newest beta (2.1.0.5 15145) and it did not help.

3

Hi Michael,

Thanks for this. This does indeed seem to be a Lion bug, although fortunately I can work around it by following TextEdit’s lead. Essentially, hyphenation isn’t an on/off thing; rather, a “hyphenation factor” is set, between 0.0 and 1.0. When hyphenation is turned on, I had been setting the factor to 1.0, which means that hyphenation is always attempted. Looking at TextEdit’s source code, however, it seems that Apple only sets a hyphenation factor of 0.9 when hyphenation is turned on in TextEdit. Thus, for 2.1.1, I have reduced the factor to 0.9 so that it won’t attempt to break “different” after “dif”.

Thanks and all the best,
Keith

4

Many many thanks!

I’m glad there is a simple work around, though I do hope Apple gets this fixed since the ability to put hyphens in more frequently certainly makes for more attractive text.

Anyway, load off my mind. You can’t (OK, I’m sure you can) imagine the heart attack I suffered when I first found the “corruption” scattered randomly through my draft.

Best of luck with your future efforts. Great work as always,

Michael

5

I’ve submitted a bug report on this to Apple (bug #ID 10132042). I’ll let you know if I ever hear anything back.

Thanks again and all the best,
Keith

6

Apple just got back to me to let me know that this is a known issue and is filed under the original bug ID # 10022000, so hopefully that means that they are on it.

All the best,
Keith