Still having substitution problems!!

Hello,

Weird problem here, happening for the second time. I’ve got Preferences > Corrections set to automatically replace double-hyphens and straight quotes with em-dashs and smart quotes, as well as have got the same substitutions set up in System Preferences > Language and Text. And sure enough, Scrivener was doing exactly that, and everything was fine … until I rebooted and restarted Scrivener (that’s what fixed this exact same problem a week ago). Now, even though those preferences are STILL set the exact same way, the program no longer makes the appropriate corrections; the double-hyphens remain exactly that; as do straight quotes. At least – in Page View. If you switch to Composition mode, then it DOES do the replacements. Back to Page View – nada. I tried closing down and restarting Scrivener, but the behavior just persists. Composition mode does the correct substitutions, but not Page View. Neither does the main editor in “normal view.” Any ideas?

–Andy H.

Hi Andy,

This is very strange - I still can’t reproduce this. Does this happen on the first page of a document, too, or only on subsequent pages (this is pertinent because subsequent pages copy their attributes from the first, so it could be a bug whereby they are not grabbing the attributes properly)?

Could you please attach your preferences to your reply so that I can test them out? You can save them from Scrivener by clicking on “Manage…” in the bottom-left of the Preferences panel and choosing “Save All Preferences…” This will save them as a file that you can attach here, and then I’ll be able to download it and load it into my copy of Scrivener to try to replicate your setup.

Thanks and all the best,
Keith

Sure Keith … but how do I attach a file here in the forums?

Use the “Upload attachment” tab beneath the area where you type your message.

Well, I went to “Upload attachment” and then clicked on “Choose File” and then chose the appropriate .prefs file, and then hit “Add the FIle”, but the forum stalls out about about four or five minutes of trying to refresh the page…

Will try again later, I guess.

Send the file to support AT literatureandlatte DOT com if you’re having problems with attaching it via the forum, then.
Thanks,
Keith

Well, I had trouble sending the email because the resulting preferences file was 102MB and the smtp server kept rejecting it. So I zipped it down to 36MB (which the mail server still kept rejecting), and put it up on Google docs and shared it with support@literatureandlitte.com; I then sent an email there, with a link in it to the file on Google docs. Hope this works,

–AH.

Thanks, I’ll take a look at it later - that is an enormous preferences file, though - are you using lots of images for backgrounds and textures, by any chance?
All the best,
Keith

As a matter of fact, yep, I sure am. Didn’t know that the program actually stored the images themselves inside the preferences file, though! Specifically, I use some really high resolution images for my background and paper images, figuring that high-res images would look better when using full-screen view. Had I known that Scrivener copied those over into the actual preferences file, I would’ve turned them off before saving it out. :smiley:

Okay. So far, I’ve done some experimenting, and the only sure fire way to fix the problem is to delete Scrivener, delete the “Scrivener” folder under Library/Application Support, and then empty the trash, and then reinstall Scrivener from the App Store. I tried mucking about inside the .scriv file for the document, as well as tried to isolate the various preferences files under Library/Application Support/Scrivener, all to no avail; couldn’t seem to nail down where precisely the problem was going on.

Another thing I noticed, though, if it helps – not only was it not doing the substitutions (even though they were checked in Preferences > Corrections), it was also not underlining any spelling errors any longer; nor was it checking grammar, either, even though it too was turned on. It was, however, correctly substituting ordinals, such as 1st, 2nd, 3rd, etc., and it was detecting [[Links]], too. Curioser and curioser…

Sorry, I haven’t had time to look at the preferences file yet, but this extra information is interesting - several users have reported an issue on Lion where spell-checking stops working, and they’ve had a helluva time getting it running again. On Lion and Snow Leopard, spell-checking and auto-correction are connected - they are all part of the OS X text engine’s “text checking” system, so it makes sense that if the spell-checking bug hits then the auto-correction would break too.

All of the spell-checking and auto-correction stuff is handled at the OS level, not by Scrivener. When you tick these items in the Preferences, Scrivener simply says to the OS X text system, “Okay, turn spell-checking on,” or, “Turn em-dash correction on”. Ordinal correction is my own code, on the other hand, and not handled by the standard OS X text system; likewise, [[links]] correction is my own code too. So it makes sense that they wouldn’t be affected if this is a problem with the OS methods.

I’ve googled a bit when other users have reported spell-checking problems, and it seems that this has affected other programs too. My feeling is that it is a Lion bug connected to Lion’s new state-saving code. Lion tries to monitor an awful lot of interface states and remember them. Scrivener handles most of this itself and tells the OS not to monitor such things (and really, Lion shouldn’t be monitoring anything without asking the receiving program, but I have noticed it saving things such as scroll position without asking). So, what I think might be happening is that Lion has some private code somewhere that is saving what it thinks is the spell-checking and auto-correction state for a particular project and that is overriding the actual settings, because Lion is telling the text-checker in the text system not to do these things.

Part of my reason for thinking this is that users who have experienced the spell-checking bug have sent e affected projects but they have always worked fine on my machine. These users have also found that opening the project on another account causes it to work fine, as does copying everything in it into another project on the same account. OS X often saves settings by saving the setting against the path to the project. So what I think is happening is that Lion is saving some secret setting somewhere saying not to do text correction for the project at this particular path.

If that is the case, then here’s something to try: close the project and in the Finder make a copy of the project and then open the copy. Does the copy have the same problem?

Thanks and all the best,
Keith

I’ll give this a try next time the problem comes up. For now, I’ve got it working again, via the delete and reinstall. Since it’s working now, I can’t reproduce the problem anymore. But if it happens again – and if this is a Lion thing, then it probably will – I’ll definitely try that and see if it works.

One thing, though: If the state-saving data is relative to the path of the document why would reinstalling the program do any good at all? And, why would the substitutions all still work in Compose mode but not Page View? Not trying to indict Scrivener for a problem it more than likely is not causing (if this is a Lion thing, I would hardly be surprised – Lion does some things that are just plain weird), just trying to help narrow down the problem, is all.

Hello,

I know you guys are probably busy with other, much-higher-profile problems, but I still thought I’d report that Scrivener is acting up in the text-substitutions-and-spell-checking department again. It’s yet again stopped doing substitutions (em-dashes, smart quotes, etc.) as well as has quit spell-checking documents. For some reason it just stopped working all of a sudden. Both features quit at the same time. I tried the solution you recommended on the forums – i.e., copying the project file to another location (I even tried renaming it), and then reopening the project – but that didn’t fix it. I even tried it two or three times, with different paths each time. (Substitutions are also not working in any new documents I create within the project, either.) Is there anything else I should try? I guess I could always delete the app (and its Application Support subfolder) and then reinstall it again (which seems to work almost every time this happens), but that seems like such a kludgy solution. Just doing a quit-and-relaunch doesn’t solve it, either. The last thing I remember doing before noticing the problem was messing around with the Binder, reorganizing my document’s flow and adjusting my View from Page View, to Wrap mode, back to Page View a couple of times. But I don’t see how that could’ve caused it. Especially since … get this … both features – substitution and spell-checking – are still working perfectly fine in Compose mode and in QuickReference panels! (But not the regular Editor, in either windowed or full screen mode.) I just can’t figure it out. If I knew what combination of actions on my part was triggering it, then I’d avoid that combination of action, if possible. Oh, and if it helps narrow down the problem, this seems to only happen every three or four days or so, and almost always after I’ve messed about in the View menu, or after I’ve switched the current document displayed in the Editor by clicking on another one in the Binder.

So, yeah. Just wondering if there’s been any progress on this, or if it’s still just a huge mystery. The writer in me eagerly awaits any news, and the techie in me is baffled as to the actual cause…

Yours truly,
Andy H.

P.S. – I just noticed something that might be contributing to the problem. In fact, this may be the missing diagnostic ingredient. Maybe. On my Mac, I am using Dropbox, but in a rather unique way: There is no “Documents” folder in my home directory. Instead, inside my home directory, I have a symlink to the real Documents folder, which lives inside the ~/Dropbox folder. However, I am not accessing the file from any more than one computer (or device) at a time; thus, I’m not really wholly sure why this would be causing a problem. (And even if Dropbox is the culprit, why would a removal and reinstall serve to “fix” the problem?) As a test, I’m going to try moving my documents folder outside the Dropbox and back into my home directory, and see if that solves things.

P.P.S. – Nope, that wasn’t the problem. Went through all the trouble of setting things back to the way they were “supposed” to be, that is, with the Documents folder living directly in the home directory, but that did not fix the issue, even after a reboot. Looks like removing then reinstalling Scrivener will have to do.

P.P.P.S. – Okay, now not even THAT fixes it. Removed, deleted App Support folder, reinstalled, same problem, no matter what project I open, no matter where the project is located or moved to, no matter what document I’m in within the Project, no matter if I restore the Default preferences or trash the preferences files – nothing I do fixes it. (And the substitutions and spell checking are indeed working in other apps.) Something is seriously wrong, here. Substitutions and spell-check are still working fine in Compose mode and QuickRef panels, but not in the main Editor, where I need them.

There. FINALLY, I managed to fix track down the problem and FIX it!

What I had to do was open the main .scriv file (Show Package Contents) for the Project, drag all the files in the “Settings” folder into the trash, and then reopen the project again. And boom. spell-check and substitutions were working once again. It was SOMETHING in one of those files that was causing the problem, though I’m not sure what the heck it was or could’ve been. Anyways, I got it fixed now, and with a minimum of fuss.

Thanks for the help on the issue,
Andy H.

P.S. – It remains a mystery what caused the problem in the first place. Maybe some sort of conflict, or something in the prefs files was getting reset somehow?

Well, it did it again this morning. Quit working, that is. No substitutions, no spell-checking in anything but QuickRef and Compose. No idea what caused it. It was working just fine last night after I trashed the files in the Project Settings folder, but as soon as I got up this morning – I think the computer went to sleep – it was back to the old thing: No substitutions, no spell-checking in the main Editor, in either Page View or Wrap mode.

To be honest, as I outlined in my email to support@literatureandlatte.com, I will probably soon be switching back to Word or Pages, and writing an appropriate (given the tendency of this feature to hose itself) review on the App Store. Scrivener has such immense promise, and such great potential – IF the bugs can be worked out. If they can, I will come back to it in a heartbeat – I love the way its me organize my work and the way it lets me do things – but for right now, I need to get some real writing done, and I’m afraid I just can’t do that without these specific features; my novel is currently languishing, and needs attention. It will be a real pain to go back to the “old way” of doing things via a regular word processor, but I suppose that if I have to, then I have to. I’ll be eagerly awaiting a response, since I do not want to ditch Scrivener quite yet, especially if a solution can be found.

To recap: I’ve tried just about everything. Copying/moving/renaming the Project file, trashing the ~/Library/App. Support/Scrivener folder, trashing the contents of the Settings folder within the .scriv file, deleting and reinstalling the app, checking/double-checking the System Preferences > Language & Text settings, checking/double-checking Scrivener’s own Preferences > Corrections panel, splitting the Editor, switching documents in the Editor view, switching between Page View and Wrap mode, going full-screen and back … NOTHING fixes it. Before, I was able to fix it by deleting the contents of the Settings folder within the .scriv file, or by reinstalling the app. Now it seems permanently broken.

Any feedback, hints, suggestions, or – better still – solutions would be very much appreciated at this point, as this is really turning out to be a very frustrating user-experience, all the more so because it started off so well.

Very frustratedly yours,
Andy H.

Hi Andy,

I do understand your frustration, but I’d be very grateful if you didn’t give us a bad review on the App Store over this one issue, given that I am fairly confident that this is an Apple Lion bug and not something in Scrivener itself, and it would be hugely frustrating to be marked down for something over which we have no control. Of course, you must do what you feel is right, but I think such threats are a tiny bit mean given how much help I and others have been trying to give you with this and anything else you have asked about.

As I say, the trouble is that I cannot reproduce this bug and, although a handful of users (but only a handful, out of thousands) have reported problems with spelling stopping working since upgrading to Lion, you are the only user who has so far reported substitution problems as well.

When you updated to Lion, did you do a clean install or just update? Sometimes updating can cause odd problems whereas a clean install works fine. One user who reported spelling issues did a clean install of Lion and this cleared the issues, which would suggest a problem in Lion’s spelling system. As I’ve explained, OS X is responsible for handling all of these things - substitutions and spell-checking. As long as you have them ticked in the “Corrections” pane of Compile (and in the Edit > Spelling and Grammar menu, where you can override whether spelling is turned on per-project), then all Scrivener does is look up this setting and say to OS X, “Hey, do spell-checking,” or, “Hey, you can use substitutions in this text area.” From there on, it’s entirely OS X’s responsibility to handle these things.

Here’s one more thing to test, which will hopefully give me a better idea of what is going on:

  1. Download this version of Scrivener:

literatureandlatte.com/dlbet … llTest.zip

  1. Unzip the file and launch the copy of Scrivener it contains (don’t replace your regular version of Scrivener, but do make sure it is closed before launching this version). You can check you are running the correct version by going to Scrivener > About Scrivener - the version number should be “2.2.0.5 (15970)”.

  2. Make sure that substitutions are turned on the preferences and that spell-checking is turned on in the Edit > Spelling and Grammar menu and type away, ensuring that these things aren’t working as they should be.

  3. Once you typed for a while and confirmed that substitutions and spelling aren’t working, in the Finder go to ~/Library/Application Support/Scrivener/Logs (you can access the ~/Library folder, which is usually hidden, by holding down the Option key while opening the “Go” menu in the Finder).

  4. There you will find a file entitled Console.log - attach it to your reply or open it in a plain text editor such as TextEdit and copy and paste the contents into your reply.

I have added some logging code which will tell me whether these things are properly turned on both in the options and in the text view. If they are turned on the options but not in the text view, this will indicate a bug in Scrivener somewhere, with the options somehow not being turned on in the view properly. If they are turned on in both, this will indicate that Scrivener has done things properly but that OS X is playing up.

Also, could you please put together a small video showing this bug? You can take a video using QuickTime 10. In the video, just show that things are turned on in the appropriate places and then type to show what isn’t working (and show the substitutions that you have set in System Preferences that aren’t working when you type).

This should get us a bit further.

Regards,
Keith

Well, okay, I admit, maybe the threat of a bad review was a bit out there. To be honest, I doubt I could bring myself to do that, considering how just plain awesome-sauce Scrivener is in every other department. And yes, you have been extremely attentive and interactive – the most I’ve ever seen any developer ever be – and have gone to great lengths to provided answers, assistance, and help. So any review I did would of course factor that in, and believe me, it’s no small factor. (I will be doing a review, but only after we’ve gotten this problem licked. Thus, it’ll be much fairer … though really, Scrivener is a fantastic app). But anyway to answer your questions:

When I updated to Lion, I did not do a clean install. I upgraded from Snow Leopard, via the App Store Lion installer. I would hate to have to go that far, seeing as how it would entail then restoring everything from a Time Machine backup – plus, I don’t know WHERE the heck Lion stuck its installer on the system – but I will do as you ask with the test version of Scrivener you’ve attached, and will attach the log file as shown. It will take me a bit to do the QuickTime video (I didn’t even know QuickTime X had the ability to do screen-caps), as I have to go to class tonight and before then, must do homework. (Finishing a paper in Scrivener :slight_smile:.) I will post another reply here when I get done, hopefully before I leave … but if not, it will not be before this evening.

All the best,
A.H.

P.S. – I just installed the test version of Scrivener you sent along, and guess what? Both substitutions and spelling worked right off the bat. So what to do next? Do you still want me to send along the log file?

Hi,

I appreciate your understanding (and the kind words!). Hopefully you shouldn’t need to go as far as a clean install - I upgraded from Snow Leopard myself and have never had any issues, but it is a well-established fact on the Mac that clean installs tend to go better and can clear out random glitches that seem to appear on systems.

That is very strange that substitutions and spelling are working in the new version. No need to send the log file yet, then, as if it’s all working I know exactly what will be in there. The only thing different in this version (aside from some minor unrelated fixes) is that it logs every time a character is typed, which could slow down typing a little (as it’s only a test). It’s weird if slowing down typing is preventing the bug from triggering, though (but not entirely unprecedented).

Try working in this version for a while (but bear in mind that you might find that typing lags a little because of all the logging it is doing) and see if you can get the bug to trigger again. If it doesn’t, then try this version:

literatureandlatte.com/dlbet … NoLogs.zip

This is exactly the same but without the logging (so there’ll be nothing in Console.log for this one). If the bug happens as soon as you switch to this one but didn’t happen in the other one, then that indicates that the logging itself is stopping the bug from triggering and I’ll think of another way of logging what I need to know. Give the logging version a little while, though.

Thanks and all the best,
Keith

P.S. Recording video with QuickTime was new with Lion, I believe.

Well, I downloaded the second one, fired it up, and lo, substitutions and spell-check were working right off the bat perfectly. But, I confirmed they still will not work in the version I have in /Applications. I suppose I can try working in the logging version a while, as you suggest, to see if I can get the bug to manifest.

I think it’s interesting, though, that yesterday, I was able to fix the bug by clearing out all the files in .scriv-file/Settings, but that that didn’t fix it today, and yet the only thing different was that the computer went to sleep between the two times. I don’t know if that’s significant, but it certainly is odd.

P.S. – Just tried installing the latest version you sent me (the one without logging) over the top of my existing Scrivener install, and opening the same projects I tried working on this morning. Guess what? Perfect substitutions and spell-check.

P.P.S. – When I fired up this version (having replaced my normal install of Scrivener), something odd happened. When it opened, it opened up with the last project I had worked on (which is cool), and in which substitutions were NOT working before. Now they are, however! But now that I’ve created a new project alongside that one, and tried to get it to do substitutions and spell-checking, it won’t do it! So it’s working in one of the projects (where it wasn’t before), and is now NOT working on any new projects I create! And the weird thing is, they’re open side-by-side! Could it be a Dropbox issue? The way I’ve got it set up, my actual “Documents” folder lives in ~/andyhainline/Dropbox, and the Documents in ~/andyhainline is just a symlink. I have Scrivener saving everything there; I figured it wouldn’t be an issue since I’m the only one who uses this particular Dropbox account, and it’s never open on more than one computer/device at a time.

P.P.P.S. – I closed one of the projects and then opened it again, and now substitutions and spell-check are working in BOTH of them. Weird, huh? But then I open yet another project, and suddenly, they work in the other two, but not this one. Odder and odder…

Curiouser and curiouser - I suspected the problem wasn’t gone, though, seeing as I hadn’t changed anything in this regard. Okay, keep working with the logging version - the first one I sent. Then, as soon as it doesn’t work in a project, grab the contents of the Console.log file (following the instructions in my previous post) and paste it into your reply. That will give me a better idea about what is going on.

As well as the above, also try the following the next time the problem occurs:

In the Finder, ctrl-click on your .scriv file (one that is having problems) and select “Show Package Contents”, and drill down into the Settings folder. Take a look for ui.plist. If it’s on Dropbox, I’m wondering if some sort of conflict is happening. It could be that Dropbox is renaming this file occasionally, thus making Scrivener think it’s empty and that the settings are turned off. I doubt it, though, because this should only affect spelling (which is a project setting) and not substitutions (which is handled by the preferences).

The Console.log results will be the most useful.

Thanks,
Keith