Oh. I think I misunderstood the problem. Sorry.
Then I’ll probably have a look at the topic of mass editing and RegEx. Thanks for the tip with Karabiner, but it looks quite complicated to me.
1 Like
Oh, just make sure there were not any crossed wires, Karabiner is a keyboard mapping tool. It is perhaps capable of writing macros and binding them to keys, but from what I understand you must be a programmer to do that (I don’t know much about it though).
Keyboard Maestro, on the other hand, is a user-friendly “build a script” type program, a bit like Apple’s own Shortcuts, but without all of the limitations. It can find text, trigger menu commands, click buttons and so on—and like I say it has a record function that can get you 90% to something that works over and over, in my experience.
1 Like
Anything new about the update that will fix the problem? It’s been a while. The workaround doesn’t help ver much as everything is already scrambled up.
That shouldn’t be happening. Once the list is fixed and the workaround is put in place, it should not happen again. Are you not fixing the list as well? Just to be clear this isn’t going to fix it for you, and it is very likely that the update that fixes the bug will not either. You will have to fix these because the formatting is damaged. So you might as well do so now, and get the workaround in place so it doesn’t happen again. You can mass delete all of these markers later with Project Replace, so it’s no big deal.
1 Like
As already mentioned, I have approximately 2,000 defective lines out of > 6,000 list entries.
I haven’t fixed anything yet. So far, I’ve only experimented a little. Using regular expressions, etc., it seems that the issue can be resolved to some extent. However, inserting DELETEME in such a large number of lists means additional work, which I would like to avoid.
That’s why I’m waiting for a fixed version. However, after more than two months, I’m starting to feel that I may be waiting a long time. Will there be a corrected version soon, or will the update be delayed until MacOS Tahoe is released?
You will hear about the fix when we post the official update and can read the change logs to see if it included. If you look at the change log page you will see we almost always publish yearly, after the Apple OS updates.
1 Like
I unfortunately just purchased Scrivener 3 and the iOS version. Making a new character sheet where I have headings followed by bullet points listing character traits I am now running into this very issue.
What I would like to add is how disastrously more severe the issue is when opening a document alternating between iOS or iPadOS and the macOS version. Not only do additional bullet points get added, they appear wildly random, multiple times over, in different formatting and styles.
In one case, simply opening the document on my iPhone, changing a single word somewhere else in the sheet changed one of the headlines (Headline 2 formatting) by adding 3 bullet points with tabs and adding weird text to it, called “<$ScrKeepWithNext>” before the original text.
Like what is going on here? I am regretting my purchase. I always thought Scrivener was handling text with care and diligence. Yet it seems that behind the scenes it’s saving files in an even worse state than Microsoft Word is, the bane of my existence and what I wanted to get away from in the first place.
I wish I had run into this issue before the purchase decision, but I had “only” written my manuscript in the trial so far, and never used bullet points.
The note that Scrivener updates “usually happen every year after Apple operating system releases” is not enough for this. This is months away. It is an application-breaking bug that is ruining people’s work and tampering with text in ways I didn’t think was possible.
Please advise.
Read up-screen. There’s diddly-squat L&L can do about it. Apple has messed up the underlying infrastructure. Howl at them.
I read everything. L&L use RTF libraries from Apple in non-standard ways as no other applications using them broke in this way (or at all).
It’s easy to blame Apple.
Hmmm. Ask for a refund.
I provided a little explanation for what we are doing here, in this post.
There is a more to it than that of course. Almost everyone that is making a tool of some sophistication is going to be using it in “non-standard ways”; that doesn’t mean a whole lot by itself. The stock text engine is extremely simple, and often does not support full RTF standards (it can’t even properly open a file with footnotes without flat-out deleting them). It does not support styles at all, and it takes considerable effort to make it do so. Thus, saying “no other applications… broke in this way” doesn’t mean much. I can count with two fingers the number of programs using the native text engine that support styles, and the other one is a dedicated word processor that naturally spends all of their coding time on the text engine, to the point that they have deeply modified (in non-standard ways!!) how the native text engine writes RTF.
What we are doing for styles is about as non-invasive as it gets. Styles are saved as a plain-text code just like the text you type in yourself. The only difference is you never see them in Scrivener because they are stripped out and turned into attributes in memory.
This is what a Scrivener RTF file looks like from Windows, opened in a word processor that doesn’t look for these markers and considers them normal text (LibreOffice):
And now here is how the Mac version saves this same exact file (after a trivial update elsewhere to force an auto-save):
This all by itself isn’t too useful to us though, because we can’t really see from this what Scrivener meant to do, or what has always worked before. All we can see is what the updated Apple RTF writer decided to do with that request. So here is a look at what the Mac version saves with a fake list. I typed in the bullets and tabs, and applied the same indent and tab stop settings Apple uses for its list formatting:
You might be wondering why this is different from the Windows save pattern. The difference is in whether the terminal newline on a paragraph that contains a style is included within that styled range.
In practice it doesn’t make a lot of difference, but it can lead to some minor behavioural differences between platforms—such as how Find by Format selects this line, and whether deleting it would also remove the line fully (advancing the list up) or leave an empty line behind. Neither is really right nor wrong, but in this case, including the newline is confusing the RTF writer.
Both are valid approaches, but may produce different programmed behaviours off of functions that work with this attribute. That said, the way Windows works is valid, but unconventional, and has been a point of contention all along, because it is a more standard practice to include the entire line within the styled range, not just up to the penultimate character.
So yes, given the conventional behaviour being followed on the Mac, it has always written lists the way you see them above, even with real lists instead of fake lists, and up until recently that has worked fine, because Apple’s writer was evidently aware of those conventions that they recently have come to no longer respect in all scenarios.
It’s easy to blame Apple.
That’s not how this works, and I don’t really understand that sentiment either, what does that matter? Around here we identify the sources of the problem and spread that information so that people can know what to expect, and potentially how to avoid problems.
In this case, a minor system update to macOS broke every version of Scrivener. That is useful information in two ways: (a) because it informs you that you cannot install an older version of Scrivener to get around it, and (b) if you have the means to do so, you can roll back your OS to an update that doesn’t break it.
(This is all, incidentally, why back when I used a Mac productively, I always stayed one year or more behind the curve. Avoids a lot of drama like this, and most everything tends to get fixed eventually—either by Apple or by hoards of developers having to patch around the bugs themselves. This kind of stuff happens all of the time, and has for years, it’s part of Apple’s culture. Either live with the occasional disruptions or hold back and use last year’s OS (i.e. I’d just be installing macOS 14.x right now, upgrading from 13. It’s done, solved, isn’t going to be changing monthly, and still is recent enough for everything to support it.)
So on to that…
I never said that. We very often have to fix Apple bugs, particularly with the text engine because they often never do fix what they break. This is an ugly and frustrating process, that often involves fragile code that you’d normally never want to use, but if that’s what it takes to fix it, we will, as we have many times in the past.
3 Likes
I appreciate the considerable effort you took in replying to my issue with nuance. Thank you.
Having said that: Are there any internal plans to get rid of RTF internally? From what I can see, all that Scrivener aims to do with text does not require RTF at all. You could store all text internally as plain text and for example use Markdown for basic styling. A user’s font choice should not be something that a user needs to set per character/word/paragraph but rather something to set per project or maybe group or sheet. Dealing with plain text is much easier and would never introduce any mess. Hiding the Markdown asterisks, underscores, hashes, and escape characters would be trivial in the UI level.
It would guarantee that the text that Scrivener deals with is pristine and will always stay that way.
I will not be requesting any refunds as someone suggested above. L&L is a small company. My frustration is valid however, and hopefully the internal text engine could be a major focus for an upcoming major release. RTF isn’t cutting it, even in the best of times.
Thanks again for the kind and thorough reply.
1 Like
Of course, I understand the frustration and I feel at as well, though surely in a different form, from the other side. One can be irate about how their work is compromised by something they didn’t actually do, and how that negatively impacts those that depend on what you do.
To get one thing out of the way first, I wouldn’t say the problem is so much with RTF, at least in this specific case, based on the results we see. The problem seems to be deeper than that, and more how the writer is taking the model of the text in memory, which is abstracted from any file type syntax particulars, and converting it to output. In other words if the flaw is deep enough, it might also produce the following Markdown:
<$Scr_Ps::0>## Heading
* <!$Sr_Ps::0> * oops
* oh well
Or let’s say we use extended Pandoc Markdown instead of bespoke codes, it might end up like:
::: {custom-style="some-special-heading-style"}
## Heading
* ::: * oops
* oh well
That said, I wouldn’t say that moving away from RTF at some point in the future is off the table entirely, though it very much remains in the realm of pure hypotheticals. That’s a very big step to take as you go from having practically zero code for writing and parsing, to huge piles of code—particularly if you are targeting a standard format rather than making up your own (where you know exactly what to parse for since you control all the writers in existence), and you don’t want that because then you end up with something few people want, proprietary formats that nothing else can read.
Don’t get me wrong, I don’t know how much of my postings you’ve read around here, nobody would ever accuse me of being anything but a staunch advocate for Markdown, and continually bemused at all of the complexity rich text software seem to demand of those that use it. It’s the only way I write with Scrivener and anything else (that it stores my Markdown in RTF is beside the point because it compiles it just fine). And while I do feel it would be very messy with Scrivener’s current feature set (imagine custom character spacing in the middle of a word with th[i]{custom-style="scriv-cs-neg20}s), it wouldn’t at least be impossible. Just… messy to the point of negating the ideal of using a clean and simple format. Or you drastically reduce the formatting powers of the software.
But hey, if you are into Markdown, that is a potential solution for you. This certainly doesn’t break in the text editor, and compiles just how you want it (with the file types at the bottom of the dropdown at the top of the window):
## Character Traits
* One
* Two
* Three
Or you could even toggle the compile switch that converts rich text lists to Markdown lists, since the first line here isn’t actually styled—or since you’re using Markdown after all, it’s perfectly valid and generally desired to have an empty line between block elements anyway, which also avoids the bug, even if you use a style for the heading instead of hashes, and rich text lists.
Maybe that even works better for you all around, since now Scrivener becomes a generator for Pandoc in existing workflows you already have?
2 Likes
Hm. Again, thank you for the detailed and thorough feedback and letting me (us) know what is going on behind the scenes.
I don’t have actual coding experience with how the “writing window” hands over the written and formatted text but as of now it feels like there is actually nothing you can do about it, since it’s not your code. The only thing possible at the moment seems like sanitizing the output from memory after the fact, but this doesn’t seem feasible and would break many other things.
I don’t want to be in your shoes right now. Have you maybe checked what Apple’s libraries do in macOS 26? Is this issue fixed there?
To me and my workflow, as you correctly deduced, having less formatting options is better as I feel in control. I don’t trust rich text formatting, ever, and I’ve once again proven right unfortunately.
It would be nice to have a feature to just “turn off all of the formatting” and reduce it to what the default Gruber Markdown offers. Maybe I should just place Markdown files in the file system and work like that…
Anyway. I will find a way to make Scrivener work for me. Good luck fixing this issue. Thanks!
1 Like
Exactly what I described before as bad, ugly fragile code you have to hold your nose and just write, because sometimes it’s either that, or all the people that depend on your software have broken lists after styles. Hopefully there is something better than that though.
I don’t want to be in your shoes right now. Have you maybe checked what Apple’s libraries do in macOS 26? Is this issue fixed there?
The beta still reproduces the bug, unfortunately.
To me and my workflow, as you correctly deduced, having less formatting options is better as I feel in control. I don’t trust rich text formatting, ever, and I’ve once again proven right unfortunately.
I did once try using lists, in the user manual project in fact (which is a Markdown-based project). I regretted it ever since though. Lists on both platforms are fragile, and styles around lists in particular (this isn’t the first time the two have oil-and-watered each other). So these days I’ve been gradually converting them all back to simple Markdown lists, whenever I come across one during normal editing. I throw a little hanging indent style on them, just for the cosmetics, but the rest is all what I can type.
It would be nice to have a feature to just “turn off all of the formatting” and reduce it to what the default Gruber Markdown offers. Maybe I should just place Markdown files in the file system and work like that…
Two things:
- While it’s basically putting lipstick on the RTF, you can at least make it look like plain-text.
- I have a whole series of posts on how to integrate Scrivener with other text editors in a relatively seamless fashion, but this one in particular describes a popular combination of using Scrivener+Obsidian, and at the very top of that is a link to what I use more often (Scrivener+Sublime Text), and how productive that can indeed be in a more descriptive fashion, whereas the first link just kind of goes straight into setup for those that already see the virtues. In short, you can have that big bunch of .md files on the disk, and still use Scrivener.
2 Likes
I can’t get this work-around to work: when I quit and re-open I still get an extra dot point in the first line of the list.
Even more tedious but works for me is to make a style with (say) a 0.5cm first line indent and a 1cm left indent and a 1cm tab stop; apply it then at the beginning of each bullet point insert a bullet (option 8 on Mac) and a tab. For me this survives quit and re-open without a separator line, suggesting that the problem is triggered by the auto-bullet list tool.
I’m not sure, it still works fine for me so long as I take care to make sure the annotation itself is not carrying a style, otherwise it defeats the purpose.
Has anyone else noticed it maybe being better in the 15.6 update? I just tried again (to make sure the inline annotation trick still worked), and although it does still break the list, it doesn’t do so in a way that is visually damaged—i.e. it wouldn’t really matter unless you go back and add new lines to the list and realise it’s just bullets/numbers and tabs at that point.
Style of annotation is No style.
I have just checked whether anything has changed with 15.6. Unfortunately it has not. The error reappears immediately after restarting the application.
I bought Scrivener a few months ago and I have to admit, I have extreme remorse in doing so. Not only the $60+ price tag but the hundreds of hours I’ve put in writing over 250K words to have the document corrupted.
I have thousands of bullet points and every single set get’s corrupted every single time I restart my computer, which is daily. I’ve spent hours trying to fix just a small portion of my book just to have it messed up again.
Now, I’ve totally ceased working on my projects because of this and have been patiently waiting for this to be corrected. Upon the last update I see that this has continued to not be a priority for Scrivener. Frankly, I want my money back but I know that will never happen.
It’s a total fraud to be advertising this tool as good for long format projects when it can’t even handle bullet formatting and your suggestion is to do a manual work around on EVERY SINGLE SET OF BULLET POINTS?
With the amount of time I would spend on this, I might as well just move to another tool.
This has been an awful experience and I will be sure to tell every aspiring author I know to avoid this toolset and ignore any legacy information that makes it seem like stable and usable tool- It’s a frankenstein set of capabilities trying to be everything to everybody while not delivering on basic functionality essential to writing any length format. It’s like I’m using a 90’s version of Microsoft Word.
1 Like