Workflows for more complex text processing without messing up the project?

I suspect the answer is “no” but I’ll ask, just in case: I’m cleaning up a MS and replacing my numerous lazy first-draft patterns, especially overlong sentences with a conjunction. “He asked for this, but she did that.” becomes “He asked for this. She did that.”

Finding is easy enough. Being able to replace the text intelligently with capitalization required a two step brute-force solution:

  • Replace , but with . once
  • Then look for . a and replace with . A, and then do again for b, c, d…x, y, and z (26 checks)

This is easy enough for me to code, but I don’t want to tamper with Scrivener files on disk unless that’s safe to do so.

Is there any safe/sanctioned solution for text processing outside Scrivener on Scrivener’s source files? Something like the Compile tooling, but allowing me to feed back into the project? A plug-in architecture, I suppose it would be, except I don’t think Scrivener is built with that in mind (understandably.)

The Sync with External Folder command pulls the component documents out for access by third party tools.

There is no supported method for manipulating the contents of the project directly.

1 Like

Unfortunately our regular expression engine (on both platforms) does not support the extended replacement flags, like \u, that would let you take the original and uppercase it. Otherwise you could do something like , but (\w). \U$1

Well, that’s a tip for what you can use if you decide to use the folder sync idea, along with a text editor that supports broader replacement syntax (it sounds like you’ve got that covered if you’re talking about coding, but if your preferred doesn’t support \U either, VSCodium is a free editor that does).

For rich text, which you might need if your text is formatted in any way, LibreOffice should work fine. It has a regular expression mode, and although it also does not support the case transformation code, it does support case transformation within its Format... button, in the “Font Effects” tab. So in that case you would merely use . $1 for the replacement. Since only the first letter following the “, but” is part of that, it doesn’t matter if the entire replacement string is transformed.

1 Like

Nisus Writer Pro uses Oniguruma RegEx, which does support UPPERCASE/lowercase transformations. Only problem is Nisus hasn’t been worked on for over a year and none of us know what’s happening.

Other benefit is it native format is RTF (based on the same text-engine as Scrivener)

You could see if you can get it and use a trial installation. Otherwise P/m me if I can help.

:slight_smile:
Mark

1 Like

Excellent replies all, thanks. I’ll look into the “external folder” sync, assuming it doesn’t really care what “external” means (in my case, another directory elsewhere.) I never thought of this option!

I’m very code-aware as a 30+ yrs professional programmer. I used Makefiles to construct my pandoc-based file prep for my previous book from export and it was a delight. Now I’m finding myself wanting to perform complex find/replaces (like upper-casing) as well as corpus analysis beyond just the word frequency. “It was…” appears to be a terrible crutch of mine. I’m thinking of a draft-processing Makefile, essentially, to flag grunt work for future me to solve before compilation. I’ve been collecting all my numerous flaws in a general Notes doc, but want to make a reusable Swiss-Army fixer attuned to my needs.

I don’t mind the limited regex processing, and 99% of the time it’s just what I need. I’m now finding places where throwing the text at code of my own design would be helpful (not an LLM!) I also don’t want to lose all the structure and meta info encoded in the project. External sync may be the magic.

I could compile and re-import and let Scrivener split it back up, but this is also a sizeable work, and would rather not break it with good intentions. :sweat_smile:

Edit: Just tried it, and WOW this is exactly what I needed to know! I make a plain-text output folder elsewhere on disk, and now I can operate with impunity on the contents and sync back. Thank you, thank you, thank you, this is absolutely perfect.

2 Likes

“External folder” = “external to the Scrivener project” in this case. If Finder can find it, Scrivener can use it for this purpose.