Any way to programmatically create Scrivener project from text files?

I have several sets of dozens of text files with different sections to them that I’d like to shove into a new Scrivener project. Some files would go into the Research folder, others would go into the Draft folder as chapters and scenes. Some text may even go into the corkboard. I’d like to automate the process of creating a new Scrivener project file from these parts.

Being a programmer type, I meticulously went over existing projects as well as new, empty projects, and have figured out how to create all of the various RTF & XML files, cross-referencing by UUID into different settings files and compile.xml – it all looks pretty good, really. My problem is in the binder.backup file, which is a binary file I have no insight into. If I leave that file out of my project, Scrivener won’t open the .scriv file. If I put in a copy of the one from the empty project, it moves every file into the “recovered files” section.

Is there an existing tool that can do what I want? Or is there a way I can construct a binder.backup file that will match the contents I’ve set up in my script? I know what I’m asking for is pretty out there, and I’m not really expecting much support from the L&L staff. But if there’s just a nudge or two that can be given, that would be awesome.

1 Like

As someone who has dabbled in the internals of the scrivx format, I’m wondering if it’s really worth the effort. Is this something you expect to do on a regular basis or is this a one-off operation? In the latter case, have you explored the various ways in which Scrivener can import and split files?

1 Like

Sounds like a fun project! I wrote a little scaffolding project data creator a while back, though I made use of the iOS Mobile folder setup, as a way of programmatically injecting new data into a live project, since it is designed to accept Mobile updates on the fly while open.

The binder.backup file is simply a zipped copy of the .scrivx, so you should have no problem making one of those.

However it’s telling that you ran into a condition where lacking that file blocked loading the project, as neither of those files are mandatory, and will be automatically regenerated and overwritten by Scrivener on a regular basis.

Procedurally, this file is created when the project is successfully opened by Scrivener, thus guaranteeing it is a valid copy of the .scrivx. In the case of an inability to load the primary .scrivx, it would try to load the binder.autosave zip first, since that is going to be the most recent backup, and if that is also corrupted, then it falls back to what is known to work, even if it might be old.

Thus we can know for certain that Scrivener is falling back to loading the binder.backup file since it tries to recover from it in the case where you substitute an incorrect one, and fails to load without one. From this we can deduce there is a syntax or schema error in the primary .scrivx you’ve created, and therefore in this case, creating a zipped copy of it as binder.backup won’t help the project load.

1 Like

Is this something you expect to do on a regular basis or is this a one-off operation? In the latter case, have you explored the various ways in which Scrivener can import and split files?

I have done it often enough that I’ve wished (for years) that there was an easier way to automate it. And the lack of automation has kept me from doing it more often. I had some free time and needed a project to keep me busy (and learn a new language feature) so I thought I’d give it a try. It probably took me about 15 hours over the last 2-3 weeks.

Part of the process that I didn’t mention (because it’s not relevant here) is cleaning up the source files in various ways; I’ve put off automating most of that because it’s easy enough to do that while I’m manually creating the Scrivener project. If I can automate the creation, automating the clean up is a breeze, and I could really save a whole bunch of time, making this now a trivial task that I would probably do too often, instead of a chore that kept me from doing it often enough. That’s a better place to be in, though, IMO :slight_smile: .

@AmberV - that is awesome, awesome information, exactly the nudge I was hoping for – thank you!

If your code is presentable, I’m sure some of us geeks would enjoy if it were shared on github or something similar :nerd_face:

2 Likes

I’ve been hacking at the reverse, cleaning up Scrivener output using VBA because it’s much easier than working in Scrivener.
I’d prefer to clean up the Scrivener files directly so I don’t have to run macros after each compilation.
Publishers have specific formats that are tedious to replicate in Scrivener.
Similar to your project, I’d like to alter the Scrivener files, e.g., RTFs directly and bypass the Scrivener GUI.

Have a look at this post, and the links within it, which go into more detail on the process, including checklists. There are ways of streamlining what you are doing, I would suspect, down to 10 or 30 seconds of basically importing your compiled data into the template. It takes some setup getting the style names matching, but after that point it is very efficient.

Of course if the provided template is not stylesheet driven, then it is very bad and the author should be admonished. :rofl:

But this approach is basically what Markdown-based users of Scrivener get for free, with Pandoc (either Word or LibreOffice). Its conversion mechanism does the whole part of opening a template and injecting the raw styled text into it for you, resulting in a completed document upon clicking compile.

Gah, I’ve been meaning to update this thread for a while, now.

My problem turned out to be that I forgot to add the TrashFolder to the Binder. I created it, but never added it, and every time I went through and compared working vs. non-working, I saw the TrashFolder in the working and looked at my code and said yep, I’ve got that one.

@nontroppo - I’m cleaning things up with the thought of making it available, either here or possibly on CPAN. There’s an awful lot to do to make it something I would feel comfortable releasing, though.

2 Likes

Take your time and no pressure! :nerd_face: