Common Libraries of Scrivener Project Files?

As part of my work on my second-draft, I’ve had to split my Scrivener+LaTeX project up into two separate projects. The compile time for my first-draft project Scrivener+LaTeX draft is north of 20 minutes, which can s-l-o-w down any debugging to a crawl. The PDF output from the first-draft Scrivener+LaTeX project is just south of 700 pages.

When I split off my second-draft project file into a separate Scrivener+LaTeX file, I included a copy of the Front Matter and Back Matter Scrivener folders (what I think of as my Scrivener+LaTeX Common Library) from my first-draft Scrivener+LaTeX project in with my second-draft Scrivener+LaTeX project.

So I have copies of my Scrivener+LaTeX Common Library in BOTH my first-draft Scrivener+LaTeX project AND my second-draft Scrivener+LaTeX project.

It works, but is clumsy to say the least.

The challenge is when I edit or otherwise modify the Front Matter and/or Back Matter Scrivener folders in the second-draft Scrivener+LaTeX project, I need to then duplicate whatever changes are needed to the Front Matter and Back Matter Scrivener folders in the first-draft Scrivener+LaTeX project.

This adds quite a bit to my coding overhead.

This leads me to a question:

Is there any facility whereby I can have an actual Scrivener+LaTeX Common Library that is shared between two (or more) Scrivener projects so that if I make a change within the shared Scrivener+LaTeX Common Library, the changes are reflected automatically in BOTH my first-draft Scrivener+LaTeX project and my second-draft Scrivener+LaTeX project, without having to sync between two copies of my Scrivener+LaTeX Common Library?

If such a feature is available, I’d appreciate a pointer to where I can find it.

If not, given the prevalence of common libraries in so many digital languages, I wonder if this is a feature other Scrivener users might find useful. Given the prevalence of all manner of shared libraries, could such a quality possibly engender a sharing of Scrivener common libraries with features that all Scrivener writers with limited time could benefit from.

Thanks for reading,
scrive
:thinking:

I hope I am not speaking out of school here…

Short answer: I should think you would want to accomplish that with \include.

Long answer:

  1. It is unclear to me what a “common library” would be unless one’s destination was .tex — and so one’s common library (or front matter doc) contained a lot of LaTeX macro definitions and calls that would be useful to include in multiple projects.

  2. Of course, a common way to handle this in LaTeX per se is to maintain one or more separate .tex files containing the macros and options you like to have common access to. I maintain such a set of .tex files for writing articles.(*) Using the \include command in the preamble of my article doc brings the content of the relevant files in at typesetting time.

  3. Presumably you could be doing the same thing with any bank of LaTeX macros and settings you are keeping in your front matter doc. Throw them all in a .tex doc and put it somewhere outside Scriv and put the path to it in your new include command in your front matter and that will invoke it when you crank your LaTex typesetting engine. ((Since your common tex file would not be inside your Scrivener project, you would just probably want to do your tweaking of that in TeXshop or whatever LaTex-specific front end you have installed.

  4. To be clear, by first-draft and second-draft you mean first-half and second-half project files of a single book. If you were talking about first and second drafts in the ordinary sense, it would be puzzling why you wanted to continue to edit your first draft once you had moved on to your second draft (different project)!

gr

(*) Full disclosure: I do not use Scrivener when writing articles that need a lot of LaTeX.

1 Like

Hi gr,

Thank you so much for your detailed response… and NO, INMH, you are not speaking out of school, and thank you for the short answer using \include!

I hope my response does not alienate others, as I can envision how true common Scrivener libraries, although not a short term option, would add a notable degree of flexibility over and above what a LaTeX \include option would afford. That view is subject to a learning curve after I have worked with \include as part of my ‘second-draft’.

A few thoughts Re your excellent comments:

Re: It is unclear to me what a “common library” would be unless one’s destination was .tex — and so one’s common library (or front matter doc) contained a lot of LaTeX macro definitions and calls that would be useful to include in multiple projects.

Longer term, I envision the ability to work seamlessly within my Scrivener projects that include ‘common Scrivener libraries’ where changes I make within common libraries are automatically reflected within all of my Scrivener projects that use the common libraries, not as a .tex file, but as native Scrivener folders. One, but not the only, reason for this is that I have created unique Scrivener code that does not include any LaTeX code that I need to have the ability to verify the status of by visual inspection of the Scrivener code that has nothing to do with LaTeX.

In other posts, I have extolled to no end how I have usurped and greatly expanded Scrivener Styles to the point where I no longer post about them. The messaging I’ve received on how Styles were never meant to be used in the way I do (almost every minute of every Scrivener work session) has been received. (My rendition of Scrivener Styles is above and beyond anything I have ever come across in any word processing package I have ever been exposed to.) My need, however, to be able to confirm the status of my Scrivener code is not readily available within a .tex file as, for example, the color coding I use when I implement a unique Style simply is not available within the .tex file.

In essence, Scrivener is so far in a class by itself when it comes to how Styles (and so many other features) can be used, that throttling back to the bland flat style that is a .tex file eliminates one of Scriveners finer achievements.

That said, I realize that if the \include command is my only option (one that is nonetheless appreciated) I will have to adjust my view going forward.

Re: Of course, a common way to handle this in LaTeX per se is to maintain one or more separate .tex files containing the macros and options you like to have common access to. I maintain such a set of .tex files for writing articles.(*) Using the \include command in the preamble of my article doc brings the content of the relevant files in at typesetting time.

Thank you for detailing how, if I must, to utilize the \include command to incorporate a common Scrivener library.

Re: Presumably you could be doing the same thing with any bank of LaTeX macros and settings you are keeping in your front matter doc. Throw them all in a .tex doc and put it somewhere outside Scriv and put the path to it in your new include command in your front matter and that will invoke it when you crank your LaTex typesetting engine. ((Since your common tex file would not be inside your Scrivener project, you would just probably want to do your tweaking of that in TeXshop or whatever LaTex-specific front end you have installed.

Having to maintain up to date versions of my .tex files, without the excellent visual feedback available from within a Scrivener project, including but not limited to the excellent color coding option that my Styles implementation offers, would create a maintenance issue whenever I open a Scrivener project that relies on a common Scrivener library.

Re: To be clear, by first-draft and second-draft you mean first-half and second-half project files of a single book. If you were talking about first and second drafts in the ordinary sense, it would be puzzling why you wanted to continue to edit your first draft once you had moved on to your second draft (different project)!

This is where my particular non-fiction project may be unique. Just over the three years that I have been researching and writing on my subject (after decades of less intense effort) the data that is driving the subject has evolved considerably. Axioms that formed the bedrock of whatever solutions were thought to be available have (metaphorically and actually) gone up in smoke.

Recent developments in the arena have brought into question decades of solutions that recently have been seriously degraded going forward. Our view of the future is subject to current events that were unheard of except perhaps by those who understood the gravity of our common dilemma. For those who are watching with eyes open, the word profound cannot capture the speed and magnitude of what is happening.

Even after just three years of research and writing, I am having to review and trash wholesale chapters of writing, requiring a side-by-side analysis (first-draft alongside a work-in-process second-draft) of what can stay and what needs to go, and what can be rewritten to adjust toward what the data is now telling us.

In essence, I am having to cherry pick through close to 700 pages of my first-draft when deciding what to toss and what to keep and/or revise to suit the data for my second-draft.

As a LaTeX aficionado, my guess would be that you can appreciate just how temperamental LaTeX can be, particularly when the complexity increases when using hundreds of LaTeX packages. I regularly work with no less than three to four layers of backup versions of all my Scrivener+LaTeX projects in case I need to backtrack when I need to debug an error.

In such an environment, maintaining a compilable first-draft is an absolute necessity that I can imagine is unique to my particular research and writing, not applicable to many if any other writers.

Thank you again for your detailed response to my original posting. It would appear the \include option is one that I may need to adopt if I decide to pursue a common Scrivener library strategy.

Please note that despite what may sound like an exhaustive effort in my Scrivener projects to include LaTeX, I cannot imagine how I would possibly create my particular body of work in my particular field of research without LaTeX. Make no mistake, my Scrivener+LateX learning curve was intense, but the flexibility of the combined Scrivener+LaTeX language is simply unparalleled, particularly with the Scrivener library of Styles I have created. Given the dynamic nature of the subject matter, the challenges I am facing simply could not be addressed without the flexibility afforded by using Scrivener+LaTeX. Period, full stop.

Thanks you again for your response, and thanks for reading.
Scrive
:thinking:

It might be worth mentioning that Scrivener itself has an <$include….> placeholder functionality for pulling in the content of a file external to the project. Whether it gets you closer to what you are imagining, I don’t know, but worth a mention.

2 Likes

Hi gr,

Interesting … I never would have thought about it … I’ll have to read up on the <$include….> placeholder functionality …

Off the top of my head, as the <$include….> command comes into play during compiling, I wonder if it would <$include….> compiled Scrivener files … I’m not sure what that would look like (and not exactly what I was looking for) but a good start!

At the very least, if it helps reduce the compile time for the main project file, that could be VERY nice (as my project file now stands at 20+ minutes to compile!)

Thank you again for the tip!
scrive
:thinking:

Compile would do what $include tells it to do while creating compiled exports.

1 Like

I poked around to try and understand how to use the “<$include….> placeholder functionality for pulling in the content of a file external to the project.” as a way to simulate a Scrivener project library.

I realized that <$include….> can be useful for linking in documents, but not in the way I was thinking of. I also realized that I need to be a bit more specific …

I am not a coding expert, far from it. But what I remember after decades of what programming I have learned is that software libraries are ubiquitous across many platforms, if not all. So as a defacto standard for writing+coding, I’d imagined it would be helpful to have Scrivener project libraries that would be linkable to other Scrivener projects as a way of developing standard Scrivener project code to link common functions, images and data across Scrivener projects.

For a specific example, I imagine any number of glossary or acronym and other project files that I could link to, perhaps as part of a Merged Window. The folders and files in the supplementary glossaries/acronym project files could be linked into the Main project file, perhaps appearing in muted colors to indicate their status as linked folders and files, but otherwise available (but not necessarily modifiable) as native folders and files for inclusion in the final compiled product file.

I have NO knowledge of the underlying structure of the Scrivener files, and I realize that this may very well be a pipe dream. For myself as someone creating a non-fiction document, I can imagine many common libraries I could create that I’d like to make available across several Main project files as a way to eliminate duplicate code, and possibly reduce compile time, presently at 20+ minutes.

For fiction writers who are developing other-world sagas (Dune, Game-of-Thrones, Lord-of-the-Rings, etc.) the availability of common libraries to create common references, dictionaries, mappings etc. that could streamline and simplify the Main story line, but still available to the reader for clarification, might be a nice touch.

For me, the ability to move the many images I have scattered throughout my non-fiction document, but instantly available via active links within the compiled PDF document would be an enormous help.

Something to consider?

Thanks for reading,
scrive
:thinking:

As for this particular issue (which seems a quite different matter than the Library idea you’ve been talking about):

  1. You can already choose to store your image files outside scrivener and place them via an image placeholder code.

  2. But, since you are compiling for LaTeX, perhaps more directly relevant is the fact that images are never stored IN a tex document anyway, but are placed via a macro command and a path to the file (as in \includegraphics).

  3. Given (1), it seems clear you could a) store your images outside your Scriv project, define a custom LaTeX macro \myimage{…} to put in your docs along with a path to a relevant image file, where c) the image macro in its normal state would compute to the standard LaTeX include image call (hence rendering the pdf with images, but where d) by changing a parameter (flipping a switch) you could make that macro instead compute to a mere hyperlink to the image when rendered in the pdf.

Hi gr,

First, apologies for my delayed response. Second, thank you for your ideas.

I’ll need to read carefully through what you wrote so I can see how I might benefit from your ideas. I’ve been up for some time and I’ll need a bit of rest to absorb what you’ve written,

FYI, the issue I posted about has only become more relevant as I attempt my second draft … I’m hoping I can find a way to streamline the process somewhat … Thanks again!

scrive
:thinking:

Hi gr,

Thanks again for your ideas … I’ve been able to construct a shadow project with all of the necessary Front Matter and Back Matter folders … when I need to test a particular group of folders, I simply copy the relevant folders into the shadow project and test … I’ve learned the hard way NOT to keep anything in the shadow project beyond the testing phase as the syncing between the shadow and main project files can be time consuming … having fully functional Scrivener Libraries would be a great help, but I’ve been working with my shadow project now for a few weeks, and everything seems to work fine. As my Scrivener+LaTeX compile time is north of 30+ minutes (and growing), the shadow project setup will hopefully hold me over till Apple comes out with their M2 MPB+ (hopefully it won’t be a long wait).

Thanks again,
scrive
:thinking: