Anyone here have any experience working within Scrivener excluseively (no external apps at all), until a project is complete, until the content is done and ready for publication?
Specifically, I’m asking this question in reference to the collection and management of bibliographic reference information while I write my book. I’d like to do all of my research and passage specific linking of specific text within external papers and books… never leaving scrivener! But I’d like to do so in a way that the resulting scrivener file would be ideally situated for industry standard bibliographic citing after the manuscript is written and complete.
The solutions I see people recommending always seem to involve some sort of Byzantine workflow in which the manuscript is maintained in scrivener but the research and bibliographic linking is somehow maintained within an application (papers, zotero, endnote, et.) external to scrivener. That sort of bifurcated workflow seems an intellectual impossibility to me… like maintaining two but necessarily separate databases (changes in one necessitate matching changes in the other… nuts).
I am imagining some sort of workflow that sets up all of the relevant externa links and linked passages within scrivener in a format that could later be easily converted to properly formatted publication-ready citations, bibliography, index, footnotes, pull text, table of contents, list of illustrations and photos, etc?
Any help or tutorials would be greatly appreciated. I am new to Scrivener (first big project, a non-fiction science book).
A way to look at this is at which joints do we “cleave” this problem? The advantage of keeping at least academic research in a professional reference manager is that it is highly specialised not just at storing but also searching and linking existing and new sources. It is always going to be much more powerful than a generic store of references. For example, you can do forward and reverse citation searches, find related work, automatically find a source PDF, auto fill in the missing metadata, perform group edits (using regular expressions or SQL to really target your material). You cannot ask Scrivener to do all of this. So at least for me, I use my reference manager to store and annotate my refs and PDFs, then use linked PDFs and OPML in the Binder to refer to this work while writing. If I were to “cleave” this at the joint to keep everything in Scrivener, I would lose the extensive tools to work with and search from my reference material to new knowledge, and lose much of the flexibility to reformat my in-text citations (date-year / numeric etc.) and bibliography for different destinations.
I think the reason almost all people who have to deal with bibliographies use bibliographic software is that the benefits far outweigh the downsides. So while I do generally agree that trying to keep as much in Scrivener is a useful strategy, this is really not tenable if you need robust bibliographic reformatting at the final stages or need to seriously research using well established academic tools.
Anyway, some people do keep all this info in Scrivener, see this thread for the workflow with which to do it:
He splits BibTeX citations into individual documents imported to the binder with a name equal to the citation style which he then uses Scrivener binder links to cite to. You can manage the BibTeX entries in the binder then export and merge them back to a BibTeX, but I don’t really understand how he can easily reformat in-text citations for different formats (you’d need to manually edit document names then update scrivener links en-masse), and this workflow involves quite a bit of fiddling overall… If you would never use BibTeX you could use documents where the “formatted” bibliography entry is normal text, then use an inline annotation for notes and comments and possibly the BibTeX source. That way you can export the bibliography by removing the inline annotations at compile and get a “clean” bibliography, but still keep other metadata easily editable. What format of bibliography do you need to finally generate?
This workflow keeps your material exclusively in Scrivener, and if you don’t need all the tools and power a reference manager provides, it could work for your needs?
Thanks “Nontroppo”. The solutions you describe, the standard approach as well as the scrivener-only approch, are Byzantine at best. Huge gaping hole in the market. Kind of sad how stuck people seem. Stuck in existing pattens and unable to imagine otherwise. This is a common problem in all human domains. But it is especially ironic when it is voiced in fields as mercurial as software design. There are solutions available to this problem that have nothing to do with a super writing application that also provides research and bibliographic management, or alternatively, a super research and bibliographic application that also provides a writing environment. In situations like this, where two domains must meet and communicate, common standards can be developed from both sides, with a common published and expected data exchange architecture between. From the writing side, from scrivener for instance, there is no need to care which research package the writer is using. No need for research software to know if it was being used by people writing in Scrivener or Word or what ever. Such an solution might even allow the research software to operate within a portal in the writing software, or vise versa. This isn’t an impossible problem to solve, The current situation is inhuman and selects for a type of writer or researcher who might not in fact be the type of person best suited to brilliant insight, and are instead simply tolerant of repetitive and technically brittle minutia. This is a situation in which everyone looses. Where each of us is lass potent in whatever we choose to do. Frustrating. Solvable. One of the great advantages of Scrivener is that it combines a decent editing environment with a document finder. Some day we will be able to say the same of a package that integrates writing and research.
Note: There is a story that tells of Einstein’s reaction to his special relativity paper being rejected by the publisher.The story says that Einstein wrote “I have cited no other research because this paper will stand on its own!” on the cover of his paper and then sent the paper back to the publisher. Perhaps the truth had more to do with how maddening the process to cite other research.
There is another story that says that we’ve never meet the most potent math geniuses because no real genius would ever tolorate 18 years of nightly math homework.
Writing is a project based task. Eventually every Scrivener document will be done or abandoned. Anything which should have a lifespan beyond a particular project shouldn’t be in a project document.
Reference management is part of the infrastructure. (I regularly cite sources that I put in my bibliographic database in the early 1990s.) particular projects can link or refer to bibliographic entries and the associated documents, but they don’t belong in the project.
I agree - I really don’t see much of a down-side to keeping all references in a Biblio manager and then linking to relevant items on a per project basis. The only hiccup for me was when Sente suddenly disappeared and I had to switch everything to Bookends.
For my master’s thesis, I cited a paper from the 1800s. I’m in a technical field. Historians and literature majors (among others) routinely look back hundreds of years.
Most fields do have more or less standard conventions for modern sources; most journals allow you to download citation information in a standard, human readable format.
But across disciplines, across languages, and across hundreds of years of time? Bibliography software exists because standardization is a fantasy,
Hi Randall, while I’m sympathetic to your poetic vision of a unified environment, you are not really explaining what real problem would be solved by unification, and why is would overcome the huge loss of function that specialised academic tools provide. I use Alfred to cite my Bookends references, it is a non-modal floating environment accessed via a simple hyperkey:
It is effortless to use to cite or find a reference (author, keyword, year etc.), irrespective of the current application. But I can do so much more with that reference in Bookends than I could ever do in Scrivener (even if Scrivener contained a simple database and some basic citation linking). Losing that power would be far worse.
I keep my notes for writing in Scrivener, my PDfs linked to Scrivener but managed by Bookends, and notes are streamed from PDF annotations and can be imported into Scrivener. I could actually build a tool to make note sync two-way, as bookends has an applescript interface and Scrivener’s document format is open and flexible, but I’d rather write papers and grants for my field of study…
Brilliant books and beautiful scholastic work is produced every day even in Word, and I really doubt some minor fiddling with referencing is the thing that is really holding human potential back But yes, if Scrivener and Bookends has some mutant child who included the full functionality of both apps wrapped in one well thought out UI[1], you can count me in!
[1] as much as it will not happen, there is a reason that the best software IMO is domain-specific and often “artisan” (i.e. one-person developer, small support team): a single developer has a singular vision and passion to follow through that big teams do not.
Thanks “Nontroppo”. The solutions you describe, the standard approach as well as the scrivener-only approch, are Byzantine at best. Huge gaping hole in the market. Kind of sad how stuck people seem. Stuck in existing pattens and unable to imagine otherwise. This is a common problem in all human domains. But it is especially ironic when it is voiced in fields as mercurial as software design. There are solutions available to this problem that have nothing to do with a super writing application that also provides research and bibliographic management, or alternatively, a super research and bibliographic application that also provides a writing environment. In situations like this, where two domains must meet and communicate, common standards can be developed from both sides, with a common published and expected data exchange architecture between. From the writing side, from scrivener for instance, there is no need to care which research package the writer is using.No need for research software to know if it was being used by people writing in Scrivener or Word or what ever.Such an solution might even allow the research software to operate within a portal in the writing software, or vise versa. This isn’t an impossible problem to solve, The current situation is inhuman and selects for a type of writer or researcher who might not in fact be the type of person best suited to brilliant insight, and are instead simply tolerant of repetitive and technically brittle minutia. This is a situation in which everyone looses. Where each of us is lass potent in whatever we choose to do. Frustrating. Solvable. One of the great advantages of Scrivener is that it combines a decent editing environment with a document finder. Some day we will be able to say the same of a package that integrates writing and research.
Note: Amazing that anyone would argue that maintaining two separate research databases and doing so in two separate apps is no more effort or bother no less effective and efficient and intuitive than maintaining one research database in one app and specifically in the app where the project is being composed.
So what is your proposed solution? All you did was post the same diatribe twice without responding to any of the points people made in response. People were actually trying to help you, by pointing out that there might be disadvantages to a unified system.
BTW I am sure none of the respondents are maintaining two databases. I just throw in the relevant cite keys (I use BibDesk) for works I want to cite, then process all the citations when I have a draft to circulate. It’s a matter of minutes at the end of a writing project, not a constant maintenance process like you imagine.
This is the same argument the purist DBA will make about how far data should be normalized. My answer is thus:
The perfect is the enemy of the good.
Interfaces represent compromises. A suitable interface between research databases and applications that was sufficient for all databases and applications would either be so generic as to be useless, or so rigid and inflexible that it took too much work to implement.
There are deeper technical reasons why Scrivener has the file structure that it does, why it uses Compile instead of WYSIWYG, and those have very real implications for any reference manager that wants to work with Scrivener projects – and the workflow of each user who uses them.
If one was to write only one book and all the references one collected would be used only in this book and never again, then I can understand the urge for having one unifying app to do it all. But after having spent 35 years doing research I have never seen that situation, ever. The literature one collects and read is always much more than one actually use and refer to when writing, and the really good studies/references are often used in several articles/book chapters.
Based on my experiences, having one unifying app for both writing and searching/collecting/reading/annotating reference material would require that all writing that was ever to be done would also have to be in the same Scrivener project. While I guess that would hypothetically be possible, that is not really the way Scrivener is designed.
Another alternative is the Manuscripts app which is now being made open source. It was designed to work seamlessly with Papers 3 reference manager, as if the wore basically one app. The only problem was that they never got the Manuscripts app to work the way it was supposed to do. It had problems with simple things like cut-paste, drag-drop and so forth.
And again, Responses that have zero to do with what I am saying or suggesting.
Asking for the integration of writing with bibliographic research IN NO WAY suggests that the bibliographic research done in the context of one project would not be available to a writer/researcher in while writing another project.
It is obviously hard for people to imagine what they have never seen or used before.
How? Are you visualizing a separate cross-project database as a substrate under your hypothetical writing tool? And how does that mesh with the fundamental goal of making each individual Scrivener project self-contained?
Maybe we simply don’t understand the genius in what you are suggesting, being stuck in Byzantine ways of thinking. How about showing us? Why don’t you simply write the software you envision?
Microsoft Word now has a simple bibliography system built in. So it fits the “integrated,” but it is weak in every other respect.
Groups (academics, open source software etc) seem to come up with about one new “system to do everything for researchers” every year. Often they are specialized to a particular field, e.g. biology. Many of these are mentioned somewhere on these Literature& Latte forums, e.g. software based on “zetelkausen (?) method.”
All software involves compromises. Occasionally, someone or some group comes up with a way to rethink basic tradeoffs that solves multiple issues. Those people start companies which are eventually bought out, and they often become rich.
It sounds like the OP has never written out design specs for what he has in mind. Once he does, he can probably find something to do 80% of what he wants.
We all want a magical system that requires no wasted effort. And instead we all juggle our needs, and spend inordinate amounts of time looking at new software with the dream it will be a better fit to our own (not fully known!) needs.
But yes, if Scrivener and Bookends has some mutant child who included the full functionality of both apps wrapped in one well thought out UI[1], you can count me in!