Indexing important entries in a very large journal

One of the projects I have is a writing journal that I’ve kept for decades. While not all of the journal has yet made it into the project, the current project is pushing 300k words or more. There are hundreds if not thousands of entries. What I’m looking for is a way to index just the few entries that need to be highlighted. Each year of the journal is one, large monolithic document within the main folder. I need to create a document which links to certain points within a year, as it is not practical to have hundreds or thousands of small documents, one for each entry. This is a live document, meaning that it is not intended for publication, but rather as a continually growing document. I just need to be able find highlighted entries when I want them.

I guess this is a bit of a weird request, but the document in question is natural enough and Scrivener is turning out to be perfect for the task.

It feels like there should be a way to do this, but I can’t figure it out. Does anyone have a solution?

When you encounter an entry that needs to be highlighted, use the Document → Split command before and after, creating a separate document for just that entry. From there, all of Scrivener’s metadata is available.

1 Like

Thanks. But I do want to keep the existing structure: one year, one document. I was looking for a way to link to a passage within a document. I guess not possible??

Obviously you can do hyperlinks, or static collections such as 2003 and right click on any file and choose add to collection option. Or do keywords based on year. If open keyword panel and pick one file with inspector and metadata window open . Now hold shift key and pick all the files you want together in that year and drag keyword over to highlighted binder items and keyword will be added to all the selected files at once. Now search keywords by year and your associated files will be there in the search. ?? Is that what want.

1 Like

No. I don’t think so, unless I’m misunderstanding. For a year there is only ONE file. I’m looking for a way to link within the file. There can be no collection because there is only one file. Imagine a single document with 75,000 words. Not multiple documents. Just one. I need to index to different points within the document.

As a general rule, we don’t recommend having 75,000-word files in the Binder. That simply isn’t how Scrivener is designed to work.

Given that constraint, though, one alternative is to place an annotation at the site you wish to highlight. You can put whatever identifying data you want in the annotations, and use the project search to find them.

2 Likes

Thanks. I’ll try that.

Could you explain not having large documents like this? It’s almost exclusively how I operate. I find that having multiple smaller documents weirdly breaks my creativity. It’s a mental thing. Also, when I draft, I use no white space — no paragraphing whatsoever — so the I end up with pages and pages of a single paragraph. It keeps my creativity flowing somehow. I break things up when going into a second draft. I would have to change software before I would ever change writing method. This works too well for me.

I guess my question is: I find flexibility is at the core of the philosophy that started Scrivener, so ‘how Scrivener is designed to work’ should be however I choose to work. Right? What is the danger with monolithic documents like this? I’m genuinely curious.

Your point about wanting not to break the flow of writing only seems to get us to one document per session. It doesn’t really explain the one document per year rule. And, moreover, doesn’t explain the resistance to partitioning things in what might be useful ways after the fact.

1 Like

Well, you’ve already identified one of the problems: almost all of Scrivener’s metadata operates at the document level. You can’t use the Corkboard to rearrange scenes if you haven’t broken the document into scenes in the first place. All keywords operate at the document level, so assigning them to a 75,000 word document doesn’t help much if you’re trying to, say, pull out just the scenes involving the Jan and Dean storyline. And so on. You (probably) don’t present your work to readers in 75,000 word chunks, and you probably don’t keep 75,000 words in your head at once, so why do you want a 75,000 word document? (And if you do, what does Scrivener offer that something like Word doesn’t?)

The other issue relates to system performance. Scrivener handles large projects by only loading the specific documents that you’re trying to view at any given moment. 75,000 words demands a much bigger chunk of system resources than 5,000.

1 Like

I think you’d be much better off using Word, Pages, or other product which is better suited to your desired way of working.

1 Like

Kewms has already mentioned that you might encounter a performance hit at some point. Not really a danger though, as you can always split the document in two if things gets unbearably slow.

But-- Are you on Windows Scrivener? If so, there have been posters to this forum who have experienced Scrivener data loss due to a Windows crash. Maybe the O/S just crashed for some unknown reason, or there was a power loss, or some other unexpected outage that occurred while the poster was in the middle of editing their project. In this scenario, while the overall Scrivener project would usually survive unscathed, the document the user had in the editor and was currently working on would be damaged–irreparably corrupted. The poster’s only recourse was to recover from their latest backup.

For most users of Scrivener, this would perhaps be a heartbreaking event, but not the end of the world. But with your monolithic approach, this–admittedly rare–scenario could be catastrophic.

So, if you’re on Windows, it would be particularly important for you to regularly back up your project to zip file.

Then keep doing it. As mentioned upthread, you won’t be able to take advantage of many of Scrivener’s features, but it doesn’t sound like you’re really making use of them anyway. Kewms’ annotations alternative should handle your index requirement. It won’t work as well as document linking would, but it will work.

Best,
Jim

1 Like

I don’t use the corkboard and I don’t rearrange scenes. That’s second draft work, and even there I wouldn’t use Scrivener’s native outlining functions. My head just isn’t put together that way. A zero draft (for me) is a mad dash to the finish. Everything has to be in one document for me to properly work with them. Maybe I have a very unusual writing method. I don’t know. But I do know that writing methods are as individual as fingerprints. Anyway, no worries. The problem of needing to index certain entries in the journal is a minor one, separate from drafting the books. I can live without it.

For all that, Scrivener works amazingly well for how I work and I have no problems. The book I’m working on now will top out over 100k and that doesn’t include all the notes and structural elements I include in a zero draft. The one single document will have maybe 125k by the time I’m done.

Thanks to whoever is responsible for this great piece of software.

No. I’m a Mac user, and I manually back everything up everyday, which is easy. I copy it over to SimpleNote. You lose formatting, but as it’s a backup, it’s not a big deal. I’ve never had a data loss with Scrivener and have been using it a long time now. Worse comes worst and I just do some minor formatting on the external backup.

Best wishes, whatever method you choose. But I did want to point out Scrivenings mode, which means that you can assemble multiple documents in a single view. So it’s not really either/or.

In my own case, the ability to reorganize a document by moving chunks around, rather than cutting/pasting, justified the transition to Scrivener all by itself. YMMV.

2 Likes

No. That would be a disaster. :smile: I do use Scrivener’s folder structures. There are documents with lists of characters. Each book has it’s own journal in addition to separate monolithic writing journal. I do absolutely need Scrivener, which is amazing. I just can’t work with a main body of text which is any way broken up. I’ve developed this style of working over many years. I’ve tried traditional ways of working. Can’t make it work.

Well one document per year is for the main journal which is completely separate from the books. All my actual books in progress go into one project — not separate projects. They each get a separate folder within that project. Within those folders is a document called ‘Zero Draft’. I’ve worked for years in Scrivener and used it in many ways. This way of working is the only way that really activates the creative brain. If I break up those zero drafts, I lose the thread. Not sure I could explain if I had a year to do so.

I just have to wonder who, besides yourself, would read, and comprehend, a 100K word document with no breaks, no paragraphs, just one long word stream, I do hope the sentences are punctuated and demarcated by Capitals and periods. This is NOT being sarcastic, there are those who do write with no capitals and who have extremely long sentences.

1 Like

Ha. Yes. There is punctuation. Everything is spelled correctly and capitalized. There is also white space in the full document, usually between scenes — seldom within a scene. It’s more of a growing outline which then expands into the full text of the book. Outline a little, write a little, outline a little more. The arguments against this seem to be that I could just divide the scenes and use Scrivenings, and it’s true I tried that, but it is not at all the same thing. Psychologically it’s a very different experience, and in that difference I find creativity. Call me weird. The text for the outline is normal size, something like bullet points. The text for a scene gets copied into the document very small and indented. There is structure here. I actually draft a new passage in a new document before copying it over. I then delete the original document.

As you’ve found a process that works well for you, I’ll call you fortunate. :sunglasses:

Best,
Jim

3 Likes

That is interesting. I think it is still an unusual way of writing, but when you are writing it seems that you do ‘split’ in that “I actually draft a new passage in a new document before copying it over. I then delete the original document.” So, it is not so much just a stream of words as it is the appending of words to a running document. The caveats and worries about data integrity, etc., mentioned by others obviously remain, but if it works for you, go for it.

1 Like