Making an index?

I’m fairly new to Scrivener and trying to learn it as I work on a new project (I’m also not an experienced author either, double whammy).

My question is: can Scrivener be used to generate a back-of-the-book index for the project one creates?

Searching for “index” and Scrivener mostly shows stuff referencing the index card system, which I don’t believe achieves the needed result. Table of Contents looks pretty straightforward but I cannot find anything related to making an index.

I’m asking this question early in the project so that if there is something that can be done I don’t have to retroactively go back and insert lots of keywords or notes or what have you.

Many thanks.

1 Like

No, there is no automated feature to create an index. In my experience in the technical publishing field, index creation was the last step after the layout and proofing, done by a specialized person with a specific skill set. Since Scrivener is not really a layout tool, any page references created by a theoretical Scrivener feature would still be subject to change and would potentially create a lot more work/potential for broken references.

1 Like

I had a feeling that would be the case, thanks for taking the time to answer.

Putting on a ‘what if’ hat, for a moment, though… it would be interesting to have a feature wherein a modest nested hierarchy of ‘index keywords’ could live within the document binder. Perhaps limited to 2 levels of nesting. Then have a function whereby the hierarchy generated a controlled set of pages upon compilation. Or on demand create an editable document.

Anyhoo… thanks, again.

2 Likes

If you compile to pdf, there are a couple of commercial programs (Pdf Index Generator is one) which let you create an index, but it’s really more of a concordance. You can tweak the automatically generated output, but that might take a while, especially if you have a long work. The problem is removing common words and your work will have its own common words. In a bio of Napoleon, listing all the pages where the word “Napoleon” appears won’t be very helpful. In my experience, such a program only gets you halfway there. But in many cases, a concordance is what is called for.

You can Compile to LaTeX (or text for use in LaTeX) from Scrivener. That way, creating an index for LaTeX from inside Scrivener is technically easy. As you are writing your content, wherever you feel a reference should be included, you add “\index{my reference}” and when you generate a pdf in LaTeX (or a text file for use in LaTeX) the references and page numbers will all match. Unlike a concordance, the advantage is that the index is not limited to words that are in the text itself, you can use whatever language you like in its creation. There are different ways to style an index, but that is outside the scope of your question.

If your budget allows, you can use a professional indexer but you will still have to append their output to your Scrivener-compiled pdf.

E-pub indices are another matter. Some publishers just include the plain text of the index with a disclaimer saying that the page numbers no longer correspond to the text (ePub 2) though they should in ePub 3. Others convert the index page numbers to URL’s (after all, an ePub is just a web page) which is more useful.

Your note about nested keywords is interesting, what would this look like?

Too bad that “pure” Markdown doesn’t have index functionality (it has footnotes, though). I don’t know whether Multimarkdown has this feature built-in or not.

1 Like

While I agree with the previous assessment—that this is not usually something done by the author themselves during the very first draft of a text, but rather by a specialist who is looking at the completed whole, and can assess where best to guide readers—Scrivener does in fact have some automatic tools for producing one. It may be sufficient for personal projects, or as a starting point for later refinement.

For a basic test you can make use of one of Scrivener’s built-in conversion to generate an OpenOffice document. LibreOffice is a good free word processor for working with this format, in case you need one. I’ll also provide instructions for installing a third-party converter, Pandoc, which Scrivener integrates with. Using that, you can create Word DOCX files.

The main difference between this approach and a more general-purpose use of Scrivener is that the underlying conversion process is more flexible, and allows you to basically fine-tune it as you require—not everyone will want or need to do that, but the option is open to you. Case in point is this very capability, indexing. That is not a feature, but rather a process that was built using the software’s ability to build things like that.

On the surface though, it isn’t all that different from how anyone else would use Scrivener. I would say the main difference is a heavier reliance upon using styles to communicate the text’s structure, rather than pure and meaningless formatting. In most cases this won’t be a downside since using stylesheets is often advantageous for many reasons, and will often be encouraged by those you will work with on the publication and editing side of things.

Here’s a very simple demonstration—it may look lengthy, but mainly I’m going into detail here since you mention being new to everything:

  1. Optional: First run the Pandoc installer (or use homebrew if you prefer), and relaunch Scrivener. It should recognise the installation and add additional compile options. Again, if you’re fine testing with OpenOffice, then skip this—we’ll be using an embedded converter for that.
  2. To test the approach, just create a new Blank project. Disposable projects to test things out are something I like to do, but if you find this a useful starting point it can become your working project too.
  3. Paste some lorem ipsum into the starter document. I prefer using Edit ▸ Paste and Match Style to strip out all of the unwanted website formatting.
  4. Rename the starter document to something like “Index Test”.
  5. At this point you need to create a couple of styles. You’ll be using these to (a) mark words that should be indexed, and (b) add invisible indexing keys into the visible text.
    1. Double-click on any word in the sample text, and use the Format menu to give it some kind of visual flair. This is entirely up to you. How do you want a word to look so that you can tell at a glance that it is indexed? Maybe it just uses green text.
    2. With the formatted word selected, click the + button in the floating Styles panel, and create a new style with the following settings:
      • Name: “Index Term” (it’s important you provide the name just like that. If you want to get creative you can, but you’ll have to customise settings things later on).
      • Formatting: “Save character attributes” (you don’t want the whole paragraph in the index!)
      • I’d disable both the font switches, but the rest of this panel is arbitrary.
    3. Repeat this process for the second style, naming it “Index Key”. This will be for inserting keys into the index that aren’t otherwise printed in the output. Consider using the styles highlight feature for either of these as well, if you find a stronger visual cue is desirable.
  6. So at this point, since you’ve created two styles from a couple of the words in the sample text, they are already styled, meaning you have an index with two entries already. You could add a few more if you wish. Just select words or type them in and use the Styles panel to indicate them.
  7. When you’re ready to test output, use the File ▸ Compile... menu command and click on the Compile for setting at the top. If you installed Pandoc, then select the “Pandoc → Microsoft Word” option. Otherwise select “MultiMarkdown → OpenOffice”.
  8. From the left sidebar select either “MMD OpenOffice Document” or “Pandoc Word Document”.
  9. Optional: You could just leave the Section Layouts alone for this simple text, but if you want to see how a table of contents could also be easily added, then assign the “Text Section with Heading” layout. Click on the Assign Section Layouts button below the preview column, and click on the indicated preview tile (there should only be one type of document in this simple text, but in more complex projects you may have multiple types in the left sidebar to assign). As you can see though, with this workflow your choices are very simple and purely structure. There are no fonts, no indent settings.
  10. On the right hand side, select the General Options tab (gear icon), and enable Convert rich text to MultiMarkdown. Although in this simple example you won’t have any (styles don’t count)—you probably will at some point want to set some text it italics and have it come out that way.
  11. That’s pretty much it, click the Compile button, and when saving the file you can opt to have it automatically opened in your word processor.

Now that you’re in the word processor, you’ll need to add the index listing to the document. I have no idea how to do that in Word, but in LibreOffice just put the cursor at the bottom of the file, and use the Insert ▸ Table of Contents and Index submenu to add an index. In the subsequent dialogue, select Alphabetical Index from the type of list, and the rest can be left default.

And there you go! And of course, that’s also how you would add a ToC. But if you decided to use Pandoc to get a Word file, do note you can have Scrivener insert the ToC for you (and indeed, it should by default). You just need to update fields to populate it upon loading.

You might be wondering where the formatting is coming from—it’s just stock formatting. You’d be doing that kind of stuff in the word processor, using its stylesheets to change the look centrally, rather than per word. Ideally though it’s something your publisher would be worrying about. That isn’t a kind of thing that you would be focussing on in Scrivener, and working this way encourages the use of the software as a pure writing tool.

And it is with that point that I would encourage you to stress test this workflow a bit. Do some stuff in this sample project that you would expect to want to be able do. Insert some figures, tables, lists, block quotes (styles!) and et cetera. Compile often (now that you have it all set up, it’s a quick one-two punch to update your test .docx file), and observe the results of your tests.

Earlier I mentioned that this approach can be “peeled back” and is more flexible than the general-purpose writing approach. We put the name of the conversion engine on the format in the compile menu for that reason: to let you know what technology you’re working with for document production. Markdown is a form of simplified structural writing that you may have seen in websites and forums—if you’ve ever put asterisks around a word to emphasise the phrase in a Discord chat or Reddit comment, that’s what we’re talking about. We’re using “dialects” of that which are much more capable of producing the sort of things authors need in books, but the idea is the same.

As you’ve seen, Scrivener can generate most, if not all, of the Markdown you’ll ever need, meaning you can largely ignore that fact (indeed, all Mac users who create ePub or Kindle books with Scrivener are unwittingly MultiMarkdown users!). But if you’re inclined toward writing in that method itself, rather than having the software generate 100% of it for you, then you can start by switching off the full rich text conversion option, and maybe switching on some of the finer controls, like still converting lists and tables, or hyperlinks. A lot of the features you use in Scrivener will always convert, like footnotes and images. At that point you’d probably want to open up the user manual and skim through Chapter 21, where this integration is discussed in much greater detail.

But like I say, that’s an optional path one can take—with the defining feature being that you can take it. I think it’s good to know what the technology is though, in broad strokes, so that if something does go wrong or doesn’t work quite like you thought it would, you can look to the web or this forum, for answers with the right keywords.

4 Likes

Hi Martin_g,

Not knowing what time resources you may have available, if you are in this for the longer term and have the time available, you might glance at the attachments at the bottom of my posting (which is the very last one on page 2) on L&L at [url]comparison to other word processors and writing tools] that outlines a LaTeX option for creating an index.

I’ve been using Scrivener+LaTeX for over a year, and (without having any prior knowledge of either software package) have created a robust, real time indexing system in LaTeX that is outlined at the above URL on L&L. While LaTeX has thousands of ‘packages’ available to do amazing things, Scrivener amplifies that utility with a brilliant interface that together with LaTeX works seamlessly.

Due to the depth and breadth of what LaTeX offers, the learning curve can be steep. If you are curious and persistent, however, the pairing of Scrivener+LaTeX could prove beneficial in the longer term, but perhaps not so much in the short term,

Hope some of this may be of some help,
scrive
:slight_smile:

Wow… that’s some in-depth responses, many thanks to all of you for taking the time. I shall dive into your responses in due course. It seems like I’d be better off spending the time getting some of my ideas down in writing and seeing if it amounts to a hill of beans big enough to warrant an index.

The last time I wrote a book was about 20 years ago and all I did was write the copy, make the illustrations and shoot the images. I’m pretty sure it was all done in Word. Before that it was just magazine articles, even simpler. But I’ve been itching to get stuck into a bigger subject and to write something in a longer form for a while. So here goes.

Thanks again, I shall be reading and re-reading these replies.

2 Likes

Hello, scriveners!

I’m sorry if this has been asked before. I could not find it using a few searches, although I’m positive this must be a matter that has likely interested someone else. I’d appreciate any pointers.

I have a long manuscript that uses specific keywords or topics the reader might be interested in reviewing. I’d like to provide an index of references to these keywords at the end of the book, much like in the figure:


Is it something that can be done in Scrivener? Or should I use some other post-processing tool for that?

Many thanks in advance.

I’ve merged your post with a recent discussion on how to do this. Have a look at this post, which covers a route by which Scrivener can manage building an index directly (I use very similar for the recent updates to the Scapple user manual).

That said, I don’t quite understand how the Keywords feature would map to a concept like this. You’re typically going to want to put the indexing keys into the text itself, where the user should be taken to. Keywords are metadata attached to the whole section as a whole, not something in the text, or that can be easily manipulated into the text in a functional way.

1 Like

It has: Table of Contents (The caveat here is, of course, that the indexed words need to be a heading somewhere in the document).

Thanks for the pointer and the detailed response, @AmberV. That’s very close to what I need. I will look into that approach.

Alternatively, in order to avoid page number hell, I could simplify the requirement to something like this, where each #concept_*i* is a Scrivener keyword attached to the document where it is referenced and each chapter_*i* is the Scrivener name of the text document itself:
image

Do you think this way it would be easier to accomplish?

1 Like

So long as your indexing needs only ever require taking the user to the very beginning of the chapter, yes you could use a combination of hyperlinks to achieve a list like this. It’s also worth noting that if you hyperlink a page numbering placeholder, <$p>, to a section, it will be substituted with the actual paginated number in the word processor. That, instead of the whole chapter name itself, would be a more traditional way of formatting an index anyway. I don’t typically see “Chapter 12: The Revolution, 73” in an index, just “73”.

Like I say though, most indexes have a finer granularity than chapters, so you may run into issues that would ultimately lead you to just maybe mark the text somehow and create the index later in a word processor. Of course a higher amount of outlining in the Draft folder will also mean more granularity. If you outline down to subsubsections for instance, then your index will be more accurate and require the reader to do less page flipping.

The main question I would have is how long the document is meant to remain useful post-compile. A genuine index will be a lot more useful than a static list of bookmarked page numbers in the end. It’s kind of the same concept as the ToC. Scrivener has a simple little tool for creating a static text ToC (using the same techniques as this, in fact), but an actual ToC at the word processor level, using the up to date heading structure of the document, will be a lot better to work with and eventually archive.

1 Like

For the purpose of this particular book, which is a compilation of short stories, the very beginning of the chapter will do just fine. Thanks a lot for your tips.

I’d like to be able to print the keyword list without having to cut and paste it into a word doc. The only way I’ve even been able to do that is by taking screenshots. The normal Ctrl-A/Ctrl-V doesn’t work. It highlights the whole list of keywords, but when I paste, it only pastes one keyword.

In windows if click ctrl + A will highlight keyword list. Key is expanding folders of keywords so contents will be copied. Once do this then drag the whole list into a blank file and will see a full list of all the keywords and can then copy and paste that list into Microsoft Word/Google Docs/ etc and print.

The Key is dragging the highlighted keywords into a blank file. Just did this now.

I’ve thought about posting a sample of the Scrivener/LaTeX Index that took me more than a bit of time to implement in my project. Despite the amount of time needed, as a Scrivener/LaTeX project, the time invested was worth it.

Now all I need is to apply one of the Scrivener Styles that I created some time ago (and I occasionally update with new features - something that makes Scrivener truly unique as an integrated document and metadata management system) to any stretch of text within my project, and the text is automatically added to the index with each compilation. No additional effort is needed.

The combination of Scrivener Styles/LaTeX package allows for a kind of tiered indexing typified by how I’ve indexed Black Swan event(s):

image

There is no denying the significant time investment needed to fully implement LaTeX with Scrivener. For those with longer-term goals, however, the up-front investment may prove to be worth it.

My latest view of the intersection between Scrivener and LaTeX is one of a catalyst, e.g.: A catalyst is a substance that speeds up a chemical reaction, or lowers the temperature or pressure needed to start one.- US DOE.

In my case, it is highly likely I would not have achieved more than a tenth of what I was able to using Scrivener+LaTeX.

Food for thought …

scrive
:thinking:

4 Likes

Since it has been a while since it was posted, it’s worth reiterating from a previous post, that a number of our built-in compile formats, along with the LaTeX-oriented project template, come with indexing tools out of the box. They are described in Appendix D.6, but it really boils down to marking index keys with styles, as described above.

This does not necessitate the use of LaTeX, though it does mean using a Markdown-based compile conversion (which does not itself imply one is using Markdown directly write, though you certainly can). You can get an index in DOCX or ODT, too.

We can do this because using Markdown conversion along with Scrivener’s compile platform is far more powerful than using its native PDF or RTF-based file options. Those using Scrivener more like a word processor will be limited by what features are put into the software and supported. Those using Markdown are limited by very little, and so it is by far the best choice if one needs more than what the native compiler supports by itself.

And again, it does not require one to learn Markdown as Scrivener can create the Markdown file for you, if that’s not your cup of tea. The main operational difference between the two approaches is that one would be using their word processing or desktop publishing tools to design the document, rather than Scrivener. Most would say that’s the best place to be doing that anyway, but it’s something to be aware of as your output will look quite vanilla until you import it into a template. Markdown’s clean and well-styled output is especially conducive to radical transformation of a document’s design, and it only takes a few seconds to get from vanilla to nearly finalised design (less if you dig into Pandoc and find how to create a template file that is used to automate that process for you).

4 Likes

I’m a writer but also a publisher of massive academic works with indexes that have run to nearly 100 pages, done by professional indexers. Now that we are creating print, ePDF, and ePub versions of new books, I see a need for anchor index items that the author can put in while working. It is relatively easy for an author to consider this during an edit. What we have found is that having a static index, while excellent work and useful in print, isn’t what readers expect with ePDF in particular. They want to be able to click from the index to the location. (And sites such as JStor want PDF only, not ePub.) A good copyeditor could do this, too. While professional indexers are great, authors and editors can also do an excellent job. But it’s inefficient to have multiple people working to comprehend the text without being able to use their understanding, then and there, to do a quick mark-up. One can do this kind of mark-up in Word, by the way, but I really want to be able to do it while I am working in Scrivener! Create and update an index - Microsoft Support

1 Like

Have you gone through the exercise I outlined above? Scrivener already can do what you’re asking for out of the box.

It does look daunting but I will follow the steps and report back. Thank you!