A better way to force sequential ordering of numbered items

Scrivener Wish List
1

The title of this doesn’t really convey what I mean, and there’s quite a lengthy desciption of the behaviour, so I apologise for a long post, but essentially, I would like to be able to cross-reference numbered items that are forward in the text without causing the sequential numbering to go haywire.

I’m writing a linguistics masters thesis, and in linguistics we use lots and lots of example sentences to illustrate the phenomena we are describing, but this feature request would equally suit anyone who works with more than a few tables, figures or any sort of numbered item.

To number items sequentially, the tag used is <$n>; each time Scrivener comes to a <$n> tag, it will replace it with a number, in sequential order, from the start of the project to the end. It is also possible to have multiple independent numbering systems by specifying the type, in using the tag <$n:[type]>, i.e., <$n:table>, <$n:figure>, <$n:eg>. If this case, Scrivener will maintain three independent sequential orderings for the three types of numbered items, tables, figures and egs.

Cross referencing is also supported, so that you can link to a particular numbered item anywhere in the text. The way to do that is a third field in the tag, which should be a unique key in that list. So for example, <$n:table:x>, <$n:table:y>, <$n:table:z>. To link to them from somewhere else in the document, you just throw the whole tag into the text. So if x is the first table in a paper, it will get assigned number 1, and then later in the document, throwing the tag <$n:table:x> into the text will result in it being replaced with “1”.

Great system. However, Scrivener it seems, cannot distinguish between what is the numbered item, and what is a reference; it doesn’t know what is actually pointing to what. As far as it’s concerned, it will look down through the document finding each sequential number tag of the same label, and replace it with the sequential number. This mean that if you cross-reference numbered items that haven’t actually occurred yet, then you could get some unexpected behaviour. Take this paragraph, for instance:

Obviously the intention here is to number the example sentences sequentially all the way through, from 1 to 5, and the cross references should refer to them. So the idea is that the middle sentence there should render as

However, what actually happens is that, after numbering the first and second, and cross referencing to them in the text, the next unique codes that Scrivener gets to are the third then the fifth; the fourth comes later. So it will actually all come out looking like this:

So, my feature request would be something to ensure that the actual numbered items are interpreted as such, and the cross references as cross references, rather than treating them as all identical entities and numbering them all sequentially, as it can have this behaviour. My workaround for the time being will be to insert something into the forward-looking cross-reference, and to go through in Word post-compile and manually insert the correct number. But for a document in which someone might be numbering thousands of items and referencing back and forth quite a lot, this would be a messy scenario.

PS. If anyone happens to arrive at this post trying to figure out how to do numbering, there is a bug that you should be aware of. If you are going to cross-reference to numbered items, then you have to have unique keys in all of your tags, otherwise Scrivener will, oddly, start an independent numbering sequence for, say, <$n:table> codes that don’t have a unique key. So give them all keys even if you don’t intend to cross-reference that item in particular.

Again, apologies for the lengthy post.

2

Yup, this is a known limitation of the current system. We might look into something more robust for 3.0. The problem is precisely what you identified: there is nothing to distinguish the pointer from the marker as both use the same code. It’s the first instance of a unique value that increments the counter, meaning references forward will increment the counter prematurely, making only backwards references valid.

So, if one wanted to stick with purely Scrivener for this, they would have to adjust their writing style to avoid forward references—something that may very well not be possible, or use placeholder text as you’ve adopted.

But another problem with these in general is that they just transform into characters when you compile. There is no actual bookmark or cross-ref in the output RTF, which may limit how flexible the document is after compilation.

Another thing to consider is MultiMarkdown -> LibreOffice -> Word. MMD cross-referencing is done using its own syntax, and uses actual marker/bookmark codes and is thus forward and backward reference capable and resilient to post-compilation editing that might shift the order of the markers, just as a word processor would be. A side advantage of this workflow is that the ultimate output result is fully wired up to stylesheets. So the output is quite flexible, whereas vanilla RTF output from Scrivener uses baked in formatting. In short it is ideal for academic and technical work (the user manual PDF for Scrivener is created using MMD, which is how I can forward-ref sections, figures and tables). The main drawback is that one types using a simple marking syntax (if you’ve ever seen Markdown, it’s about the same) rather than the prettier rich text, and some consider that to be a little too “1985”. :slight_smile:

3

Wow, you said in a single sentence what took me five paragraphs or so.

MMD → Libre → Word, hmm, does it have to go through LibreOffice or can I just use MMD → RTF? Do you have a link to a howto or somesuch that might explain the workflow?

Thanks!

4

Pretty much, as Flat ODT (.fodt) files cannot be opened by most software. LibreOffice and Nisus Writer Pro are the two I’m aware of on the Mac, though if your aim is Word that makes NWP an $80 converter, and NWP cannot handle relatively linked figures, so any work with illustrations will not open correctly. There might be a free online conversion service that can handle .fodt, but since the format is fundamentally plain-text, figures will be compiled as native graphic files alongside of it—much like an HTML page would do, and most online document conversion services are not equipped to handle multi-file scenarios such as these.

As for documentation, there is a chapter on MMD in the Scrivener user manual which strictly covers integration. For MMD usage itself you should consult the MultiMarkdown homeage. The official documentation is not a tutorial however. If you want something that goes over MMD basics in a tutorial fashion, written for Scrivener users, then check out MrGruff’s beginner guide on our forums.

5

Thanks for the help!

6

Hmm, I wonder if something like this would work (not implemented, I’m just thinking of how it could be implemented):

<$n:ref:eg:third>
<$n:num:eg:third>

That is, two tags, one telling Scrivener it is the number that should be incremented and another telling it that it is a reference to that number. Or, it could just use a different separator between number and tags rather than a separate word:

<$n:eg:third>
<$n^eg:third>

Something like that should, in theory, be possible.

Incidentally:

This isn’t a bug at all. If you just have a single keyword with no other tag, then it is an entirely different type of numbering stream.

All the best,
Keith

7

Actually, having written the above post, it occurred to me that it would take very little code to implement this - all I’d need to do was save the number tags then go through the text looking for the reference tags afterwards. So here’s how it works:

Whenever you want the actual number to appear, you carry on as before:

<$n:eg:blah>

When you want to forward-reference a number, use:

<$n#eg:blah>

In fact, it is probably best to use <$n#eg:blah> for all your references to a number, as you know that these tags won’t cause the numbering to increment at all. But where forward-references aren’t needed, users can still use the old method, so nothing gets broken.

Your example thus becomes:

[code]Blah blah, examples below.

<$n:eg:first> Example sentence
<$n:eg:second> Another example sentence

Blah blah, some text, look specifically at examples <$n#eg:first> and <$n#eg:second> above, and compare them with <$n#eg:third> to <$n#eg:fifth> below.

<$n:eg:third> Yet another,
<$n:eg:fourth> And another,
<$n:eg:fifth> Last one.[/code]

I have uploaded a special build of 2.4 that incorporates this code here:

literatureandlatte.com/dlbet … efTest.zip

I’d be grateful if you could test it and let me know if it works as expected.

All the best,
Keith

8

Far out, you guys are quick! I just opened scriv a few minutes ago to find a new version was available, including this in the list of changes:

Thanks! Testing now.

9

Works beautifully.

Only noticed now that you did this back on Thursday; sorry for not seeing the forum messages!

10

:slight_smile:

11

Wow. That is fantastic.

I hit this problem when numbering images and wanting to forward-refer. My workaround was the kludgy option of inserting a white-on-white reference list in an earlier page to get the numbering to happen correctly, and I figured that it would be fixed by a ‘list of illustrations’ at the head of the document ultimately.

But this solution is elegant, simple and works beautifully; I’ve only had to change that one-off symbol for an exceptional advance reference, which now is numbered as intended. Is this method now a keeper?

Thanks for this eye-opening, mind-expanding app. :slight_smile:

What are the options for autonumbering – or checking autonumbering – without having to compile?

12

Wow!

I’m not using advance referencing, I’m just procrastinating, cruising the forum instead of thrashing out scenes and dialogue, as I should. But wow!

Heres a guy presenting a very specific problem, and wham! the support team of Scrivener, who are also the creators, rewrites the code and have an update out the same afternoon.

There is no software company that can compete – imagining telling MS that there is something you want to do that you can’t do in Word, and before the sun sets a new version of Word is ready for download.

Just fucking great. I love this program!

Cheers,
Mats

13

There are none, but I might not be understanding why you would want to do this without compiling, in the first place. Wouldn’t the process of checking numbers be directly integrated with needing all of your work together in one document? Or to phrase it another way, wouldn’t the portion of the proofing process, where checking numbers is important, be something you’d need the final look of the document prepared for?

14

A couple of clarifications: Firstly, I found I had to do a lot of checking and rechecking when tackling this misnumbering problem. I wondered whether there could be an easier way to check that numbering was working as expected. But perhaps you’re right, and this is so dependent on compile settings that it just has to do the lot. Seems wasteful when all you really want is to run the numeric substitutions as a one-off…

Secondly, It’d be useful to force a numbering run on certain document headings or titles. As with the autonumbering of cards in corkboard view, it’d be useful to be able to choose this in other views.

I’ve been experiemnting with using Scrivener for course design – as have others – and being able to get numbering generated in the titles of a given folder, say, would make it a lot easier to re-order things whilst keeping the weeks or lessons tidily numbered. Perhaps this is possible already…?

15

I’m an obsessive re-compiler, perhaps it is something that comes with a growing familiarity with the software, but compiling isn’t expensive, and doesn’t waste bytes. You might as well do it as many times as you need to get a particular job done. I’ve even set up a macro using Keyboard Maestro to compile and overwrite the previous output file without any further input from me. I just hit Shift-Opt-Cmd-E and a few moments later I have an updated PDF/TeX/etc file.

But, it would take nearly as much processing to preview token substitutions anyway. Remember that placeholders can appear in the compile settings themselves (into section header prefix or suffix fields are the most common, but Replacements would be a second common location, and more rarely in Separators). While it is true that most of the counter types being used in the compile settings would probably not be manually typed into the draft text as well, this cannot be assumed. Then you’ve got all of the myriad ways in which a document can be included or excluded from the final content, which if there are counter tokens within these excluded documents, would impact the number of the other documents around them.

Have a look at the View ▸ Corkboard Options ▸ Show Card Numbers option. We would like to look into broadening this to the Outliner as well—but again the whole problem of compile exclusions makes this problem particularly difficult. There are certainly straightforward compile settings that would produce predictable results in the binder, but what about cases where you are filtering items out of the compile via the assignment of a keyword? Should that item be numbered? Probably not, but using the CPU to constantly do all of this background calculation would drain batteries and just generally be an uncool citizen on the computer.

I do better understand what you mean now, thanks for the clarification. Hopefully this response better explains some of the reasoning behind the limitations. It’s just a side-effect of working in a variable state multi-file outliner system.

I guess the moral of the story is: don’t be shy about compiling. It doesn’t hurt anything to compile, and you keep a fresh copy of the output as a reference as a result.

1 Like
16

Good points about compile and filtering and the like.

One of the great things about Scrivener is the ability to have it adapt to your needs rather than vice-versa. My use of Scrivener is much more straight-forward; if it’s in the Manuscript section of the Binder, it’s in the final output; no need to check Compile settings. The ability to auto-number in the binder would be a huge help to me since I’m a numbers-oriented person and I’d rather see a chapter number in the folder title and text files under them (e.g. “9: Pirate’s Lair”, “9-1: Treasure Division”, “9-2: Fight Over Spoils”). I can do that manually, of course, but inserting a chapter becomes a big PITA to renumber everything, and isn’t that why we use computers?

So there is a valid use-case there, and I certainly wouldn’t impose a feature on the features; it would have to be optional. Just because it can’t be hyper-flexible doesn’t mean that someone might not find it useful. Whether there’s enough demand and/or pain there to warrant development is a different question, of course.

Thanks!

17

Have you tried using <$hn> as the prefix in your titles when you compile? <$hn> stands for heading numbers and renders as hierarchical numbers based on the depth of your project.

So for instance if the documents in your binder looked like:

-Introduction -foo -bar -main part -foo -blah -bar

the with <$hn> as your title prefix in compile, it will result as:

1 Introduction 1.1 Foo 1.2 Bar 2 Main part 2.1 Foo 2.1.1 Blah 2.2 Bar

And the you can also do things like turn off the numbering for too many levels. I try to keep my sectioning 4 levels deep at the most, so in the compile settings I have levels 1, 2, 3 and 4 formatted separately, and then level 5+ I have removed the numbering (by deleting the <$hn> from the prefix) and made the titles italic and indented. I’ve also made the folder titles (which all correspond to chapters) new pages, right aligned and small caps.

1 Like
18

Just to satisfy my perverse sense of correctness, <$hn> actually stands for “hierarchical number” I believe.
:wink:
Mark

1 Like
19

Thanks for the tip, Jangari. I’m looking to do something like this in the Binder, pre-compile. It’s a pipe-dream, but it never hurts to ask.

20

I stand corrected!

You mean to have scrivener autonumber the nodes in the binder? Kind of goes against the idea of having the numbering apply at the compile stage.

A friend of mine is compiling her thesis using MMD → LaTeX, and she’s gone for numbering her headings using the LaTeX numbering system instead of using hierarchical numbering, but I’m basing that on having had a quick look over her shoulder at her binder. I might ask her what she’s doing and report back.