Advice sought for academic manuscript writing in Scrivener

Dear members of the Forum,

I apologize for the long post, but I thought you would find it helpful if I gave as many details as possible.

I have been, and still am, writing academic manuscripts with Multimarkdown Composer. Typically, I have one file for the Abstract, one for the Introduction and one for each of the different parts of the Results, Discussion, Materials & Methods and Figure Legends sections. For each manuscript, all those files are stored in a private git repository, so I don’t need to save multiple versions of each file: I save over top of the previous one and use version control to manage the changes. I then create an Index file, which is used by Marked to combine, in the order I want, the different parts of the manuscript into one single markdown file. I use Pandoc from the terminal to turn the markdown file into a docx file, which is what the journals we submit our manuscripts to require. I have a series of docx templates for different journals that Pandoc uses, so I can quickly reformat a manuscript for a new journal if it gets rejected by different one.

It seems to me that the idea of breaking down a long text into smaller, more manageable sections, which is what I have always been doing, is one of the core principles of Scrivener, so some time ago I purchased it, and slowly — unfortunately, I cannot take time off to learn how to use it, and as I explore Scrivener in my spare time, I keep on using my old system to avoid delays in publishing our research — I have been trying to understand how to use it best.

So far, it seems to me that by using Scrivener I could have all the different parts of each manuscript into a single Scrivener project instead of having them as separate files in a git repository. Further, Snapshots in Scrivener could be considered the equivalent of version control, and I would not need to create an Index file that tells Marked how to combine all those parts of the manuscript and export them as a single markdown file. Finally, I could still use all my docx templates with Pandoc in Scrivener, whether I write in Scrivener in markdown or in rich text (against which I have nothing).

So I have tried to put together a Scrivener template for our manuscripts, trying to recreate what I do with Multimarkdown Composer. My use of Multimarkdown is not orthodox: for example, manuscripts formatted for submission to a journal must have a page break between the title, authors, affiliation, etc. section and the Abstract; in order to achieve that, for the Abstract heading I use a level 4 heading, which I use nowhere else in our manuscripts. In my docx templates, this is styled exactly like a level 2 heading, which is what the Abstract, Introduction, Results, Discussion, Materials & Methods and Figure Legends sections would require, but has a page break before it. Even though nonorthodox, this stratagem has allowed me to overcome limitations of Multimarkdown. I reproduced this system “verbatim” in Scrivener (please see attached .scriv project file), which results in a perfectly formatted docx file (attached in PDF format) by compiling for Multimarkdown and by processing with Pandoc using my templates. Only, as you will notice, the structure of the Manuscript folder in the Scrivener binder is a bit awkward — with all those untitled documents — and I wondered whether there were a simpler way to achieve in Scrivener what I am trying to do, without necessarily trying to recreate “verbatim” in Scrivener my old system, including its stratagems to overcome Multimarkdown limitations. For example, I have tried to use Separators to achieve that space break between the title, authors, affiliation, etc. section and the Abstract but was unable to make it work. So I thought I would turn for advice to the members of this forum, who clearly know more about Scrivener than I do, and see whether they had any suggestions on how to achieve what I am trying to achieve in Scrivener.

Finally, if it matters for your feedback, we don’t have math equations in our manuscripts; I have no use for the Research section in Scrivener (all my research material is in Bookends and Tinderbox); the References section in our manuscripts is created by adding Bookends citations in Scrivener (or, at the moment, in Multimarkdown Composer) and by scanning the final docx with Bookends; and figures are never inserted in the docx file: they are created in other programs and uploaded directly on the website of the journal’s submission system, which from the docx file and the figure files produces a PDF that is used for peer-review.

Thank you very much for your consideration.

Best regards,
Enrico
Manuscript.scriv.zip (768 KB)
Manuscript.pdf (103 KB)

Something like attached project? HTH.
THE RESEARCH PAPER IN BASIC MEDICAL SCIENCES.zip (943 KB)

Dear rbenit68, (Dear Forum Members,)

Thank you for your feedback; unfortunately, that is not what I was looking for. But it’s my fault: I should have been more specific in my request. Please allow me to do that.

  1. Is it possible to create in Scrivener a Title Page for an academic manuscript, i.e. a page that contains Title, Authors, Affiliations, Keywords, Word Count, etc. that, when compiled to Multimarkdown and processed with Pandoc and docx templates (specified in the Processing pane of the Compile Format Designer), will give rise to a properly formatted title page that satisfies the requirements specified by the journals (please see the first page of the pdf attached to the opening post for an example of such page)? If so, how?

  2. Is it possible to add page breaks in Scrivener between specific parts of the manuscript (for example, in the .scriv project file I had attached to the opening post, between the last part of the Results section (“Results Section 6”) and the Discussion section), such that those page breaks are respected when compiling to Multimarkdown (which does not have a syntax for page breaks) and processing with Pandoc using docx templates (specified in the Processing pane of the Compile Format Designer)? If so, how?

The rationale for these questions is that I would still like to compile to Multimarkdown and process it to docx through Pandoc with my exsiting or with new templates, so that I will have to use Word as little as possible.

Thank you for your consideration.

Best regards,
Enrico

P.S. I realize Multimarkdown and Pandoc are central to the workflow I would like to develop with Scrivener, so perhaps I should have posted this in the “Multimarkdown and Latex” section of the Forum. Perhaps the moderators could be so kind as to advise on whether I should re-post this in that section? Thank you.

Download and make accessible pagebreak.lua: https://github.com/pandoc/lua-filters/raw/master/pagebreak/pagebreak.lua

Compile to MultiMarkdown with Basic Pandoc Copy preset.

pandoc --lua-filter=pagebreak.lua inputfile.md -o ouputfile.docx

You can format title and author with two MS Word styles in the final document.

HTH.
THE RESEARCH PAPER IN BASIC MEDICAL SCIENCES.zip (967 KB)

  1. Scrivener has special provision for “front matter” which would include title page and which would naturally page-break before any ensuring material. I would look into that and see if this does for you.

  2. More generally, how documents are treated during Compile is determined by the Section Type assigned to the document, and what Section Format the chosen compile format assigns to that Section Type. It is perfectly possible to designate that docs of Section Type X should be assigned to Section Format Y and where Section Format Y specifies that, say, a page-break-before should occur. You suggested in your post, I think, that you tried something like this but it did n’t work. This is definitely a thing, so instead of looking for other means, maybe you should spend time sorting out how to do this one thing?

I am not sure how any of this interfaces with compiling to Markdown as I don’t regularly use Markdown with Scrivener, so I don’t have specific help to offer in that regard.

gr

Sounds like an Academic Paper template is needed - maybe academics could work with Keith and his coders to produce one? It would certainly bring in bunches of customers.

If only there was an actual standard for that, but I don’t think there is. Every discipline has their own expectations!

What the OP describes sounds like some kind of research report – something that does have very established parts – though the use of a separate title page is unusual in my experience.

gr

https://www.elsevier.com/__data/promis_misc/lrtitlepage.pdf

For academic title page I think that you can use the suggestion for creating YAML block that will contain the needed elements.

The YAML block and explanation is given on the following site:

https://santa.mk/2018/12/30/yaml-block-for-academic-paper-title-page/

Dear rbenit68, Dear gr, Dear Maelduin,

Thank you very much for all your valuable suggestions and feedback.

Thank you, rbenit68, for kindly reminding me of that lua filter for Pandoc: I had completely forgotten about it!

The points raised by gr are very relevant: I am sure Scrivener can produce a manuscript format such as the one I need without the use of Markdown and Pandoc, and I hope my current need to stick to a combination of Scrivener, Markdown and Pandoc — which is the result of a series of factors, including shortage of time, as mentioned in my opening post — has not been interpreted as if I thought that Scrivener were not sufficiently capable; if so, I apologize: that was not my intention. With time, I do plan to try to achieve all I am trying to do with Scrivener only, so thank you gr for pointing me in the right direction.

I am sure all the academic users of Scrivener would be delighted to help Keith and his team develop academic manuscript templates — after all, we are always much too eager to talk about out work — but gr is right: it would be difficult to come up with a single template that satisfies the different requirements for submission to journals in the different disciplines. The format I use is pretty standard — with minor deviations — in my discipline (Molecular, Cell and Developmental Biology).

At any rate, rbenit68’s suggestion of a page break lua filter for Pandoc, the existence of which I had completely forgotten, prompted me to go back to the Pandoc manual and to a suggestion that AmberV had given me some time ago in a different post, and with the use of custom styles in Pandoc and Word, I have been able to solve the issues I had.

I have attached here the resulting .scriv project and its .docx output saved as .pdf.

Please note that to use the .scriv project as intended, you will need to compile for Multimarkdown with the “My MMD To DOCX (By Pandoc With Template)” format. You will also need a reference docx for Pandoc that includes the custom styles I have used (Manuscript Title, Authors, Affiliation, and Abstract and Figure Legends); I would be happy to make it available, but I cannot attach .docx files to this post. Unless the moderators have additional suggestions, one solution would be to contact me directly through this Forum and provide an email address to which I could send the file. Finally, you will need to indicate the path to that reference file in the Processing pane of the Compile Format Designer (it currently indicates the path to mine).

I will also post this in the “Multimarkdown and Latex” section of the Forum.

Thank you again for your help.

Best regards,
Enrico
MCDB Manuscript - New.scriv.zip (768 KB)
MCDB Manuscript - New.pdf (103 KB)

Dear mijalce,

Thank you for your suggestion and the link. I was working on the reply I have just posted, so I had missed yours. I will certainly explore your suggestion — for example, I was thinking whether using the Pandoc bracketed_span syntax for customs styles could be used instead of your html span syntax… At any rate, thank you again for taking the time to help me out!

Best regards,
Enrico

There is already a basic version which is easily afapted by each academic. And someone posted a template for medical papers a few years ago, which can also be adapted to your liking. Just make one project the way you want it and then save it as a template.

I think there is a misunderstanding; please read the rest of the sentence of mine you quoted.

Frankly, not very helpful or friendly… At any rate, no need, thanks: problem solved with help from rbenit68, gr, Maelduin and mijalce (and AmberV, in a different post), as explained in my posts above.

In general I need to deal with author, affiliations, correspondence and equal authorship etc. So I use Pandoc Filters to transform metadata for output to LaTeX, Word, HTML and plain text depending on the requirements (controlled via pandocomatic to apply one or more templates at compile time).

---
# Standard Scrivener+Pandoc YAML metadata
title: "<$projecttitle>"
author:
  - name: John Doe
    affiliation: 1
    equal_contributor: true
  - name: Joanna Doe
    affiliation: [1,2]
    correspondence: jo@caos.org
    equal_contributor: true
institute:
  - ^1^ International Cephalopod Research Institute (ICRI), Pacific Ocean.
  - ^2^ Center for Advanced Octopod Studies (CAOS), Atlantic Ocean.
keywords:
  - Pandoc
  - Scrivener
  - Workflow
compiled: <$date>
comments: "To err is human; … err?"
# Select which templates pandocomatic will use
# you can enter overrides to the templates here as well
pandocomatic_:
  use-template:
    - paper-with-refs-docx
    - paper-with-refs-html
    - paper-with-refs-latex
---

My output looks like this:

HTML:

DOCX:

LaTeX (but without my custom packages, my output looks better typographically):

I haven’t integrated that pagebreak filter yet, but will also do so, it is helpful… You can see exactly how it works by downloading a workflow.scriv project:

github.com/iandol/scrivomatic/r … .scriv.zip

And the filters themselves can be reviewed here: github.com/iandol/dotpandoc/tree/master/filters

Thank you very much, nontroppo, for generously sharing your workflow and the link to the repository with your filters!

I am traveling right now and responding from an iOS device, but I should be able to download your file on my mac already tomorrow: I can’t wait to test it: your use of the YAML block, as also suggested by mijalce, in combination with filters seems to be very flexible and powerful, so thank you very much again for that too!

Best regards,
Enrico

You can also check out Pandoc Scholar, which also uses metadata in a similar vein, and can be integrated into Scrivener similarly:

github.com/pandoc-scholar/pandoc-scholar

Dear nontroppo,

Thank you very much again for making your workflow available. I was trying to modify it / take from it what I need, and would have a few questions. I was therefore wondering whether you could be so kind to, once again, help me out or at least point me in the right direction.

Because the only output I need for manuscript submission is docx , and because I would scan that docx with Bookends before submission to generate a bibliography, I was thinking of using the Pandoc YAML Metadata you have as Front Matter in your Workflow.scriv project to generate the title page of my test manuscript. If I understand things correctly, to do so I would only need your simplifyMetadata and prependAll filters and your custom.docx template. So I simply added those filters and template to the argument in the Processing pane of the Compile Format Designer:

-t docx+styles --filter ./simplifyMetadata --filter ./prependAll --reference-doc=./custom.docx -s -o <$outputname>.docx <$inputfile>

My test .scriv project is on the Desktop; I am saving the compiled Multimarkdown file on the Desktop; and I have saved on the Desktop a copy of your simplifyMetadata and preprendAll filters and of your custom.docx template. However, this is what the YAML Metadata file looks like in the docx output:

So clearly I do not understand things correctly and am doing something wrong; would you know what? Thank you for your consideration.

Best regards,
Enrico

Sorry, for the late reply. There is an error in the markdown, as the YAML metadata is not being correctly parsed by Pandoc, please check the compiled .MD file to see if it looks well formed…