Scrivomatic - Compiling issues with Scrivomatic/Pandoc (disappearing tables, unknown LaTex)

Scrivener Markdown & LaTeX
21

OK I can see no YAML validation errors in your file. I copied your pandocomatic.yaml to my desktop, giving it a new name:

> cp pandocomatic.yaml ~/Desktop/pmatic.yaml

Then cd to desktop and use echo to make a small test file using latex-refs to run with pmatic.yaml

> cd ~/Desktop
> echo "---\ntitle: Hi\npandocomatic:\n  use-template: latex-refs\n---\n\nTest" > test.md 
> pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test.md
> latexmk -logfilewarnings -interaction=nonstopmode -f -pv -time -xelatex test.tex

And it works just fine, with a PDF generated, can you try the same set of steps? If it works we know this pandocomatic.yaml is OK.

If it works then you can try to now use scrivomatic with the same yaml:

> scrivomatic -v -y ./pmatic.yaml test.md

There may be some pandoc or pandocomatic bugs with going straight to PDF, when I try to run pdf-refs my pandocomatic just hangs and I must close it with ctrl+c:

ε ➪ echo "---\ntitle: Hi\npandocomatic:\n  use-template: pdf-refs\n---\n\nTest" > test.md 
pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test.md
^C/Users/ian/.pixi/envs/ruby/share/rubygems/gems/paru-1.4.1/lib/paru/pandoc.rb:282:in 'IO#read': Interrupt
	from /Users/ian/.pixi/envs/ruby/share/rubygems/gems/paru-1.4.1/lib/paru/pandoc.rb:282:in 'block in Paru::Pandoc#run_converter'
	...
Exception: pandocomatic killed by signal interrupt
  [tty 30]:2:1-96: pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test.md

In general I normally prefer to run latexmk myself, and this is also what -b in scrivomatic does so with latex-refs I still get a PDF at the end:

> echo "---\ntitle: Hi\npandocomatic:\n  use-template: latex-refs\n---\n\nTest" > test.md 
> scrivomatic -v -b -y ./pmatic.yaml test.md

Let me know how it goes…

2 Likes
22 
❯ echo "---\ntitle: Hi\npandocomatic:\n  use-template: latex-refs\n---\n\nTest" > test-latex.md
❯ pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test-latex.md
No such template: 'latex-refs'.

❯ echo "---\ntitle: Hi\npandocomatic:\n  use-template: docx-refs\n---\n\nTest" > test-docx.md
❯ pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test-docx.md

[INFO] Running filter citeproc
[INFO] Loaded /Users/X/.local/share/pandoc/csl/apa.csl from /Users/X/.local/share/pandoc/csl/apa.csl
[INFO] Loaded /Users/X/.local/share/pandoc/cite-abbr.json from /Users/X/.local/share/pandoc/cite-abbr.json
[INFO] Loaded /Users/X/.local/share/pandoc/Core.json from /Users/X/.local/share/pandoc/Core.json
[INFO] Completed filter citeproc in 140 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/prependAll.rb
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/prependAll.rb in 1 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/simplifyMetadata.rb
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/simplifyMetadata.rb in 0 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/convertIndex.lua
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/convertIndex.lua in 5 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/pagebreak.lua
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/pagebreak.lua in 2 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/abstract-section.lua
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/abstract-section.lua in 2 ms
[INFO] Running filter /Users/X/.local/share/pandoc/filters/pretty-urls.lua
[INFO] Completed filter /Users/X/.local/share/pandoc/filters/pretty-urls.lua in 1 ms
[INFO] Loaded /Users/X/.local/share/pandoc/templates/custom.docx from /Users/X/.local/share/pandoc/templates/custom.docx

I am puzzled.

23

You’ll be happy (or not) to know, i was puzzled too. OK lets cut this yaml to a minimum:

templates:
  refs:
    pandoc:
      reference-links: true
    metadata:
      notes-after-punctuation: false
  latex:
    pandoc:
      from: markdown
      to: latex-smart
      standalone: true
      template: templates/custom.latex
    metadata:
      documentclass: 'article'
  latex-refs:
    extends: [latex, refs]
  docx:
    pandoc:
      from: markdown
      to: docx
      standalone: true
      reference-doc: templates/custom.docx
  docx-refs:
    extends: [docx, refs]

BUT when I run this it seems to ignore the -c option, I still see filters being used from my pandocomatic.yaml:

pandocomatic -c ./pmatic.yaml --log pmatic.log --log-level debug --enable pandoc-verbose test.md
...
[INFO] Loaded /Users/ian/.local/share/pandoc/csl/apa.csl from /Users/ian/.local/share/pandoc/csl/apa.csl
...

There should be no specififcation for apa.csl in pmatic.yaml, therefore it is still using my data dir, lets move it temporarily:

> mv ~/.local/share/pandoc/pandocomatic.yaml ~/.local/share/pandoc/p.yaml
> rm pmatic.log
pandocomatic -c ./pmatic.yaml --log pmatic.log --log-level debug --enable pandoc-verbose test.md

OK this is finally only using our pmatic.yaml, so it tells me -c is combining yaml content not replacing it, therefore my test was not reliable. If you move/delete your current pandocomatic.yaml and then try with my new minimal pmatic.yaml, does it work now?

2 Likes
24

I filed this issue: --config option does not replace the default pandocomatic.yaml? · Issue #119 · htdebeer/pandocomatic · GitHub – not sure if this is a bug or a feature, may just need clarification in docs :slight_smile:

1 Like
25

After a momentary confusion where I was using a pmatic.yaml that was only a renamed copy of my original file (why…), I did as you suggested, moving out my pandocomatic.yaml file.

This worked! I can compile on the limited pmatic.yaml.

On a hunch, I moved pandocomatic.yamlback to the pandoc folder, to test this --c option.
Indeed, pandocomatic ignores pmatic.yaml. But also, miraculously, for seemingly no reason at all, it compiled perfectly using the latex template from pandocomatic.yaml.

The baffling logs: 
2025-03-29 20:02:44 INFO : ------------ START ---------------
2025-03-29 20:02:44 INFO : Running /opt/homebrew/lib/ruby/gems/3.4.0/bin/pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test-latex.md
2025-03-29 20:02:44 DEBUG: Validating command-line arguments:
2025-03-29 20:02:44 DEBUG: ✓  Option '--input' not used:  treat all arguments after last option as input files or directories.
2025-03-29 20:02:44 DEBUG: ✓  Convert single input file or directory.
2025-03-29 20:02:44 DEBUG: ✓  Input files and directories exist.
2025-03-29 20:02:44 DEBUG: ✓  Can read configuration file './pmatic.yaml'.
2025-03-29 20:02:44 INFO : -  Checking feature toggles:
2025-03-29 20:02:44 INFO :    ✓ Enabling feature 'pandoc_verbose'.
2025-03-29 20:02:44 DEBUG: Start conversion:
2025-03-29 20:02:44 INFO : (2) + converting /Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md 1 time:
2025-03-29 20:02:44 INFO : (1)   - convert test-latex.md -> test-latex.tex
2025-03-29 20:02:44 DEBUG:   #  Using template 'latex-refs'.
2025-03-29 20:02:44 DEBUG:   #  Selected template mixed with internal template and pandocomatic metadata gives final template:
                                  extends:
                                  - latex
                                  - refs
                                  glob: []
                                  setup: []
                                  preprocessors: []
                                  metadata:
                                    notes-after-punctuation: false
                                    documentclass: article
                                  pandoc:
                                    reference-links: true
                                    from: markdown
                                    to: latex-smart
                                    standalone: true
                                    template: templates/custom.latex
                                  postprocessors: []
                                  cleanup: []
2025-03-29 20:02:44 DEBUG:   →  Reading source file: '/Users/X/Dropbox/X/DocumentsTestScrivo/test-latex.md'
2025-03-29 20:02:44 DEBUG:      | FileInfoPreprocessor. Adding file information to metadata:
                                     pandocomatic-fileinfo:
                                       from: markdown
                                       to: latex-smart
                                       template: templates/custom.latex
                                       path: '/Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md'
                                       src_path: '/Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md'
                                       created: 2025-03-27
                                       modified: 2025-03-27
2025-03-29 20:02:44 DEBUG:      | MetadataPreprocessor. Adding mined YAML blocks to metadata:
                                     notes-after-punctuation: false
                                     documentclass: article
2025-03-29 20:02:44 DEBUG:      #  Changing directory to '/Users/X/Dropbox/X/Documents/TestScrivo'
2025-03-29 20:02:44 DEBUG:      #  Running pandoc
2025-03-29 20:02:44 DEBUG:      |  pandoc	--reference-links \
                                           	--from=markdown \
                                           	--to=latex-smart \
                                           	--standalone \
                                           	--template=/Users/X/.local/share/pandoc/templates/custom.latex
2025-03-29 20:02:45 DEBUG:   ←  Writing output to './test-latex.tex'.
2025-03-29 20:02:45 INFO : ------------  END  ---------------

2025-03-29 20:04:42 INFO : ------------ START ---------------
2025-03-29 20:04:42 INFO : Running /opt/homebrew/lib/ruby/gems/3.4.0/bin/pandocomatic -c ./pmatic.yaml --log-level debug --enable pandoc-verbose --log pmatic.log test-latex.md
2025-03-29 20:04:42 DEBUG: Validating command-line arguments:
2025-03-29 20:04:42 DEBUG: ✓  Option '--input' not used:  treat all arguments after last option as input files or directories.
2025-03-29 20:04:42 DEBUG: ✓  Convert single input file or directory.
2025-03-29 20:04:42 DEBUG: ✓  Input files and directories exist.
2025-03-29 20:04:42 DEBUG: ✓  Can read configuration file './pmatic.yaml'.
2025-03-29 20:04:42 INFO : -  Checking feature toggles:
2025-03-29 20:04:42 INFO :    ✓ Enabling feature 'pandoc_verbose'.
2025-03-29 20:04:42 DEBUG: Start conversion:
2025-03-29 20:04:42 INFO : (2) + converting /Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md 1 time:
2025-03-29 20:04:42 INFO : (1)   - convert test-latex.md -> test-latex.tex
2025-03-29 20:04:42 DEBUG:   #  Using template 'latex-refs'.
2025-03-29 20:04:42 DEBUG:   #  Selected template mixed with internal template and pandocomatic metadata gives final template:
                                  extends:
                                  - latex
                                  - refs
                                  glob: []
                                  setup: []
                                  preprocessors: []
                                  metadata:
                                    notes-after-punctuation: false
                                    link-citations: true
                                    link-bibliography: true
                                    csl-hanging-indent: true
                                    documentclass: article
                                  pandoc:
                                    reference-links: true
                                    verbose: true
                                    citeproc: true
                                    bibliography: Core.json
                                    csl: csl/apa.csl
                                    citation-abbreviations: cite-abbr.json
                                    from: markdown
                                    to: latex-smart
                                    standalone: true
                                    template: templates/custom.latex
                                  postprocessors: []
                                  cleanup: []
2025-03-29 20:04:42 DEBUG:   →  Reading source file: '/Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md'
2025-03-29 20:04:42 DEBUG:      | FileInfoPreprocessor. Adding file information to metadata:
                                     pandocomatic-fileinfo:
                                       from: markdown
                                       to: latex-smart
                                       template: templates/custom.latex
                                       path: '/Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md'
                                       src_path: '/Users/X/Dropbox/X/Documents/TestScrivo/test-latex.md'
                                       created: 2025-03-27
                                       modified: 2025-03-27
2025-03-29 20:04:42 DEBUG:      | MetadataPreprocessor. Adding mined YAML blocks to metadata:
                                     notes-after-punctuation: false
                                     link-citations: true
                                     link-bibliography: true
                                     csl-hanging-indent: true
                                     documentclass: article
2025-03-29 20:04:42 DEBUG:      #  Changing directory to '/Users/X/Dropbox/X/Documents/TestScrivo'
2025-03-29 20:04:42 DEBUG:      #  Running pandoc
2025-03-29 20:04:42 DEBUG:      |  pandoc	--reference-links \
                                           	--verbose \
                                           	--citeproc \
                                           	--bibliography=/Users/X/.local/share/pandoc/Core.json \
                                           	--csl=/Users/X/.local/share/pandoc/csl/apa.csl \
                                           	--citation-abbreviations=/Users/X/.local/share/pandoc/cite-abbr.json \
                                           	--from=markdown \
                                           	--to=latex-smart \
                                           	--standalone \
                                           	--template=/Users/X/.local/share/pandoc/templates/custom.latex
2025-03-29 20:04:43 DEBUG:   ←  Writing output to './test-latex.tex'.
2025-03-29 20:04:43 INFO : ------------  END  ---------------

Inexplicably, a hasty test-pdf.md file indicates no pdf-refs template found, and my main Scrivener project still does not find the latex-refs template.

26

When latex-refs builds on the command-line are you still using the -c option (i.e. combining data-dir pandocomatic.yaml and local ./pmatic.yaml)? What happens if you remove the -c option on the command line?