Transitioning LaTeX typesetter engine from PDFLaTeX to LuaLaTeX

I’m transitioning my LaTeX typesetting engine from (PDF)LaTeX to LuaLaTeX. Although not strictly a Scrivener issue (and some assistance is available online for traditional LaTeX users) I’d appreciate any thoughts or suggestions those Scriveners who have already transitioned may have.

Before anyone asks, the reason for the transition is to increase the available memory. I’ve reached the hard memory limit of main_memory = 12435455.

Thanks,
scrive
:thinking:

As I mentioned in another post, I learn’t from the Pandoc template and great derivatives like Eisvogel:

…that using the iftex package allows you to tweak any templates to optimise across all three engines without needing to hard-code for only one engine.

https://ctan.org/pkg/iftex

Now one may argue that PDFTeX is legacy and the new engines are the present/future of TeX compiling there is no need to keep supporting PDFTeX, and that would also be a fair assessment. I stick to using iftex largely just in-case one engine has a bug, then I can jump to the other engines as a fallback (my route is XeTeX > LuaTeX > PDFTeX [though I’ve never had to fallback to the crufty PDFTeX]). Also there are super cool projects like tectonic:

That are optimising all the crufty parts of LaTeX that choose one engine (XeTeX for tectonic for example), and using iftex should allow one to switch to tectonic even if you normally stick with LuaTeX and TeX-Live…


As to what needs change, again read through the Pandoc/Eisvogel template to see where iftex is used, that will guide you on what can be tweaked to benefit each engine… I personally refused to use any legacy 8-bit TeXisms and use UTF-8 characters everywhere, I simply refuse to write It is \SI{17}{\degreeCelsius} outside. when It is 17℃ can be used… LaTeX sadly suffers from font-fallback issues (i.e. if a character is not in your main font, LaTeX cannot search for an alternative font, which is normal behaviour in the OS itself). My hacky solution was to make a script to wrap characters I use and force them to use a font I know has them.

4 Likes

Hi nontroppo,

Thank you …

I feel like a newcomer to this would of XeTeX > LuaTeX > PDFTeX … I was Sssooo naive about switching over to LuaLaTeX … now I’m having to learn about fonts (and a few bits more) unlike I ever needed to do with PDFLaTeX.

I wasn’t even sure what the difference was between the various TeX flavors (that are more like realms) until I read A guide to LuaLATEX … that, although a decade old, outlines the various flavors/realms with the following table (on pg. 2):

If I REALLY want that extra main_memory that LuaLaTeX offers, I’ll have to rearrange my near-term schedule .

Fortunately, my upgraded hardware is able to make short shrift of the 800 page project.

Thanks again,
scrive
:thinking:

P.S. Your make a script (to search for an alternative font) looks very interesting … mind if I ‘borrow’ the code?

Well, [Xe|Lua]TeX are different to PDFTeX, but I don’t think they are more difficult. LaTeX traditionally had a system that only supported packages, and behind the scenes a lot of complexity if you wanted to tweak typography.

[Xe|Lua]TeX uses the universal opentype font standard used across all major OSs, a single and consistent system which supports typographical mechanisms that LaTeX needed multiple additional packages or simply didn’t offer if a package was not available. PDFTeX is easy if you never want to tweak the default typography, but becomes a field of landmines when you do…

Yes of course, that code simply wraps a bunch of symbols not present in my preferred main font to use Cambria Math or Arial Unicode MS. In the LaTeX header you need to add (see my header template here):

\newfontfamily\fixfont[]{Cambria Math}
\newfontfamily\fixfontB[]{Arial Unicode MS}

The script (written in Ruby), uses stdin and stdout, so you can pipe (|) your .TEX file into the script and it spits out the modified .TeX which you can redirect (>) to a file. So for example:

cat my-document.tex | fixLaTeX.rb > modified-document.tex

The script is simple, it is just a series of regular expressions that search and replace text. Often quick fixes like this save a lot of hassle trying to find the correct package or best practive to resolve a complex LaTeX issue…

1 Like

Can a change in the MacOS platform cause previously un-erroring code to start showing (Lua)LaTeX typesetting errors? I’m surprised … should I be?

In my transition from the Catalina MacOS to the Ventura MacOS platform, the same EXACT Scrivener/LaTeX code (when using the LuaLaTeX typesetting engine) will typeset w/o any errors on the Catalina MacOS platform, but results in a number of errors when typeset on the Ventura MacOS platform.

Note: As the Scrivener code is compiled to LaTeX independently on the two platforms, I’ll need to compare the resulting LaTeX code that is compiled on the two platforms to see if there are any differences. Given that the Scrivener code is EXACTLY the same on both platforms, I would not have expected their to be any differences in the LaTeX code that is generated on the two MacOS platforms. Both platforms are nominally running Scrivener image. More on this later …

Is this difference in typesetting behavior on the two MacOS operating system platforms to be expected, or at least not surprising? I’m curious as to what may be causing the errors to only occur on the Ventura MacOS platform.

Surprised,
scrive
:hushed:

1 Like

After considerable time and effort, I’ve been able to complete the first half of my transition from:

  • Scrivener/(Pdf)LaTeX project, on MacOS Catalina (Intel) hardware

toward a

  • Scrivener/(Lua)LaTeX project, on MacOS Ventura (M2) hardware

Somewhat haphazardly and serendipitously, almost by accident, I located two unrelated pearls of wisdom, e.g.:

A. Comments/entries on StackExchange (SE) regarding the arara package,

the second:

B. Nestled in a highlighted comment by Nicola L.C. Talbot within the LaTeX glossariesbegin.pdf documentation (1) regarding the glossaries \glsnoexpandfields command.

In greater detail, a full description of both issues is as follows:

A. In the SE posting: How to use arara with TeXShop, there is a discussion and comments about how to:

  1. “Move the arara engine from the ~/Library/TeXShop/Engines/Inactive folder to the Engines folder itself.”

  2. “Then, In a Terminal window you need to do chmod +x ~/Library/TeXShop/Engines/arara.engine.”

to set the executable bit for the arara engine so that it will run when called by LaTeX. I had NO prior knowledge regarding either arara functioning as an engine (who knew??), and duly clueless as to how to set the executable bit for the arara engine.

Before making the above adjustments, arara simply failed to do anything on my MacOS Ventura hardware. Nothing, nada …

With the above adjustments, arara now runs perfectly on the Ventura hardware, just as it did on my macOS Catalina hardware. Go figure.

B. Nestled within Talbot’s The glossaries package v4.52: a guide for beginners documentation, is the comment:

For some unknown reason, I was suspicious that all the ‘kitchen sinks’ that I had thrown at the many glossary entries in my project (and at this point grasping at straws knowing that my commands were likely anything but robust), I threw in a single line of code to include the glossary’s \glsnoexpandfields command.

Miracles upon miracles, it worked. Whereas my hair was starting to thin due to all the manual extraction that had been taking place, all dozen glossaries in the project were now printing out to the .pdf file perfectly, as opposed to a thoughtful, but frustrating, warning screen offering suggestions that did not address the issue, e.g:

Now with the platform migration from MacOS Catalina to MacOS Ventura accomplished …

My next step is to pursue the transition from (Pdf)LaTeX to (Lua)LaTeX running on the MacOS Ventura platform where the absence of the main_memory = 12435455 limit will (hopefully) allow me to recombine the two projects that I had to cleve apart (in order to circumvent the memory limit).

This will allow me to continue to expand my Scrivener/(Lua)LaTeX project file to whatever size is necessary.

My plan is to post updates once I’ve made significant progress toward that goal.

Thanks for reading,
scrive
:thinking:

(1) “The glossaries package v4.52: a guide for beginners”, by Nicola L.C. Talbot

1 Like