Yes, I would say that if LaTeX is your comfort zone, there is nothing about how Scrivener is set up that would get in the way of you using it. And personally I would always prefer to use that to any kind of Word-based workflow, even for very simple (from a formatting and content standpoint) documents. As you say, the typesetting is nowhere near as good, and really if one is going that route they should be using desktop publishing to handle the final formatting rather than Word. Compiling is also, as you say, a bit more complicated if you’re used to a simpler markup based environment. You also benefit from better cross-referencing than the stock Scrivener setup provides, being able to refer to specific equations and figures by label.
At the most basic level of usage, imagine Scrivener is a plain .tex file editor and compile to plain-text. That’s all there is to it, though that very basic level of usage does mean composing in raw LaTeX for all things.
You can then build up from that basic concept by delegating routine syntax output to the compiler. The built in “General Non-Fiction (LaTeX)” starter template is a good place to go if you want to see an example of that pressed to the limits (for example having headings turned into \section{…} code with automatically generated labels and functional internal cross-referencing). This template is designed for pure LaTeX-based writing, rather than using Markdown as an intermediate generator. However one could get away with very little raw LaTeX in the editor (you do still have to be mindful of the fact that what you type into the editor will be dumped straight into the .tex file—which wouldn’t be seen as a downside by some).
Of course, because we are converting Scrivener’s outline to heading structure, it does need a little more Section Type / Layout wiring (though you’ll find it is set up with suitable defaults). Not having to worry about that at all is one of the chief advantages to using Markdown, I’d say.
For a Markdown-based workflow, I tend to start extremely simple, but if you select “MultiMarkdown → LaTeX” as your compile output, you’ll note a number of starting point Formats you can choose from in the left sidebar. Those are all pretty vanilla and there is a bit of a learning curve in customising them. Generally I’d say take a look at how the “Modern” example format is set up (double-click to duplicate and edit it), where you can put your preamble and such right into the compile settings. For most things, that will be the easiest way to go.
Since it sounds like you have a preamble already developed, that’s what I’d take a look at! I do still use a second tool to typeset, but do note from the LaTeX template’s “Processing” compile format pane, you could automate straight to PDF if you wanted to. I provide a little sample script for that.
By all means!