I would like to support a writer on our game project and automate compiling their .scrivx file into mmd for import into the game.

Are there any command-line flags I can use on the .exe to automate compilation with certain settings?
Or an API?
Or a plugin for this?
I have tried searching and not really found anything.

There are, as far as I know, no APIs or hooks to allow external automation like this. The best that you can do, based on past queries, is use some sort of programmable macro system like AutoHotKey that allows you to control the UI and simulate key presses and so on.