I often find the Format ▸ Font ▸ Show Fonts panel to be more efficient than using the dropdown anyway. For one thing you can type in the name of the font you want in a more natural way (rather than just cycling through first-letter matches one by one), and since it isn’t previewing the font there are no rendering issues.
But otherwise no, it just pulls from the system font list like most other programs will. If you have too many fonts installed for what you need, you can always go in and do some spring cleaning.
I have a similar issue, though for me it hasn’t risen to the level of more than an extremely minor annoyance. I wouldn’t even bother to say anything were the topic not already here…
I have the proverbial Sh*t-Ton (a technical term) of fonts installed on my system, that I use in other programs (drawing programs mostly) so “simply uninstalling them” isn’t really a practical idea (though it is a good one).
I have often wished there could be a way I could go “pre-select” some fonts to utilize in Scrivener that would be available in the various drop-downs and font dialogs, instead of the aforementioned “ton” of fonts that are there now.
You might be able to find a font manager application that would let you make fonts selectively available on a per app basis. Especially on Windows, there might be some small free open source utility that could perform this trick.
If you don’t mind a slightly dated UI or running applications as admin, I have been using AMP Font Viewer for around a decade, including a few years on Windows 10. It allows for temporary installation of fonts.