One of the advantages of Scrivener compared to LaTeX is that Scrivener supports powerful annotation. This feature is very useful, however, frankly speaking, it does distract the attention of the writer. I would suggest that please allow the user to define font size for inline annotation. So that we could give it a smaller font size. It will be a perfect workaround.
There isn’t an option for that, however you can choose a colour that is less dashing. 
Just put your cursor anywhere in the annotation and use the Format Bar text colour tool to change it to something a little more subtle.
Thanks AmberV! I already do that, but I think they are still a distraction for me. If you could save font size as well it would be perfect.
I’ve got a solution to this! You can use predefined formatting to apply annotation in specified font size!
This is true, that should work for most scenarios, and you could even override the Shift-Cmd-A shortcut to trigger it. The main drawback to using a preset is that you cannot toggle its usage off by triggering it again. So if you tend to use inline annotations to jot down notes while you’re writing (as opposed to inserting them later), it’s a bit of a bother getting your normal font back.