Hi X0X0. Welcome to the forum. 
Margins are a bit tricky.
As there are no pages in Scrivener, the right margin is set relative to the left side.
It is easy to tune in Sigil.
Other options are 1) to set it as a style:
Or 2) custom set the right margin to match the left one:
(I am not 100% sure about this one. Been a while since I last compiled to epub.)
. . . . . . .
In Sigil (free to use – and personally the solution I’d go for), all you have to do is match the second value to the fourth, and your paragraphs will be centered.
Of course, 10rem is quite extreme (don’t do that 
). Just use whatever value you’ve got as the fourth, and type it in as the second.
/* Default paragraph formatting */
p { margin: 0rem 0rem 0rem 0rem; o; }
→ That’s Space before - Right margin - Space after - Left margin.
. . . . . .
If you used styles in your book, the answer will be a combination of option 1 and Sigil, or option 1 and 2.
Or you can manually set them all right in Sigil, one by one in the stylesheet. (Should take no more than a few minutes.)
If you do that, keep all versions, as you will otherwise have to redo it after each new compile.
Keeping a previously tweaked version (YourCompile.epub) would allow you to simply copy paste your previously tuned stylesheet from a version to another. (In Sigil)
. . . . . . . .