When you are creating or modifying a style, note the Highlight options in the bottom half of the window. This is something that needs to be optionally enabled, and once you do so you can then select a colour.
You may notice in practical usage that this works a bit more like a line highlighter than a box. We’re employing a little trick in the Windows tutorial, where this paragraph style has one tab stop at the very right edge of the editor. So on the last line we add a tab, which extends the highlight over and makes it look like a box. This is okay for something meant only to live within Scrivener; I wouldn’t want to do that for anything meant to export.
Lastly these are purely cosmetic, a writing tool if you will. They do not compile to formatting. However since you will be using styles to achieve them, it is pretty simple to add true box formatting to the styled text in a universal fashion, once compiled and opened in a tool that can do so.
The Mac tutorial uses a different method entirely, that will export, but the result is a bit sub-optimal in that it is a single-cell table. Tables aren’t strictly speaking meant to make boxes like this, and won’t always work the way you want depending on the output type—but some do consider it a valid approach. I would myself prefer to use styles and real boxes though (along with a template that has all of my styles formatted the way I want).