Well one approach is to get a decent web browser. 
But what you propose, I think, is something I did for ages. I made a little “portal” (as we called them back then) of my own, stored locally as an .html file, and set that as the browser’s home page.
It’s really not hard to learn enough to do that, especially if you’re not set on it looking terribly fancy.
<h1>Name of Section</h1>
<ul>
<li><a href="https://www.literatureandlatte.com/">Lit & Lat homepage</a></li>
</ul>
There, that’s enough for a heading and a bullet list of links, where each <li> is a List Item.
Are there still books written to understand the pragmatic basics for creating HTML documents and to be able to build a foundation for learning more if necessary?
Sure! Piles upon piles, but these days I’d start with the bounty of free tutorials on the web itself. I’d only buy a book if I was looking to get into the field of web design.
But I would maybe look at software that could help with this basic idea, first. Something like Logseq is very good at annotating lists of links and connecting pages of thoughts together. Even Scrivener has a decent feature set for going above and beyond what most browsers give you for bookmark management.
I did what I did with .html because I already knew it, and so it was for me a natural way to tweak my browser setup. I don’t know if I’d go into learning HTML just to organise my bookmarks into some pages though. I’d want some other reason to invest the time into it, myself—like learning to make better ePubs.