Each entry in the draft folder can have text (regardless of the icon it uses, folders are a bit of a fiction). So yes, you’re scrolling through each potential chunk of text when using those buttons, and there is no way to skip some items based on their icon type.
Think of it as a visual cue that you’ve switched major sections (chapters, whatever). 
They can also be, since by default they don’t compile their text content, a convenient place to put chapter notes.
But if it does bother you, something you could consider is not using folders so much, and just nest sections within sections. The first part of the chapter can be at the top, and then the subsequent parts of that chapter would be text sections nested beneath it, then you don’t have blank spots. That is how I work, almost exclusively.