And by this I mean explanations of how things work in this universe. ... 4-5 pages here, 4-5 pages there.
That's what I call the tour-bus approach.

The boundary is crossed, in my opinion, when world-building is introduced that isn't directly involved with understanding the story at hand. And you rarely, if ever, need 4-5 pages of exposition for that. Such lore dumps will get in the way of the story. When you stop the story to explain something, you're putting on hold the core elements that make people want to read on because the questions in their head are about the
characters, not the world. So if you're talking about something in the world that
doesn't affect the characters in an obvious way, the mind will wander and the information will likely be lost. We need to engage emotionally with the material if we are to remember it, and we primarily do that through the characters.
(As with everything in writing, there are exceptions to this, but I believe it requires a lot of skill to pull it off. In short I think you need to make the world itself, or parts of it, into a character, and that's a tall order even for someone with a lot of experience.)SFF writers are always treading a fine line when it comes to balancing the need for people to understand the world with the need for the story to be the main driving force. You may write tons of worldbuilding material for your background work (I know I do), but the less you can put into the final text, the better.
However strange the world is to the reader, it's everyday life for the characters. Stopping to explain concepts that are common to them will cause increasing the narrative distance, causing detachment.
Of course, sometimes it's necessary for the reader to know certain things about the world in order to understand the story, but this is where I think the old and well-worn adage
"show, don't tell" is important. If a writer can show us how the world works as a part of the ongoing story, that's way better than an impersonal lore dump.
This is the advice I keep giving to writers: learn what story is, then attach everything else onto that. I guarantee you it will make for a more engaging novel -- and there are plenty of clever techniques that allow you to build your world convincingly while adhering to that basic principle.
(Sorry if this came out too much as writing advice; it's just how I think.)