Absolutely you can have fantasy without extra species, Psychic powers, magic, paranormal, supernatural etc. The simplest is "Alternate history". It's more difficult without the support of common fantasy tropes and stereotypes. The less like our own world it is, the more difficult to do well.
Ultimately more "magical" fantasy or "High" fantasy is either using ad hoc "magic" to fill plot holes (JK Rowlings) or it's a background and really a support, almost a maguffin to the plot.
There are many gradations of fantasy. One reason is to write mediaeval flavour stuff without having to get every pesky detail right. Such as when did they have hops so non-lager beer would keep (see non-hopped ale, hopped ale and Beer vs Lager). When did they have gunpowder? Earlier than most think. Maybe you want 17th C style stuff but without black powder.
Maybe you want 14th C. style but with chilli or potatoes or turkey or prickly pear or tomatoes or large strawberries (all new world, old world strawberries were small, a little like alpine).
Or you want a "what if" 5th C. were the Roman Empire didn't fail, or one where the Romans didn't destroy mainland European Culture. Or The Inca conquer Spain.
Or the Chinese don't abandon exploration, or don't fall to Mongols, or Europe DOES fall to the Mongols, or does fall to the Moors (no Charlemange). Or Napoleon beats the Russians and the UK.
Joan Aiken's 'Wolves Chronicles" might have no supernatural in
most books.
https://en.wikipedia.org/wiki/Joan_Aiken#Wolves_ChroniclesThey are obviously fantasy.
Wikipedia
Many of Aiken's most popular books, including the Wolves Chronicles (also known as The Wolves of Willoughby Chase series or the James III series), are set in an elaborate alternative history of Britain in which James II was never deposed in the Glorious Revolution, but supporters of the House of Hanover continually agitate against the monarchy. These books also toy with the geography of London, adding a Canal District among other features. Wolves have invaded the country from Europe via the newly built Channel Tunnel. The novels share a varying cast and a variety of interlinked child protagonists – initially Bonnie Green, but subsequently her itinerant friend Simon, Simon's intrepid Cockney friend Dido Twite (the heroine of most of the books), Dido's half-sister Is and Owen Hughes (son of Dido's Royal Navy ally Captain Hughes).
I'd be delighted if I could write 1/2 as well.
She has other more "magical" books. While "Harry Potter" has far more "magic" ... well I know which author I prefer.
Similarly some of the Mary Stewart books have a fantasy feel with no "magic". Thornyhold, leaves you unsure. She did do an Arthurian series, but I prefer all her other books more.
Georgette Heyer's Regency books are well very researched (she bought letters and had 1000+ books on the subject). I'd call them "fantasy lite" Romance. Some more so than others. But all good fun. Victoria Holt OTOH is more stereotypical less fantastical period romance.