If you're allowed to write your own rules, *anything* can be made possible. So, yes, in theory it should be possible to invent a 4D physics that allows orbits. Just don't expect it to look anything like what we call "physics" in our universe.
Having said that, though, trying to come up with a self-consistent system that has some particular desired emergent properties is HARD. Extremely hard. So hard, in fact, that it's difficult to come up with anything that is both non-trivial and intuitively comprehensible. This is part of the reason we usually start with real-world physics, which we already know has interesting emergent properties (y'know, like chemistry which allows life to exist, etc.), carry it over to 4D, and make some minor adaptations to work with the additional dimension. The idea being that, hopefully, the minor adaptations will preserve most of the interesting emergent properties in a way that we can intuitively understand.
Unfortunately, this has turned out not to be the case -- yes, the resulting system has a lot of characteristics that are analogous to our 3D world, but not everything carried over, and included in that number is the existence of stable orbits. This is rather unfortunate, since the lack of stable orbits means that stars and planets as we know them can't exist in 4D, at least, not with the physics that's familiar to us. OTOH, it's not altogether unexpected -- modifying a complex self-consistent system like physics has a way of producing lots of unexpected consequences. After all, it's hard to design a system from scratch with some given desired emergent properties; the same applies to modifying an existing system while preserving its emergent properties.
This is why most fiction authors build their worlds by rationalization instead of deriving everything from first principles. It's far easier to rationalize something than to invent a system that will somehow through a series of complex interactions produce the phenomenon you want. The same can be said of 4D orbits: it's probably easier to take the existence of 4D orbits as an axiom and then try to rationalize how this might have come to be, rather than trying to invent some new physical laws that will eventually result in stable orbits. However, this approach comes with its own disadvantages: sometimes a particular rationalization appears to work well at first glance, but upon further investigation turns out to have unintended consequences that may not be desirable. And even if there are no obvious unintended consequences, the result may not be very insightful into the nature of 4D -- it won't tell you more about 4D than Harry Potter tells you about the real world, since the rules are made-up.