I found some cloud video service which apparently produces decent videos. I doubt it can produce looner stuff but at least it'll be better than my attempts
Here's an example what can be done https://civitai.com/images/79887240
So the question is, do I try messing with that service?
The foremost goal with all of this affair was for ME to learn how to produce AI stuff, hence it should be me tinkering with my PC at home not relying on some cloud services. Thus using this tool goes against my goal to some extent.
But if you want to see results soon, that is a way to go. Their subscription isn't very expensive too. And I'm hitting my hardware limits when I do AI videos, so I have to upgrade my PC before making quality stuff, and that's not gonna happen very soon