I am mainly a nature (scenery, flower, animal) photographer, though sometimes I get lucky and snap a nice picture of a person.
And looking at the stock photography sites, not only did I find out that there are a lot of nature pictures to deal with, but very few nature pictures get downloaded more than five times, if that.
Should I try to change topics, or should I stay with nature photography?

I think it is important to have a diversified portfolio. Maybe these list photo.hlehnerer.com/WhatIsSelling.html from the different agencies will help you a little with the inspiration of what else to shoot.