Yes, the rating could go down. Because we don't have some kind of game rounds, it's one continuous play, the ratings will decrease over time if you would become inactive in the game.
Here is how it works. At first, the manager's rating is calculated basically combining his bands' rating and adding rating from shop items and money (including your bank account).
The real question is how band's rating is calculated. Nothing too complicated here. For all songs and albums, that was create in the last 3 months, band gets 2x of ratings. For all songs/albums between 3 and 6 months, band gets 1x ratings. For songs/albums created between 6 and 9 months band gets 0.5x ratings. Any song or album, created more than 9 months ago, isn't included in band's rating.
Because you are playing this game for more than 3 months, you start to get fluctuations of your bands' ratings and yours including.
Hope i answered your question. If not, let me know.