100% up to the ref. They have to be careful to make sure they're being fair and consistent. If they allow you to switch balls throughout the game, they can't change it to help speed up the game for the team who is behind. Their are several teams who change balls after a long pass, bad pass, have a kicking ball, ect. It's a no-win situation for the official and nothing in the book to stop it from happening.
