I agree that sitting it right is important so it doesn't move around too much. I read somewhere that the bag should rest in the small of your back and try to set up mine that way.
I thing you're going to carry anything on your back that won't impact on a jumper.
This article might help
How to carry your carry bag
