That feeling goes away with time and experience Kuro. I know your relatively new to riding and just got a new ride recently so it's understandable that you'd feel that way but over time it'll stop bothering you. Adapting to countering a passenger's weight and lack of skill is something you just kind of get used to. Another rider on back you'll pretty much not even notice under normal conditions, they instinctually lean properly, and a poor passenger will generally weigh far less than you and be easy to counter.

I'd def have to agree with Seaford, it's nothing to get offended by. People love bikes and generally want rides, no big deal.