As a current flight scheduler, I feel well-qualified to answer this question. In fact, I actually had to deal with this issue last night, again.
If you don't jive with a certain IP, tell your scheduler or flight commander. At least in all the flights I have been in, nobody has ever had a problem with making the change, and the student isn't looked at any differently because of his/her request. If it happens twice.......well, its still a toss-up. 3rd time? It ain't gonna happen, and the problem isnt with the IPs.... its you!
If the problem is that the IP is talking too much, professionally/tactfully, tell the IP. When I was a student, my best bud was flying with an IP in the formation. After the first set of wingwork, which he wasn't performing too well on that day, he was being overloaded byt talking from the IP. He finally said "Sir, please, I can't concentrate!" The IP then went cold mic for the rest of the sortie, and he proceeded to fly a very nice sortie from then on.