I agree. At the most you can complain about the other player if you are sure that he is the cause of the lag, but in the end the match should remain always fair because the handicap affects equally both the opponents.
I assume that we both are being screwed and I try to refrain from shittalking unless its fired at me first.