If so, you disagree with the one who gave us that rule. Christ never confined moral truth to this principle, as important as it is. Treating others as you'd like to be treated isn't enough. If it were, a masochist would be justified in inflicting pain on others simply because he derives pleasure from receiving it himself.
Jesus made it clear that sin can be committed even when no action against another is involved. In his teaching about lust, for example, he said you don't have to commit adultery with another to be guilty of the sin. Merely to look lustfully on a woman is to commit adultery with her in your heart (Mt 5:27).