No. Jesus stated in Matthew 5:28 that a person can commit sins of sexual impurity even in his thoughts: "But I say to you that everyone who looks at a woman lustfully has already committed adultery with her in his heart." The same is true of fornication (premarital sex).
Looking at a woman (or man) to whom you are not married and indulging in lustful thoughts counts as committing fornication in your heart. If indulging yourself in mental lust for a man to whom you are not married counts as fornication, how much more so will intimate touching in which you partially act out the sexual desire you have for another?
Though some priests may not like to say so, fornication is a grave (mortal) sin. Paul says, "Now the works of the flesh are plain: fornication, impurity, licentiousness . . . and the like. I warn you, as I warned you before, that those who do such things shall not inherit the kingdom of God" (Gal 5:19-21). This is a severe teaching. It is one many unmarried people find hard to accept, but it is the clear teaching of Scripture, and we must hold to it.
Sometimes people rationalize extramarital sexual practices on the grounds that by committing a lesser sin one may avoid a greater one, such as fornication. There are two problems with this. First, as the Holy Father made clear in his encyclical Veritatis Splendor, one may never do something intrinsically wrong in order to avoid a problem. We cannot do evil that good may come of it.
Second, this strategy simply doesn’t work. If you find it difficult to restrain yourself sexually, following this priest’s advice will not make it easier to control yourself—quite the opposite.
By the way, you might want to ask this priest yourself and not rely on your fiancé to interpret the priest’s remarks.