We are not born with the indwelling of the Holy Spirit. The Catechism of the Catholic Church states, "The human body shares in the dignity of ‘the image of God’: it is a human body precisely because it is animated by a spiritual soul, and it is the whole human person that is intended to become, in the body of Christ, a temple of the Spirit" (CCC 364, emphasis added).
Perhaps your instructor was speaking of baptismal birth: "Baptism not only purifies from all sins, but also makes the neophyte ‘a new creature,’ an adopted son of God, who has become . . . a temple of the Holy Spirit" (CCC 1265).