Where Did the Idea of Infants with Original Sin Come From?

No. It comes from divine revelation and is found clearly in Sacred Scripture. There is, admittedly, a different theological approach to the matter in East and West, but the doctrine is substantially the same. St. Augustine reflected on original, or ancestral, sin extensively and profoundly, and his treatment of this teaching has been by far the most influential in the theology of the Church.

This being said, certain of the Eastern Fathers treat of it as well, such as St. John Damascene, St. Maximos the Confessor, St. Gregory of Nyssa, and others. It is important not to pay too much heed to the dichotomizing approach of some Eastern thinkers today who make far too much of the difference of approaches. Eastern theology tends to emphasize personal ascetical struggle, and Western the universal fallen state of man. These are complementary, not contradictory, perspectives.