A recent court case involved a situation in which a prison system had taped a prisoner's confession without his knowledge or the priest's. (The prisoner and the priest communicated through a glass partition, I understand.) A Protestant minister told me that the tape should be allowed in court because the Bible requires us to tell the truth, not keep secrets, and thus there is no basis for a priest-penitent privilege.
The Protestant minister is flat-out wrong. There are multiple verses in the Bible that talk about not revealing secrets (e.g., Prv 11:13, 12:23). While there is a biblical requirement not to lie, there is no biblical requirement to spill all the truth one knows whenever anyone asks.
Second, the minister appears to admit that the attorney-client privilege is valid and to be honored. But if what he says about the Bible requiring full disclosure to the authorities were true, this would demolish the attorney-client privilege as well, since what the Bible says about truth-telling applies just as much to lawyers as it does to priests.
Third, society deems the attorney-client privilege beneficial, even if some guilty people go free because of it. How much more beneficial is the priest-penitent privilege, which entails a requirement of true contrition! The advantage to the many in knowing that their sins will not be broadcast justifies the practice, even though it means some people guilty of criminal offenses (which are by no means the worst offenses discussed during confession) will escape civil (though not divine) justice.
Fourth, the Bible requires us to confess our sins to others (Jas 5:16). On the Protestant model one confesses to just anybody and does so for purely therapeutic purposes, not for absolution. Even in that model the confider-confidant privilege would be needed. People need to know that their sins will not be publicly revealed, or they will not confess them.
Finally, Catholics believe that the priest, in administering sacraments, assumes the role of Christ; what is said by the penitent is as inviolate as anything said directly to God in prayer.