The JEDP theory claims that the Pentateuch, the five books of Moses, were compiled from four sources or traditions, which are respectively known as the Jahwist, Elohist, Priestly, and Deuteronomist sources (hence the abbreviation JEDP). This theory is also known under other names, such as the Wellhausen hypothesis (named after Julius Wellhausen, who pioneered the theory).
This theory states that a final editor or compositor (or a number of them) drew upon these different traditions in assembling the Pentateuch and that one can identify the source by noting certain clues in the text. For example, the Jahwist source is supposed to favor the divine name Yahweh "(I AM"), while the Elohist source is supposed to favor the term Elohim or El ("God").
Three of the sources, J, E, and P, are thought to have gone into the writing of Genesis. The reason many think this undermines the historicity of Genesis is that all of these sources are considered inaccurate, written centuries after the time of Moses.
In fact, the conclusion that they are inaccurate does not follow from the idea that they are from a late date. Under the inspiration of the Holy Spirit even late sources are infallible, just like early sources. The conclusion that Moses did not edit or oversee the editing of the Genesis is also not required. Even if J, E, D, and P were real sources, Moses could have been the one who wove them together. There are other problems with the JEDP theory. For example, many passages in Genesis contain references to God using both divine names ("the Lord God" or "Yahweh Elohim").
For an excellent discussion and critique of the JEDP theory, see Before Abraham Was by Isaac Kikawada and Arthur Quinn.
At Catholic Answers, we prefer the GELND theory, which claims that the Pentateuch is a compilation of five sources: Genesis, Exodus, Leviticus, Numbers, and Deuteronomy.