Clement of Rome
Our Apostles knew through our Lord Jesus Christ that there would be strife for the office of bishop. For this reason, therefore, having received perfect foreknowledge, they appointed those who have already been mentioned, and afterwards added the further provision that, if they should die, other approved men should succeed to their ministry (Letter to the Corinthians 44:1 [A.D. 95]).
Ignatius of Antioch
You must all follow the bishop as Jesus Christ follows the Father, and the presbytery as you would the Apostles. Reverence the deacons as you would the command of God. Let no one do anything of concern to the Church without the bishop. Let that be considered a valid Eucharist which is celebrated by the bishop, or by one whom he appoints. Wherever the bishop appears, let the people be there; just as wherever Jesus Christ is, there is the Catholic Church (Letter to the Smyrnaeans 8:1 [A.D. 110]).
It is possible, then, for everyone in every Church, who may wish to know the truth, to contemplate the tradition of the Apostles which has been made known throughout the whole world. And we are in a position to enumerate those who were instituted bishops by the Apostles, and their successors to our own times: men who neither knew nor taught anything like these heretics rave about. For if the Apostles had known hidden mysteries which they taught to the elite secretly and apart from the rest, they would have handed them down especially to those very ones to whom they were committing the self-same Churches. For surely they wished all those and their successors to be perfect and without reproach, to whom they handed on their authority (Against Heresies 3:3:1 [A.D. 180-199]).
It is necessary to obey those who are the presbyters in the Church, those who, as we have shown, have succession from the Apostles; those who have received, with the succession of the episcopate, the sure charism of truth according to the good pleasure of the Father. But the rest, who have no part in the primitive succession and assemble wheresoever they will, must be held in suspicion (ibid 4:26:2).
Moreover, if there be any [heresies] bold enough to plant themselves in the midst of the apostolic age, so that they might seem to have been handed down by the Apostles because they were from the time of the Apostles, we can say to them: let them show the origin of their Churches, let them unroll the order of their bishops, running down in succession from the beginning, so that their first bishop shall have for author and predecessor some one of the Apostles or of the apostolic men who continued steadfast with the Apostles. For this is the way in which the apostolic Churches transmit their lists: like the Church of the Smyrnaeans, which records that Polycarp was placed there by John; like the Church of the Romans where Clement was ordained by Peter. In just this same way the other Churches display those whom they have as sprouts from the apostolic seed, having been established in the episcopate by the Apostles. Let the heretics invent something like it. After their blasphemies, what could be unlawful for them? But even if they should contrive it, they will accomplish nothing; for their doctrine itself, when compared with that of the Apostles, will show by its own diversity and contrariety that it has for its author neither an Apostle nor an apostolic man. The Apostles would not have differed among themselves in teaching, nor would an apostolic man have taught contrary to the Apostles, unless those who were taught by the Apostles then preached otherwise.
Therefore, they will be challenged to meet this test even by those Churches which are of much later date for they are being established daily and whose founder is not from among the Apostles nor from among the apostolic men; for those which agree in the same faith are reckoned as apostolic on account of the blood ties in their doctrine. Then let all heresies prove how they regard themselves as apostolic, when they are challenged by our Churches to meet either test. But in fact they are not apostolic, nor can they prove themselves to be what they are not. Neither are they received in peace and communion by the Churches which are in any way apostolic, since on account of their diverse belief they are in no way apostolic (The Demurrer Against the Heretics 32:1 [A.D. 200]).
Clement of Alexandria
After the death of the tyrant, the [Apostle John] came back again to Ephesus from the Island of Patmos; and, upon being invited, he went even to the neighboring cities of the pagans, here to appoint bishops, there to set in order whole Churches, and there to ordain to the clerical estate such as were designated by the Spirit (Who is the Rich Man that is Saved? 42:2 [inter 190-210 A.D.]).
Firmilion of Caesarea
But what is his error, and how great his blindness, who says that the remission of sins can be given in the synagogues of the heretics, and who does not remain on the foundation of the one Church which was founded upon the rock by Christ can be learned from this, which Christ said to Peter alone: "Whatever things you shall bind on earth shall be bound also in heaven; and whatever you loose on earth, they shall be loosed in heaven;" and by this, again in the gospel, when Christ breathed upon the Apostles alone, saying to them; "Receive the Holy Spirit: if you forgive any man his sins, they shall be forgiven; and if you retain any mans sins, they shall be retained." Therefore, the power of forgiving sins was given to the Apostles and to the Churches which these men, sent by Christ, established; and to the bishops who succeeded them by being ordained in their place (Letter to Cyprian 75:16 [A.D. 255-256]).
SI Moderator - Greg Gordon