2 Corinthians 5:18-19

All this is from God, who through Christ reconciled us to himself and gave us the ministry of reconciliation; that is, in Christ God was reconciling the world to himself, not counting their trespasses against them, and entrusting to us the message of reconciliation.

Christ did not tell every schmoe, "If you forgive the sins of any, they are forgiven; if you retain the sins of any, they are retained." He only told ordained and apostolic schmoes this.  And these apostles did not confer the sacrament of Holy Orders on every convert, but only on certain people.  For this reason, the minister of the sacrament of Reconciliation is necessary in order to celebrate the sacrament.  This does not mean you can't say "sorry" to God except in the confessional.  Nor does it mean you can't patch things up with the friend you hurt without some priest acting as arbitrator.  But it does mean that the priest really is given to us as a means of grace and a "minister of reconciliation."  Today, thank God for that gift of grace.