[T]he children of the covenant are by baptism distinguished from the world and solemnly received into the visible church.
Although our young children do not yet understand these things, they are covenantally holy before baptism, and therefore are to be baptized. For the promise of the covenant is made to believers and to their seed, as God declared unto Abraham: “And I will establish my covenant between me and you and your offspring after you throughout their generations for an everlasting covenant, to be God to you and to your offspring after you.” (Genesis 17:7) In the New Testament, no less than in the Old, the children of faithful believers, born within the church, have, by virtue of their birth, interest in the Covenant and right to the seal of it and to the outward privileges of the church. For the essence of the Covenant of Grace is the same in both the Old and New, and the grace of God for the consolation of believers is even more fully manifested in the New Testament with the coming of Christ. Moreover, our Savior admitted little children into His presence, embracing and blessing them, and saying, “…to such belongs the kingdom of God.” (Luke 18:16) So the children of the covenant are by baptism distinguished from the world and solemnly received into the visible church.
By virtue of being born of believing parents, children are, because of God’s covenant ordinance, made members of the church, but this is not sufficient to make them continue as members of the church. When they reach the age of discretion, they become subject to the covenant obligations of: faith, repentance and obedience. They then must make public confession of their faith in Christ or become covenant breakers, subject to the discipline of the church.