The Catechisms clarified it for you? What did you stumble over, and what particularly about the Catechisms clarified this issue for you?
I know you didn't address this to me, but I've spent a lot of time in the Larger Catechism, so I'll throw in the whole section on the Covenant of Grace with some commentary. I'll say too that along with
@Reformed Covenanter and his comment you are replying too, and the article by
@RWD in post #14, I'm almost amazed I didn't catch this in the WLC either.
Q. 31. With whom was the covenant of grace made?
A. The covenant of grace was made with Christ as the second Adam, and in him with all the elect as his seed.
Q. 32. How is the grace of God manifested in the second covenant?
A. The grace of God is manifested in the second covenant, in that he freely provideth and offereth to sinners a Mediator, and life and salvation by him; and requiring faith as the condition to interest them in him, promiseth and giveth his Holy Spirit to all his elect, to work in them that faith, with all other saving graces; and to enable them unto all holy obedience, as the evidence of the truth of their faith and thankfulness to God, and as the way which he hath appointed them to salvation.
Q. 33. Was the covenant of grace always administered after one and the same manner?
A. The covenant of grace was not always administered after the same manner, but the administrations of it under the Old Testament were different from those under the New.
Q. 34. How was the covenant of grace administered under the Old Testament?
A. The covenant of grace was administered under the Old Testament, by promises, prophecies, sacrifices, circumcision, the passover, and other types and ordinances, which did all fore-signify Christ then to come, and were for that time sufficient to build up the elect in faith in the promised Messiah, by whom they then had full remission of sin, and eternal salvation.
Q. 35. How is the covenant of grace administered under the New Testament?
A. Under the New Testament, when Christ the substance was exhibited, the same covenant of grace was and still is to be administered in the preaching of the word, and the administration of the sacraments of baptism and the Lord’s supper; in which grace and salvation are held forth in more fulness, evidence, and efficacy, to all nations
The Catechism is clear here in Question #31 that the non-elect has no title to the benefits of the Covenant of Grace. The only recipients of its benefits are the elect in Christ. The elect are Christ's covenant children, the true Israelites and children of the promise in Romans 9. No one else. But even then, no title is given to them until as shown in Question #32 that they come to have faith in Christ, in which all the covenant blessings become theirs'.
Questions #33-35 are simply commentary on how God administers the Covenant of Grace to His elect in time. In the time before Christ there are certain ordinances to teach, represent, seal and apply the benefits to the elect (circumcision, passover, etc.), then when Christ came the same Covenant of Grace is administered to the elect, but with different ordinances--preaching, baptism, Lord's Table. These are not two different covenants with different substantiae, but one covenant delivered in two different forms.
Yet despite the Abrahamic and Mosaic being for the sake of the elect, nonetheless the sign of the covenant was given to the children of believers. The exclusivity of the definition of the church, or the restriction of covenant children per Question #31 to the elect in Christ, apparently has no bearing at all on who is to be circumcised in WLC #34 even though the Abrahamic was meant to bring salvation to the elect in the time before Christ. If it does not restrict the subjects of circumcision, it does not restrict the subjects of baptism.
Looking at the article in post #14, it now makes wonderful sense how it can be said that the Abrahamic Covenant is established with the elect only. After all, if the purpose of the Abrahamic Covenant was to bring about the One Seed, Jesus Christ, then it makes sense that the
Abrahamic Covenant and all the Mosaic institutions are aimed at the salvation of the seed in Christ in the time before Christ.
Thus, it is not necessary to define the church in such a way as to include unbelievers, nor is it necessary to define covenants in such a way that unbelievers are in covenant with God. John Murray was critical of those who made the attempt to do so. I wonder if the church has often been defined in Presbyterian circles in such a way to try to grasp the visible/invisible distinction, or purportedly remove a Baptist argument. Now I see so clearly, it doesn't matter.
@Reformed Covenanter forgive me if I stole your thunder. I couldn't help writing now that I've seen this.