I think to answer this question, you have to answer when the left by the early church and then when it was left within the Reformed tradition.
With regards to the early church, there seemed to be the use of non-inspired song pretty early on, though different histories disagree. As
@Ben Zartman mentioned, we have hymn like songs from early church fathers; some even try to say the Apostles themselves quote "hymns" in their letters (though I would believe these fragments to be inspired regardless). Louis DeBoer in his book Hymns, Heretics, and History claims that there is a pattern throughout the church, including early on, of heretics writing songs to propagate their doctrine and the orthodox sometimes wrote their own to counter them. I think the overall point can be made that the early church was predominant psalmody, starting with exclusive psalmody, but exclusive psalmody was not held as a consistent position by many early on and in some cases hymnody crowded out psalmody. Another interesting anecdote is that the Eastern Orthodox church predominately (but not exclusively) uses chanted Scripture in their worship, based on a liturgy they claim dates back to John Chrysostom (4th century).
Then at the Reformation, the Reformed tradition saw a revival of psalmody. Even in Calvin's day, where Calvin tirelessly worked to promote Psalm singing, he was not able to get full buy in to his ideas everywhere he preached. For example, the liturgy at Strasbourg where he preached for many years included hymns sometimes. However, it was uniquely the Scottish Presbyterians and some Puritans who adopted a consistent position of using the Psalms exclusively.
Most of the particular Baptists never adopted an exclusive psalmist position, though this has continued to be a minority position among them.
Most of the continental Reformed used a few non-Psalm songs as well and more quickly went to a broad range of hymnody. Many of the more conservative Dutch Reformed groups have held on to the Church Order of Dordt which permits the singing thusly: "In the churches only the 150 Psalms of David, the Ten Commandments, the Lord’s Prayer, the 12 Articles of Faith, the Songs of Mary, Zacharias, and Simeon shall be sung. It is left to the option of the churches whether to use or omit the song, O God, who art our Father." It's my understanding that this last song is a non-inspired hymn and at the time, the Apostle's Creed was understood to be written by the Apostles and inspired by God. I believe this general position is held by groups like the HRCNA, PRCA, and FRCNA today as well as many churches in the Netherlands (e.g., HHK). The German, Hungarian, and other Continental Reformed churches did not take as strong of a position on psalmody.
Within Scottish Presbyterianism, as has been mentioned the single biggest culprit for the introduction of hymnody has been Isaac Watts, who wrote an imitation Psalter; essentially a book which claimed to be a Psalter, but many of the selections were made in such a way that they no longer were the Psalms of Scripture in any meaningful way. You can
see his Psalter on-line here. This opened the doors to his other hymns and finally to other songs in worship. The American Presbyterians (mainline) were no longer exclusive psalmody by the time they established themselves as a unique body in the US, but the descendants of the Seceders and the Covenanters maintained their exclusive psalmody position for much longer, including the RPCNA to this day.