According to "Gnás na Gaedhilge" the Munster form ("go") of the indirect relative particle ("a") actually comes from "ag a" as in the following examples from Céitinn:
"An ceirtbhreitheamh ag a raibhe an Iodh Morainn aige";
"Ón muicidhe ag a raibhe i gcoimhéad aige".

I had assumed is was from "go" as used with indirect speech.

That's very interesting. To this day there is a tendency to repeat the preposition in Munster Irish e.g. 'fear darb ainm dó Seán' rather than 'fear darb ainm Seán' or 'fear arb ainm dó Seán'.

But it doesn't really explain why 'go' should be used in the absence of 'ag' e.g. 'Sin é an fear go rabhas ag caint leis' nó 'Sin í an bhean go rabhamar ag caint fúithi.' The prepositions in those sentences are 'le' and 'faoi' rather than 'ag'. Perhaps 'ag' has been generalised to cover all eventualities?

More I see, the more I think Irish was easier centuries ago!

Tá sé sin grannmhar, a Bhearn!