City’s first goal came after only two minutes. Simon Clist struck from 25 yards before Matthew’s strike made it 2-0.
Former Bristol Rovers striker Kevin Gall grabbed one back for the home side before Ryan Harley restored the two goal advantage.
Yeovil new boy Jake Edwards got one back for the Glovers but it was too late as City held on to progress to the second round.
City: Stowell; Simpson, Aubrey (Hart 65), Fortune, Harley; Lambu, Hulbert, Clist, Hawkins (McGuire 79); Matthews, M Brown (Lita 65).
Subs not used: Stabler, Anyinsah.