I have left my car overnight at Gospel End Pass and it was fine. If you are planning to go up Waun Fach you could go via Y-Grib. This is a good route and you can park at the Castle Inn car-park at Pengenfford.
If it is the prioroy you are more interested in then you could probably leave your car at the Half Moon Inn at Llanthony (especially if you have a coupke of scoops first).