All I know is that I have followed a bearing and ended off being out by some way. Sorry if its stating the obvious but the old grid to mag add and mag to grid get rid thing currently at about 3 degrees? Can't remember right now (always remember on the hill just not without map and compass to hand). Not saying that is one reason why you were out but could be part of it IF you were not paying attention and forgot it.
Also, without something to actually sight on it is rather hard to follow a bearing accurately I think just by watching the needle and walking. You always veer a round a bit. Depending on how far out this could also be a factor.
I also know of someone who (despite being very, very experienced) fell through a cornice on a wide, flat plateau type of hill in Scotland. The group deliberately walked on the side away from the steep clope and crag on it but in the cloud walked to a bearing. After some time the ground opened up and they were through a cornice.The reason was simple, they didn't allow for the wind blowing them off course by quite some way.
All these are small factors and simple things to correct but are all capable of resulting in walking a bearing and ending up in the wrong place. It doesn't take much over some distance to be way off your desired location.
However I have no idea why FMA's compass was behaving erratically. However when I got temporarily, locationally displaced (lost) while using my compass (that I was convinced was wrong) I replaced the compass. I did check it against the old compass and they both pointed in the same direction. However on a recent walk my newer compass did move around on Tarn Crag area. It pointed about 3 or 4 degrees off but would then point back the right way. I compared it to the old one that I thought had gone wrong and that was pointing in the direction the other one settled to so I assumed that to be right. I just needed to allow it to settle properly. Not good while moving.
Since that time I got another type 4 silva (second one, I've used a 7nl and another one I can't remember the code of). This pointed to the same direction as the other two in my house.
All I can say is some compasses can behave funny from time to time. Although I'd say something is externally acting to make it so. This can be rocks like in the Black Cuillins or a mobile phone in your pack. I put it down to my new smart phone and my old work phone both being on in my pack. Although I just can't be sure. I still got a new compass anyway to be sure. You do need faith in yoru compass I think.