If it is only your neck that feels a bit loose, then a buff will solve the problem.
When you are toiling uphill, you will appreciate the extra venting at the neck area, then put the buff on when you stop generating heat.
If it is close enough fitting around the rest of your body, then you have a good fit.
The fit of Buffalo doesn't suit me, but the Montane Extreme does. But I prefer the pile thickness/weight of the Buffalo over that of the Montane Extreme.
It sounds as though you have a good buy. It is unlikely that you will ever find a perfect fit.