Providing that you're ALL sober on the day...
There is a 'half-way point', and believe it or not, it's pretty much half-way too! It's where the first steep slog levels out before a steeper slog continues to the summit. I'd suggest that the 'walkers' in the group get cracking and set off early, with a view to reaching the summit, then come back down to the half-way point. The more reluctant folks can take a bit more time and just aim for the half-way point. Keep in touch with each other by mobile phone and see if you can all make it to the half-way point around the same time. Have lunch together, then head back down the way you came. There isn't really an alternative route that's going to work if you have people basically doing two different things.
If you all set off together early, and keep together, and reach the half-way point in good time, then you could have an early lunch there. Afterwards, let those who want to climb to the summit get cracking... but they'll need to keep moving. If they really are 'walkers', then they could come down another way, but I think time won't be on their side after being in the company of slower folks until lunchtime.