Yeah, it wasn't a good idea to try playing for fun in a party-based gameplay system with a predetermined, closed group. Because it's so great when you have to worry about every step you take, so you don't accidentally put the whole group in danger.
Added after 48 seconds
I don't see an option to "refuse." So it's not recruitment.