On day 44, the purple faction opens a border guard at (107, 58, 0) and sets off on a free roam.
On day 57, the orange faction opens a guard at (126, 117, 0) and also goes for a walk.
Although, unlike Boss and Verdigo, they are quite far from Dragons Eye and don't cause any particular trouble.
Also, in the late game, gold and resources are regularly reduced, but I don't know how critical this may be.