They happened to leave Reinfield just after one of the legendary nobles left the city to go to the capital. This was not by chance. They had inside information, so they knew when the legend would leave.
They didn't leave after the legend so that they could attack it. They didn't have that intention yet, as they didn't need to attack someone that was going to be out of the picture.
The reason why they waited until the legend left was to see for themselves that he truly left, not just pretending to leave. For that, they sent their illusionist to follow the legend and make sure that he left.
After they were sure that the legend had truly left and they didn't need to eliminate him, they left Reinfield City themselves.
They didn't leave alone. They left with some slaves. In particular, they left with slave number 1, their giant and guardian slave; slave number 3, their walker and traveler; slave number 4, their thief and rule breaker; and slave number 6, their knight and farmer.
