Because everyone's goal is the same, the Professionals have all withdrawn and haven't taken action.
Those people outside were hunted down, but they completely ignored it, and instead watched the spectacle from the sidelines. As time passed, these people were gradually forced by the Dragon Clan's pressure to unite once more.
However, due to their internal conflicts, it's difficult for them to cooperate as closely as before.
Also, because their numbers have significantly reduced, they've formed several different teams. In the Dragon Clan's encirclement, they were no match for these empowered giant dragons. Not long after, they were scattered once again.
And this time, having been scattered, these people no longer had the strength to resist. No one knew how many were hunted down; those remaining had already hidden themselves away. Facing the giant dragons, they could only pray not to be discovered.
