The Grand Secretary's first reaction was that this Military Affairs Office could divide the power of the Cabinet. Originally, military affairs, no matter the size, would have to be discussed by the Cabinet. Now, it is entirely possible to allocate this portion of authority to the Military Affairs Office alone, and other Elders would have nothing to say about it.
Because the biggest beneficiary of this matter is Your Majesty.
Moreover, border military situations used to go through the Cabinet for discussion, which was inefficient. Now, they can be handled by a Minister specifically responsible for military affairs, making the process naturally much more efficient.
The problem is, this matter is digging into the roots of the Cabinet's power.
As the Cabinet Chief, he could naturally be the chief Minister of Military Affairs directly, but the appointment of other Ministers of Military Affairs is not something other Cabinet Ministers could decide—it's entirely up to Your Majesty.