Abiy announced a raft of dismissals on Sunday as the conflict escalated and is pushing the country towards civil war, Reuters reported.
His office announced the changes on his twitter feed, giving no reasons for the changes.
Nine million people risk displacement from the escalating conflict in Ethiopia's Tigray region, the United Nations said, warning that the government's declaration of a state of emergency was blocking food and other aid.
Abiy is pursuing a military campaign he announced on Wednesday, despite international pleas for dialogue with the Tigray People's Liberation Front (TPLF) rather than risk civil war.
Tigrayans dominated Ethiopian politics for decades until Abiy took office in 2018 and are fighting his efforts to reduce their influence.