It is utterly broken, like most of the diplomatic contracts. Treaties are doing something different than described, non-agression can be broken any time, when your allies are attacked you don´t even get a dialogue prompting you to join them or break the alliance. Even Total War games have this and in Stardock games diplomacy is broken in this regard since FE
Hopefully it gets fixed some day...