Comparator stays on when subtraction mode and 2 signals have same strength. However it doesn't actually output a signal, so this is just a minor bug. And if you unpower the main power first or both at the same time, it will stay on. You can make isolated on state comparator and it survives block updates and reload. Toggling it or powering it then unpowering it turns it off.
And this is not a duplicate from MC-8776 in which comparator does output a signal

Steps to reproduce

1. Place a comparator and toggle it
2. Power the comparator with same strength from back and side or use setup in the attachment
3. See that comparator stays on and its block state says it is on but actually isn't

I think this happens because in compare mode comparator does output a signal if 2 signals have same strength. So on this condition redstone part of the game tells it to stop outputting a signal but block state part doesn't tell it to turn off as if it were in compare mode, and game assumes it is turned off thus surviving block updates.