OnPointerExit not called, if a new UI element suddenly covers the old one

So I have two UI elements A and B, with B overlapping A. Now I move the mouse over A, and it registers the OnPointerEnter. However now suddenly B is activated, covering A. Even though the pointer is no longer over A, no OnPointerExit is called for A.

Anyone have an idea on how to get around this? It's making my tooltip solution behave oddly whenever a button opens an overlay.