RectMask2D and ScrollRect update

I have a ScrollRect with a viewport that contains a RectMask2D.
The content of the ScrollRect has a list of children, expanding beyond the viewport (so they get culled by the RectMask2D).
I remove the top most child and move to the top of the list.
I do so by first setting the childs parent to null and then destroying it, then I set the ScrollRect's verticalNormalizedPosition to 1. This works fine.
But any element (or subelement) not visible, when doing the verticalNormalizedPosition = 1,...

RectMask2D and ScrollRect update