The zoom level on the user interface resets to “fit to display” every time the lowest layer (i.e. the background) is selected from the sidebar.
This gets annoying rather fast; so, in my opinion it’d be best not to touch the zoom level when a layer (no matter which one) is selected from the sidebar – my intuition is that I (i.e. the user) is the one who has set the zoom level to something, so it shouldn’t be fiddled with without the user explicitly making the choice to do so.
Example images attached.