Regardless of turbo sizing, which has nothing really to do with surge. You already said yourself that surge is pressure attempting to slow or reverse the turbo impeller in an off throttle condition. Changing the turbo itself will not change that, regardless of how perfectly it is matched to your engine.
If you have no means to relieve the pressure on the closed throttle, the pressure will cause slowing of the impeller. Period.
So by not having one you are losing performance every time you lift and the turbo slows down in response to back pressure. When you get back into the throttle you have a delay in boost as the impeller spools back up to speed.
There is a reason every OEM turbo and blower on new cars has some form of BOV/bypass valve. It creates less stress on moving components, better economy, and more performance.
