I've only seen planet size shrink when this ocurs usually because this event only seems to hit my ocean or terran planets. These planets usually have 60+ (had a size 100 planet get hit with that once IIRC) population, so it's hard for a size 10-20 planet to ever catch back up to that.
