If you warm it up a bit with a "hot air pistol" <- dont know the american word for it...
and put on a glove and try to wiggle it while you pull i belive you gonna succed.
I removed my muzzle sight with a quick tap on a block of wood against the sight. It is a snug fit though. We then machined an aluminum muzzle brake and mounted a standard rifle sight on top of it, looks way better. That plastic glob of a sight was uuugly. Good Luck