Now we've gotten into philosophy and it's hard to continue this debate because I agree with you more than not.
Any discussion about "rights" has to be philosophical. Where does a business owner's right to run his business end and Joe's right to have a job begin? Either way one of them loses. There is no easy answer- no laws of physics determine this- it's a judgement call society makes as to which is the better good- or conversely- the least harm.