I mean, what if you keep going down the road you are on now only to find a career in what ever is boring and drab and unfulfilling.
If something appeals to you, think about for a while, and, if it is worth it, GO FOR IT! You only get one shot to go through life and make it what ever you want to be.
I started out in Economics. It sucked. I thought it would be great. It was not. I switched to philosophy and never looked back.
Speaking from my own experience, psychology is fun. (It was my minor) Any sort of psy. Ab psy, para psy, it's all fun and good.
And, if this is a deciding factor... DO IT FOR THE WOMEN! Find a nice one that you can get chummy with and do lesson plans together with. You can do fun psychological experiments with each other. Study human chemisty. Psychological reactions to various stimuli. (Like strawberries and whipped cream