>>I just don't understand why he doesn't love our son like I do<<
He does. Your husband does love your son very much. He just doesn't know how to show him.
>>I've caught my son whispering to himself that he is a loser and stupid<<
And even though your son will love his son (your grandson) very much, he'll have to work very hard not to communicate to his son (your grandson) this he is a loser and stupid and that he'll never live up to his father's expectations.
We perfectionists are all the same - we're contagious.
>>My thinking is that I wasn't meeting these needs before and that's what made him vulnerable to an affair so I better wake up and smell the coffee now<<
Some day, I hope to be present (maybe even preside) at the graveside of that particular view of affairs.
>>He once told my brother-in-law that he didn't care if I had a boyfriend because he's not a jealous guy<<
And I can shoot beer bottles off of fence posts at a 100 yards with a pistol. I eat nails and schiit barbed wire. Your h has no idea. I know because I didn't either. I said more or less the same thing to my (then) fiance 25 years ago - that if she ever had an affair, I would forgive her. Talk is cheap in this business.
>>That hurt. What that tells me is that I am nothing special to him and that he doesn't feel lucky to have me, like I feel about him<<
I can understand why it would hurt but I very much doubt what your h said is true. He just doesn't know. Ignorance may be bliss but it can also be dangerous.