There are many reasons Ristis adore their watches. For me a Panerai has a unique style that is a magical combination of traditional and modern/bold at the same time. And, one of the things that makes them bold is that they are oversized. I have a 7" wrist and the smallest watch in my collection is 44mm. My daily watch is the 372 at 47mm. My wife with her 6" wrist wears a 45mm radiomir. Is it big and bold? Absolutely! And, in my opinion, that is how it should be.
Hate to disappoint, but while I am sure the that 40mm fits your wrist perfectly, I believe that bigger is the way to go.