Okay I find it pretty shocking since both of you are married :S
I don't know if you should get a divorce or not, but I think that you shouldn't allow yourself to live a lie.
You might also want to read a little bit about Body Language and falling in love.
Since I can't post links, please check out the links in my signature.
There's a section about Body Language, with very detailed, illustrated gestures. A definite must-see.

Good luck