What do you do when youve fallen in love with someone you shouldnt have?

Im engaged to a wonderful man, but I also love another, iv been with this one for 4 years, we have a 3 year old together and most of the time are happy. I'm not very emotionally stable, I get angry quick so we have a lot of little spats. When I'm with this other man (no I haven't cheated, nor will I ever) I feel so happy. Iv known him for 4 years. Idk how to handle it. Should I just stop talking to him? I can't leave my fiance, that would kill both of us, but I really don't know how to handle the feelings for the other guy. I love and care about them both a lot, I don't want to hurt either one, my fiance knows I love him and this other man, but he still continues to let me hang out and he comes with me sometimes. I don't understand why he trusts me so much, or why he continues to stay with me when I'm so... Psychotic sometimes. I have been diagnosed with bi-polar and depression, so I know for the most part why it happens.. I won't go on medication for it though. I don't like taking pills or the way they made me feel when I did take them. Any advice? Please don't be nasty.. I'm just looking for a little help. I'm just lost :-( I don't know what to do to help my own mind I guess