I'm thinking the OP didn't so much want solutions to a problem, but rather, wanted confirmation that their own feelings on the matter are sound.
That being said, I'll give my thoughts on the matter anyhow.
A successful partnership between two loving people is built on respect and trust. You can love someone but not respect them, and/or not trust them. Not having respect or trust does not make the love any less real or important. But the partnership will never be successful without those.
Our finances is one critical area in which respect and trust are paramount. If you know you can't trust him/her with the finances, but you love them enough to stay with them anyhow, then you'll need to work hard at finding SOME way to keep all the bills paid as well as keeping harmony in the house.
If there's any way to do it, maybe set up separate accounts....one main account for all the household bills that is controlled only by the most responsible partner...and then separate spending accounts for each person to spend as they wish. If they want something expensive, they'll have to save "their own money" for it, but won't need to ask permission to spend it.
There ARE ways to get around this kind of problem and keep the relationship together, but it will take a lot of effort, open communication, and a willingness of both partners to work on it. In the end though....a partnership without full trust in all areas is destined to be an uncomfortable road to travel, at the very least.