See, i read that as his frustration that Mal seems to take priority over him when it comes to Zoe -- that even though Zoe pledged herself to him (Wash) with marriage vows, she acquiesces to Mal far more than she does to him and even does so when it is explicitly contrary to his (Wash's) desires.