Fly In The Ointment Meaning

(idiomatic) Something which ruins or spoils everything else; a nuisance or problem; an unpleasant or disagreeable detail.

Example: 2003, Ron Ross, Bummy Davis vs. Murder, Inc, page 287:
  Bummy Davis is no longer a pugilist. Willie is very aware of this fact and lets Johnny Attell know that there is a fly in the ointment, and Johnny, who is a very shrewd article, has his chauffeur drive him to Bradford Street so he can change the kid's mind.