keep one's nose clean

(idiomatic) To stay out of trouble, especially by avoiding unlawful behavior.

Example: 1947, American Correctional Association, American Journal of Correction, page 6:
  They tell us, "Keep your nose clean and you'll no doubt make a parole." So we keep our noses very clean.
2008 Nov. 5, Nathan Thornburgh, "Ted Stevens Sins, and (Likely) Wins," Time (retrieved 25 August 2013):
  Alaska Congressman Don Young . . . spent a huge share of his campaign donations on legal fees to keep his nose clean in the face of an FBI investigation into his dealings.