the thing of it

(idiomatic, colloquial) The important point to consider.

Example: 1855, Azel Stevens Roe, A long look ahead; or, The first stroke and the last
  But the thing of it is, when to begin
1997, Youth and Violence.: Developing Local & State Solutions
  But the thing of it is , everybody was rooting for the other individual and I was the only one rooting for him.