When I was doing really well, I still had some dairy (butter, yoghurt and cheese in food, thick cream in frozen berries from Coles /woolies/aldi and as an alternative to milk in coffee).
Following the Primal Blueprint by Mark Sisson helped me get to the leanest is ever been in my life.
A back injury led to weight gain and I haven't been as strict with diet and sleep as I should either.
All the best.