People metabolize foods differently. Most people I know (myself included) do better on a diet that emphasizes meat and quality fats.
You have to pay attention to what works for you.
Another thought is that you might be sensitive to what is in the meats. Clean grass raised meat and dairy might do better for you. Could it be you are just back to eating more processed foods and fast foods? At the very least read the ingredients on what you consume.