Why is Almond Milk Bad for You? One food that is thought to be goitrogenic is almond milk. Consuming high amounts can cause thyroid damage