Eat for Health Infant feeding guidelines. Place the frozen container in the refrigerator the night before Storing breast pumped milk intend to use it. When thawed, this milk is often described as smelling unpleasant, rancid or soapy. September 18, Content source: Spoiled breast milk will have that unmistakable sour taste. Consider these do's and don'ts for breast milk storage.