Is my headset bad, frame bad, something else…?

Pull the lower headset bearing to re grease. When I put it back in the frame I noticed there is some side to side play between the bearing and frame cup. Put the fork back on and tightened it back up and there is some pretty noticeable play. What to do?