DIAGNOSIS: That's usually a combination of new machine "smell" and sanitizer residue from manufacturing, clinging to the internal food-grade tubing and plastic parts.
FIX: Run several clean cycles, but don't use just vinegar. For taste, you need to use a citric acid-based cleaner made for ice machines and coffee makers (like Dezcal or similar). Mix it the package directions, fill the reservoir, and run a clean cycle. Then, run 2-3 cycles with just fresh water to flush it all through.
IF THAT DIDN'T WORK: The charcoal filter in the water reservoir might be the source of the taste if it's sat dry for a long time or is defective. Remove the filter (it's in the lid of the reservoir) and run the machine without it temporarily to see if the taste changes.
HEADS UP: Avoid using baking soda or strong-smelling cleaners. The plastic and silicone inside can absorb those odors and make the problem worse. Stick with citric acid or approved ice machine cleaners.