DIAGNOSIS That off-taste is usually from the new plastic components (a 'new appliance' smell) or from odors in the air being absorbed by the water during the freezing process. Ice is a great odor absorber.
FIX For a new unit or after storage: 1) Do a thorough cleaning cycle with vinegar as described in the cleaning question. 2) After the rinse cycles, don't stop. 3) Now, make 3-4 full batches of ice and throw them all away. Don't skip this. This continues to flush the system and coats the internal water path with a layer of tasteless ice. 4) Use only filtered or distilled water for your consumable ice. Tap water contains chlorine and minerals that can impart flavors.
IF THAT DIDN'T WORK Consider the environment. Is the ice maker near the trash can, cleaning supplies, or strong-smelling foods? The fan that cools the compressor pulls in surrounding air. If that air is smelly, it can transfer to the ice. Move the unit to a location with cleaner air flow.
HEADS UP If you're using the ice basket's lid, make sure it's clean and dry. A damp lid sitting on top of the ice can develop mildew, which will definitely make the ice taste bad. Let the basket and lid air dry completely between uses.