Diagnosis
This is common with new portable ice makers. There's manufacturing lubricant and plastic off-gassing in the internal lines that needs to be flushed out thoroughly.
How to Fix It
You need to do a extended flushing process. 1) Fill the reservoir with a mix of 1 cup lemon juice to 1 gallon of water. 2) Run the machine, discarding the first 2-3 batches of ice it makes. 3) Dump the lemon water, rinse the reservoir, and fill with fresh tap water. 4) Run and discard another 3-4 batches of ice. That should clear the taste. Always discard the first few batches from a new machine or one that's been in storage.
If That Didn't Work
Check if you're using filtered water from a pitcher that hasn't been cleaned, or if there's an old filter in your fridge line. Sometimes the taste is coming from your water source, not the ice maker.
Heads Up
Avoid using baking soda for cleaning the reservoir if you can smell plastic. It can be hard to rinse out completely and will leave its own taste. Lemon juice or vinegar rinses away cleaner.