Diagnosis
Probably not a bad filter. That off-taste is almost always because the new filter cartridge wasn't flushed properly. The carbon media needs a good rinse to remove fine carbon particles and any preservative coating from manufacturing.
How to Fix It
Run cold water through the filter on the "filtered" setting for a full 5 minutes. Do this into a bucket or pitcher, not a glass you're about to drink from. This is PUR's official priming instruction. After that, discard that water. The next batch should taste clean and normal.
If That Didn't Work
If the plastic taste persists, check that the filter housing and its internal seals are clean. Sometimes old debris or biofilm can get trapped. Remove the filter, rinse the housing with warm water, and reinstall a fresh filter, flushing it again.
Heads Up
If you let the filter sit unused for a week or more, bacteria can grow on the damp carbon, causing a sour or musty taste. If that happens, you need to flush it for a few minutes or just replace the cartridge.