DIAGNOSIS Yes, that's not good to drink. Those are granules of activated carbon from inside the filter. It means the filter's internal mesh sleeve has ruptured or the end caps have failed, letting the media escape.
FIX You need to stop using that filter immediately. Discard any water in the pitcher. Remove the faulty filter and set it aside. Give the entire pitcher—the reservoir, the main body, and the lid—a thorough, complete wash with warm, soapy water. Rinse everything multiple times to flush out any lingering carbon dust. Install a brand new, certified Brita filter. Remember to prime it by soaking in cold water for 15 minutes before the first use to settle everything inside.
IF THAT DIDN'T WORK If black specks appear again with a *new* filter, the issue might be damage to the filter holder in the lid. Inspect the plastic spikes or sockets where the filter locks in. If they are cracked or broken, they could be damaging new filters as you install them. In that case, you'd need a replacement lid.
HEADS UP While the carbon itself isn't toxic, it means the filter is compromised and isn't working as designed. Don't try to rinse it out and keep using it. Contact Brita customer service about the faulty filter; they often have good warranties and might send you a replacement.