Diagnosis
You're right to want to clean it—debris inside the attachments can get heated up and create a burnt smell. The nozzles aren't designed to be taken apart, but you can clean them safely.
How to Fix It
Unplug the dryer and let it cool. Remove the nozzle from the dryer. Use the crevice tool attachment from your vacuum cleaner (a Dyson vacuum crevice tool works perfectly). Set the vacuum to a medium suction setting, place the crevice tool over the holes of the nozzle, and move it around. This will pull most lint and dust out from the inside.
If That Didn't Work
For stubborn particles you can see, use a dry, soft-bristled brush (like a clean makeup brush or a new paintbrush) to gently dislodge them through the holes, then vacuum again. Do NOT use anything metallic or sharp.
Heads Up
Never run water through the attachments or submerge them. The concentrated nozzle has a metal rim that can rust, and water trapped inside other attachments can drip into the dryer's heating element when you next use it, which is bad news.