Diagnosis
The fabric mesh is durable but shouldn't be soaked or aggressively scrubbed. You're right to be cautious.
How to Fix It
The best tool for the job is a soft-bristled detailing brush, like the kind used for cleaning car interiors or keyboards. Gently brush the fabric in one direction to lift and dislodge the dust. For smudges or spots, slightly dampen a soft microfiber cloth with plain water (wring it out so it's just damp, not wet) and gently dab the spot. Do not rub in circles. Let it air dry completely.
If That Didn't Work
For more stubborn marks, you can use a tiny drop of mild dish soap diluted in a bowl of water. Dampen the microfiber cloth in the solution, wring it out extremely well, and dab the area. Immediately follow with a second cloth dampened with clean water to rinse, then dry with a third dry cloth.
Heads Up
Never use a vacuum cleaner attachment directly on the fabric, as the suction can stretch or damage the mesh. Never spray any cleaner, solvent, or water directly onto the bar. Always apply it to the cloth first.