DIAGNOSIS You're probably dealing with a worn-out or damaged blade clamp. The little nubs on the oscillating plate that grip the blade's mounting hole can get rounded off or packed with material, so they can't get a proper bite anymore.
FIX First, clean everything. Remove the blade and the bolt. Use a wire brush or a pick to thoroughly clean the oscillating plate, the bolt, and the hole in the blade itself. Get all the gunk and rust out. Reassemble and tighten the bolt firmly with the provided Allen key.
IF THAT DIDN'T WORK Inspect the oscillating plate closely. If those gripping nubs are visibly worn down smooth, the plate needs to be replaced. It's a part you can order (often called the "shoe" or "adapter") and swap yourself with basic tools.
HEADS UP Always use the OEM DeWalt bolt or an exact duplicate. Generic bolts from a kit are often a fraction of a millimeter too small in the shoulder diameter, which creates slop and accelerates wear on the clamp.