Diagnosis
Not normal. The 3rd Gen Cube has an active cooling fan, but it should only spin up audibly during intensive tasks like gaming or 4K HDR playback, not at idle.
How to Fix It
First, ensure it's not obstructed. The Cube needs several inches of clear space on all sides, especially the top and back where the vents are. Move it away from other hot components like an AV receiver or off a thick carpet.
If That Didn't Work
Check for software bugs causing high CPU usage. Go to Settings > My Fire TV > Restart. A full software restart can stop errant processes that are overheating the processor and forcing the fan to run constantly.
Heads Up
If the noise is a grinding or rattling sound, it's likely a defective fan bearing. In that case, contact Amazon support for a replacement, as the unit is not user-serviceable.