Diagnosis
This is usually a Wi-Fi latency issue or microphone interference. The speaker hears the wake word but then packets are dropping as it tries to send your voice query to Google's servers.
How to Fix It
Improve your Wi-Fi signal. Move the speaker closer to your router and away from other electronics like microwaves or baby monitors. Power cycle your router and the speaker. Also, physically clean the microphone holes (small dots around the top) with a dry brush.
If That Didn't Work
There might be a conflicting wake word from another device. If you have multiple Google Assistants in earshot, they can interfere. Try temporarily muting the mic on other devices. Also, check the Google Home app for pending firmware updates for the speaker.
Heads Up
The Nest Audio uses multiple microphones for beamforming. If it's placed in a corner or against a wall, sound reflections can confuse it. Keep at least a few inches of space on all sides, especially the top.