Diagnosis
You're not doing it wrong. The 5-way joystick/navigation button on this model is notoriously stiff and has vague tactile feedback, especially when new. It's a common design complaint.
How to Fix It
The trick is in the technique, not brute force. 1) Use your fingertip, not your nail. 2) Press directly into the center of the joystick to click (select). It requires a firm, deliberate press. 3) To navigate the menu, tilt the joystick in the direction you want to go (Up, Down, Left, Right). Don't press it in while tilting. The tilt action is also stiff, but it will loosen up slightly with use. Think of it like a very small, stiff game controller joystick.
If That Didn't Work
If it feels completely stuck or unresponsive, it's possible it was damaged in shipping or has a defect. Before assuming the worst, try powering the monitor off and on again. Very rarely, the OSD controller can get stuck in a software sense. If the physical stick itself is truly immobile, that's a hardware issue.
Heads Up
Because it's stiff and on the back, you'll often bump the monitor when trying to use it. Just be gentle. The good news is you won't use it often once you have your settings dialed in. If you need to adjust settings frequently, consider using the Acer Display Widget software on your PC, which lets you control most OSD settings with your mouse.