Although both of your thumbs rest naturally on the controls up top, you’re not supposed to use them at the same time — you can either arrow around the UI with the left thumb or control the mouse with the finicky optical pad under your right. The two things are totally separate, so you’re using the arrows and you accidentally click the right button, you end up selecting whatever’s under the mouse cursor and going off in a totally different direction. It’s weird, especially since it’s supposed to feel like a PS3 controller — a controller that generally uses both thumb controls together very effectively.
Sony has never been any good at UIs, and neither has Google, but this is severely atrocious. I bet they didn’t do much hands-on user (i.e., not their own developers) testing, if any, and zero field testing. Or if they did, they ignored the results.
Update: the remote in question