Note those characters along the number keys.
That’s Keyboard Viewer, which is built into OS X. You can enable it as a hovering overlay window in System Preferences > Keyboard somewhere. It will show you what your Mac is seeing from the keyboard.
As for whether the keyboard or the converter is the source of the problem: I’d guess the converter. One way to test is to switch around your modifier keys in that bit of System Preferences. See if swapping the mods around also swaps the stuck key. And keep an eye on when it shows up.