There is a smiley-face button at the bottom left of the keyboard that allows you to type emojis. I never use emojis. How can I remove this button from my keyboard?
This is not a user configurable setting, and currently there does not exist any way to swap the supplied keyboard for an alternate (which from a security point of view, is a good thing)
This is (somewhat) possible to do on the current Windows 10 preview, as long as you have more than one language installed.
Go to your keyboard settings and select "More keyboard settings." Make sure the box next to "Show the language switching key" is checked.
This will replace the emoji key with the language switching key. Emojis can still be accessed by long-pressing the numbers and symbols key.
The answer is sadly no, it is also not possible in the forthcoming windows 10. You can disable emoji recommendations but not remove the button itself at this time.