Why does metal feel colder than wood?
The short answer
Metal and wood can be at the same room temperature, but metal feels colder because it draws heat away from your hand faster. Your skin senses heat flow, not temperature alone.
Which way does heat move?
If your hand is warmer than the room, heat moves from your hand into the object. Metal conducts heat well, so it carries heat away from the contact point quickly. Wood conducts heat poorly, so heat stays closer to your skin and the object feels less cold.
What would a thermometer say?
After metal and wood have been in the same room long enough, a thermometer will show nearly the same temperature for both. The difference you feel is the rate of heat transfer, not necessarily a difference in temperature.