I can't really recall, but I think you're suppose to give it to someone in the Church near the Dairy Hut I believe? It's been a long time since I have done that quest...
Edit : Yup, I was right. You're suppose to give it to a guy named Owen in the Church. There's a guide for it here : http://www.graaldepot.com/candy-land-key-quest-guide