Honestly baddy's are too irrelevant. Just go to bandits as they have potential map drops, they may drop chest where loot can be found, somewhere in the cave it has these chest that close everyday so you can collect loot in them.