You should fear the base hit more than a walk. At the most, a walk only advances runners one base, while a basehit probably moves them 2 or more.
At the right time, a walk can actually be a good thing, while allowing a base hit is never good because that basehit may go over the fence.