The town of Bar Harbor has tons of B&B's. If you're staying on Mount Dessert, a B&B is probably the best way to stay. Most have excellent reviews, and by staying directly in the main town, you can walk to your dinner or simply meander around the town without having to dodge pedestrians with your car. If you're looking for a quieter place to stay, there are several more B&Bs in in Bass Harbor, which would allow you to stay close to the park without actually leaving the island.
If you're looking for a small town away from the park, I can highly recommend the Blue Hill Inn, situated in the small village of Blue Hill, about a half hour away from Acadia. The owner is kind and gracious and the food and lodgings are excellent. The town of Blue Hill is one of my personal favorites and the scenery there is top notch (I'm allow to be biased since my family has lived in the town for decades.)