In Belize, about 70% of the country has safe, potable water. That includes nearly all areas where tourists are likely to go such as Ambergris Caye, Placencia, Cayo, Corozal Town, Belize City, etc. (The 30% where the water may not be potable is generally in remote villages without a dependable water supply.)
Some visitors buy bottled water, but in general pipe water and ice are safe to drink.
The only place I'd probably buy water is on Caye Caulker -- there's no municipal system, and some of the shallow wells may be contaminated. Also, a lot of the water smells bad (due to sulfur).
--Lan Sluder