In a perfect world, yes, we'd all be using one browser and said browser would work with all the sites that we browse. The real world, however, is diametrically opposed to the perfect world. Divergent Web standards create compatibility problems with many site/browser combinations and it's utterly frustrating to say the least.
My default browser these days is the Webkit nightly build de soir (Safari's open source web browser engine) because it's blazing fast. My blogging browser is Flock because it supports the WYSIWYG editing toolbar in WordPress and it's kick-ass "blog this" feature. I keep Firefox 3.0 beta 4 handy for everything that doesn't work with Webkit or Flock.
The problem is that certain sites don't work with certain browsers forcing me to remember that one financial site doesn't work with Safari, but another bank site must use Firefox. One music download site I frequent (Beatport) only works with Flock – I think. It's gotten to the point that I keep a cheat sheet of what works with what. Ridiculous.
In addition to my "big three" I also keep a copy of the released versions of Safari and Firefox around for those sites that don't play nicely with my preferred browsers, but I have to admit that it's annoying to have to juggle three browsers on a daily basis. (And yes, I tried using only released browsers for a spell).
Can't we just all get along?