There are two primary paths between my house and downtown Bellevue. One is an arterial road, and the other is a highway. The highway is very slightly quicker.
Even though they're the same speed in both directions (i.e., the surface streets take the same amount of time whether I'm going to or from Bellevue) I always feel like it's quickest to take the surface streets...but only when leaving my house.
The reason? The on-ramp to the highway is in the wrong direction from my house, but the one near Bellevue is on the route. When I'm leaving the house it seems counterintuitive to drive off on a tangent, even though it yields the shortest route. In contrast, leaving Bellevue it's natural to jump on the highway since it's right there.
It's all so broken.
I'm in the same boat. When heading to Bellevue, I tend to take Bel-Red Rd., but take I-405 -> SR-520 when heading home (unless the weather is super nice and I'm driving with the top down).
I would argue that the highway is only slightly quicker during no traffic. During rush hour, the clog at the SR-520/I-405 interchange is so horrible that I find Bel-Red Rd. to be much better.
Good point about traffic—but even without that added variable, it's weird how the mind works.
--D