Yeah, ideally, we want to use the Smart Move *every* time, because it's quicker and more efficient.
As for the pool, that's mostly true. The Pool driver mostly passes the file system commands to the underlying drives. There are some times that this doesn't happen, but that should be rare, depending on the circumstance. There is some special handling, depending on a number of factors, such as the balancer settings.
And yeah, the driver appears to be simple. But let me tell you... it's some really complicated/sophisticated code.
In fact, we have a joke internally. "that should be simple... six months later...." Because the stuff that seems simple tends to be very complex, just to appear that way.
But like I said, if you think that this shouldn't be happening (such as you have plenty of room on all of the drives, it appears to be moving data around on the same disk, etc), don't just take my word. Grab logs and we'll double check to make sure it's not an actual bug.
Also, if you're moving a LOT of small files (smart move or otherwise), this can take a while, since each file has to be updated and verified by the file system. Moving a couple large files is always faster than a lot of small files.