I was running some numbers about scalability recently, and they provide some perspective. Lets talk about a fairly busy website. 1 million hits a day. If evenly distributed, that's 12 hits per second. If you estimate the peek to be about 4 times the average, then you can pretty much plan for about 50 hits a second.

Depending on what your service is, the basic infrastructure for this number is entirely doable with just a few machines. probably a dedicated database server with some query caching, one or two web and/or business logic servers. Its easy to forget just how fast computers, disks, and networking really are these days.

All too often large numbers seem larger than they really are.