The main reason that simplicity is the key when it comes to web design is that it will make it much easier for your visitor to use your site. Remember they are there to read the content of the site not to look at a bunch of fancy graphics or features that you think are neat. In most cases this will just distract them from what they are actually trying to do. A well designed website should be easy to read and easy to navigate, this requires that it have a simple design.
The other big reason that you want to keep your website design simple is that it will allow it to load faster and work better. If you have a site that takes a long time to load people will just move on to the next one, they are not going to wait. The same thing is true if they get on to your site and find that it doesn't work properly. The more complicated you make your site the more likely it is that there will be bugs and the longer it is going to take to load. This will cause you to lose a lot of visitors.
While the main reason to keep your website design simple is for the benefit of the visitor it also benefits the sites owner. If you keep the design simple it will be much easier to build the site. This means that it will get done faster and cheaper. There is little value in adding a whole bunch of features that are just going to make your site harder to build when adding them is really just going to be wasting your time and money. A simple website will also be easier to maintain since there will be less to have to deal with.
Although it is well established that a simple website design is the way to go actually doing it can be a problem. The main challenge is that you want to create a site that is different from all of the other sites that are on the internet. This tends to lead to a temptation to start adding features. There is also the issue of having a hard time determining what is really necessary on a site and benefits the visitor and what will just irritate him. This is where an experienced web designer can really help you. You have to be careful here however, a lot of designers don't have the necessary experience for this. A lot of failed graphic designers decide that website design is an easy way to make money and set up a company to offer the service. These are the guys who build overly complicated sites since they want to have a chance to show off their design skills. In the end they just end up proving that they really don't know how to design a site.