Out of the box performance settings
Throttle: congestion control does something which I've never understood. I turn it on, though, and every so often I see messages about it throttling something. It is supposed to detect large numbers of concurrent users and scale back features under high load.
In Administer / Performance is the basic performance settings. Drupal has its own caching system. The cache is a software technique to store commonly requested items so they don't have to be regenerated over and over and over and over. Since pages on a Drupal are (out of the box) regenerated completely on each page view, the cache saves a lot of work by not having to regenerate every molecule of every page.
Server tuning considerations is the official overview of tuning Drupal performance.
PHP caches improves PHP performance considerably. The problem here is that PHP interprets its code by reparsing the text of every PHP file on every page request. PHP caches save the binary form of the parsed PHP files, which lets PHP avoid reparsing every file on every page request. I installed eAccelerator and have had no problems.
Tuning MySQL for Drupal has some good recommendations. However one recommendation it makes, to use DBMaintanence and to use the 'OPTIMIZE TABLE' command, is very dangerous. I've seen this command screw up the tables on which it's used.
I don't have many specific recommendations for tuning MySQL, instead it's been very confusing. Looking at the MySQL slow query logs is interesting, but it's not clear what to do with this information. What I see often is slow queries in watchdog and cache. The watchdog table stores the system log requests, so if you act to limit the log entries (such as making sure you have a favicon file to aviod the notices about the missing favicon file) will improve things by making it less likely watchdog table queries will be slow.