The jury is still out on this question. As a general rule, though, there fewer rewards (and probably greater penalties) for most websites developed in Joomla if you use Joomla's built-in caching features. In particular, J! 1.5 system caching has been sorely criticised by many users from around the time of J! 1.5.16.

Server caching may yield improved performance on websites whose content is completely static - in other words, on those sites where nothing changes at all. However, even the seemingly minor activity of logging-in to a website causes an interaction with server caching.

If you have a website whose content is changing rapidly, from one moment to the next (such as if you are using a discussion forum), then server caching is counter-productive in these circumstances.

To summarise, the System - Cache plugin (disabled by default) should remain disabled except in the rare case when your site's content never (or rarely) changes; the plugin should never be enabled during the construction stages of new sites. Other settings to consider:

J! 1.5: Global Configuration » Server » Cache Settings » Cache = No
J! 1.6: Global Configuration » Server » Cache Settings » Cache = OFF - Caching disabled