Chances are that as long as you use normal, static files (ending in .js, .gif and so on), your server already does caching for you.

You can find out using a tool like Firebug in Firefox, or the built-in developer console in Chrome. The easiest way to get there is by right-clicking an element, and choosing "Inspect element".

If you switch to the "Net" tab, you will see all the requests that had to be made for the current file.

304 not modified and (from cache) means that the resource was not loaded again. If you see those for static resources, there is some caching already in place. The main document usually gets reloaded every time; the same goes for tracking codes like Google Analytics, and Ads. On a big site, you will expect to see a number of 200 OK requests as well.

In the tab, you can also find out the response headers of each resource. This tutorial could help deciphering what each header means: Caching Tutorial for web authors and webmasters

Fine-tuning caching is a very complex field. Here is a question that deals with caching images on Apache. Stack Overflow has a variety of existing questions on this.

If you want to generally look into speeding up your site, check out this question: Optimizations to reduce website loading time