Eclipse CVS plug-in (all platforms)
Eclipse IDE is a mature cross-platform development environment. It is open-source and free to use.
For new Drupal developers it should be clear that Eclipse can do everything you will need to do: checkout code, make patches, apply patches, etc. You do not need to learn command line CVS (although you many want to eventually anyway).
Eclipse can be used to develop in many languages using different development environments, so at first glance it can be intimidating. The fact is that most Drupal developers will not use 95% of the features of Eclipse, so it is simply a matter of learning which features you need for each task.
There are some good, cross-platform instructions for installing Eclipse and PHPeclipse. You should follow sections 1.1 - 1.3 at least.
Eclipse + Drupal Coding Standards
Eclipse IDE Setup for debugging
Eclipse on Ubuntu
If you use (or are thinking of switching to) Ubuntu then you may want to check out this HOWTO on setting up a Drupal develepment environment using Eclipse and Ubuntu: http://dev.civicactions.net/moin/FreeDevelopmentEnvironment
Updated Instructions
If you have Ganymede, you should follow these instructions; http://www.howtoforge.com/setting-up-eclipse-pdt2.0-on-eclipse3.4-ganymede . Target Management/RSE & Subclipse in the Remote Files section are not strictly necessary.
General instructions for using Eclipse with Drupal are here:
General instructions for configuring eclipse for Drupal can be found here:
http://drupal.org/node/75242