LaTeX is a document typesetting system, which uses a markup language to describe the content using plain text files. It is excellent for writing long complicated papers/books. For short, casual documents, Office/Open Office would arguably be easier to use, although it depends on your familiarity with LaTeX.
For more control (although with less features), take a look at ConTeXt.
For a more WYSIWYG feel (although not completely), take a look at LyX.
The windows LaTeX compiler. This is the best option for running LaTeX on windows. Get the basic installer and it can automatically install packages on demand. It comes with a nice package manager as well.
The LaTeX compiler on most Linux systems. Although they have a windows port, I recommend MikTex for windows and TeX Live for Linux systems.
Not technically a compiler, but good for creating a few quick images manually.
Not technically a compiler, but good for automatically creating images by sending the equation in the URL. Type in your equation after the 'chl=' part of the URL, and you will get back a rendered PNG of your equation.
A big part of writing scientific papers is generating plots and diagrams. While you can use your analysis tools (whether it is MATLAB or R or SAS or ...) to generate diagrams to be included in your paper via the graphicx package, it might be easier or more desirable to dump the data and have LaTeX plot it for you (LaTeX usually require a space delimited file). Also remember if you're generating graphics to be included, use a vector graphics format (PDF, EPS, EMF, etc).
An excellent high level data plotting tool with many examples, my preferred plotting package.
While PGFPlots just handles simple plotting, the datatool package allows you to manipulate the data as if in a database before plotting, it has its uses for some quick calculations, but if you're going to get more involved with the computation, you might consider using doing it in another package.
This is a full blown vector description language that is geared towards LaTeX users. Worth looking at if you want to do some advanced vector drawings, but otherwise perhaps a little too much effort to learn it for regular plotting.