We discuss the use of gravitational wave sources as probes of cosmology. The inspiral and merger of a binary system, such as a pair of black holes or neutron stars, is extraordinarily bright in gravitational waves. By observing such systems it is possible to directly measure a self-calibrated absolute distance to these sources out to very high redshift. When coupled with independent (electromagnetic) measures of the redshift, these "standard sirens" enable precision estimates of cosmological parameters. We review potential standard sirens for the LIGO (and LCGT!) and LISA gravitational wave observatories, including gamma-ray bursts and supermassive black-hole inspirals. Percent-level measurements of the Hubble constant and the dark energy equation-of-state may be feasible with these instruments. We also discuss a recent proposal for the "ultimate" cosmology mission, through the gravitational-wave observations of stellar-mass compact binaries from space.