In Marvel comics, the name Guardians of the Galaxy goes all the way back to 1969. The series has evolved a lot over time, but the first team with Star-Lord, Rocket Raccoon, Gamora, Drax the Destroyer and Groot was created in 2008. This series has a much more of a cosmic-war guerrilla-unit feel to it. You will also see a very different Groot persona, though not nearly as different as his original appearance.
Both books of the 2008 series, in order:
The 2013 series is much closer to the 2014 movie everybody knows and loved, which was in production at Marvel at the time.
Every book of the 2013 series, and onward, in order: