I wanted to weigh in on this when we got back from our little vacation, because like Lori, I was curious to see if there was a distinction between the two words. I found this specific usage note at Answers.com:
USAGE NOTE Aggravate comes from the Latin verb aggravāre, which meant "to make heavier," that is, "to add to the weight of." It also had the extended senses "to annoy" and "to oppress." Some people claim that aggravate can only mean "to make worse," and not "to irritate," on the basis of the word's etymology. But in doing so, they ignore not only an English sense in use since the 17th century, but also one of the original Latin ones. Sixty-eight percent of the Usage Panel approves of its use in It's the endless wait for luggage that aggravates me the most about air travel.
It is also listed as a synonym in the thesaurus section.
Definition source: "aggravate." The American Heritage® Dictionary of the English Language, Fourth Edition. Houghton Mifflin Company, 2004. Answers.com 03 Dec. 2008. http://www.answers.com/topic/aggravate
Thesaurus source: "aggravate." Roget's II: The New Thesaurus, Third Edition. Houghton Mifflin Company, 1995. Answers.com 03 Dec. 2008. http://www.answers.com/topic/aggravate