Some use these interchangeably, but there really is a difference that should be observed.
Imply means to to hint or suggest. Infer means to draw a conclusion.
I try to remember the difference between the two by thinking that "imply" is something that is done to you. Someone implies that you are a nincompoop. "Infer" is something that we do...based on hearing someone's remarks, we infer that they are a nincompoop. Or sometimes I think of it in dramatic, cinematic terms. I say to the bad guy, "Just what are you implying with your remarks?" He says to me, "I'm not sure. What are you inferring?"