Bismillah.
Narrated Abu Huraira:
Allah's Apostle (p.b.u.h) said, "A man said that he would give something in charity. He went out with his object of charity and unknowingly gave it to a thief. Next morning the people said that he had given his object of charity to a thief. (On hearing that) he said, "O Allah! All the praises are for you. I will give alms again." And so he again went out with his alms and (unknowingly) gave it to an adulteress. Next morning the people said that he had given his alms to an adulteress last night. The man said, "O Allah! All the praises are for you. (I gave my alms) to an adulteress. I will give alms again." So he went out with his alms again and (unknowingly) gave it to a rich person. (The people) next morning said that he had given his alms to a wealthy person. He said, "O Allah! All the praises are for you. (I had given alms) to a thief, to an adulteress and to a wealthy man." Then someone came and said to him, "The alms which you gave to the thief, might make him abstain from stealing, and that given to the adulteress might make her abstain from illegal sexual intercourse (adultery), and that given to the wealthy man might make him take a lesson from it and spend his wealth which Allah has given him, in Allah's cause."
It's good to keep a check on the person/place you're spending on. But, let's say, your charity went to someone not really 'deserving' (or so it might seem), then you don't need to think it won't get accepted. Your job was to give. And it's for Allah to accept. So leave it on him. :)
Taken from: http://ilookilisten.wordpress.com/2012/11/10/your-job-is-to-give-hadith-no-1161/