UN Secretary General Ban Ki-moon said Mr Annan had decided not to renew his mandate when it expires at the end of August.
Mr Annan authored a six-point peace plan for Syria which was intended to bring an end to the fighting.
But the plan was never fully adhered to by either side and the violence has continued.
Mr Ban said Mr Annan deserved "our profound admiration for the selfless way in which he has put his formidable skills and prestige to this most difficult and potentially thankless of assignments".
He said he was in discussion with the Arab League to find a successor to "carry on this crucial peacemaking effort".
"I remain convinced that yet more bloodshed is not the answer; each day of it will only make the solution more difficult while bringing deeper suffering to the country and greater peril to the region," he added.
Mr Annan's internationally backed peace plan called for an end to the use of heavy weaponary, the free passage of aid, freedom of media and demonstration and for a Syrian-led political process to address the aspirations and concerns of the Syrian people.