Diplomats running the executive bureau of the court's governing body said last month the British barrister had an inappropriate sexual relationship with a junior ICC lawyer and should be fired, documents seen by Reuters showed. On Friday, a large majority of the ICC's 125 member states voted to follow that recommendation.
"The assembly decided by a majority of 82 states parties that the elected official, prosecutor, Karim Khan, committed serious misconduct and a serious breach of duty ... and to remove ... Karim Khan from office," the president of the court's governing body, Paivi Kaukoranta, said.
Khan, 56, denies any wrongdoing.
"Mr Khan will challenge the lawfulness and fairness of the decision through all available legal mechanisms," his lawyer, Tayab Ali, said after the vote.
Khan's dismissal immediately triggers the election process for a new ICC prosecutor who is not expected to be voted in before next year.
"Members of the court acted swiftly and decisively to remove an embattled and discredited official so that the court can move forward with its important work in all the situations on its docket," said Leila Sadat, professor of international criminal law at Washington University.
Khan's dismissal follows an announcement by the United States that it will launch a renewed diplomatic campaign to dismantle the ICC, which it says is a threat to its sovereignty.
During a visit to the Philippines, US Secretary of State Marco Rubio said there were "lunatics and crazies involved in the ICC" and there was no way any US political or military officials would ever go on trial at the court, reiterating previous criticism.
State Department spokesperson Tommy Pigott welcomed Khan's removal but said, "he is only one small cog in this irredeemably corrupt institution. Today's outcome will have no impact on the United States' plans to dismantle the ICC."
The US, which is not a member, has imposed sanctions on 11 ICC judges and prosecutors, including Khan, citing the ICC's arrest warrants against Israeli Prime Minister Benjamin Netanyahu and former defence minister Yoav Gallant, and a past probe into US troops in Afghanistan.
Khan's accuser spoke publicly for the first time last week in an interview with CNN in which she repeated her allegations the prosecutor had a non-consensual sexual relationship with her.
Lawyers for Khan told Reuters he denied "any sexual relationship of any kind" with the junior lawyer.
Any active ICC warrants will remain in place because only ICC judges have the authority to withdraw warrants.
1800 RESPECT (1800 737 732)
National Sexual Abuse and Redress Support Service 1800 211 028
8°C