Less than two years after he won a landslide election victory that promised to end chaos in British politics, Starmer said it was clear his party wanted him to go.
He said nominations for anyone to replace him would open on July 9.
However, his rival Andy Burnham is the clear frontrunner.
Starmer announced his decision on Monday, stepping out of Downing Street accompanied by his wife Victoria to cheers and applause from his team.
He said he had spent the weekend considering whether to step aside or fight a leadership contest.
"The question my party is asking now is whether I am best placed to lead us into the next general election," Starmer said on Monday.
"I have heard the answer of my parliamentary party to that question, and I accept that answer with good grace."
He said he had advised King Charles of his decision and would remain as prime minister until a successor was chosen.
Australian Prime Minister Anthony Albanese said he considered Starmer a friend and was "thinking of him on what must be a very tough day".
"Serving in public life is a tremendous privilege but politics can also be a harsh business," Mr Albanese said.
He said Starmer could be proud of the contribution he had made to Britain and the Labour Party.
The threat to Starmer, which had been building for months, increased sharply on Friday when Burnham, the Greater Manchester mayor, decisively won a parliamentary election to return to Westminster, beating a candidate from Nigel Farage's Reform UK party, which has led national opinion polls for more than a year.
That victory gave hope to Labour MPs that Burnham, a career politician known for his communication skills, could transform the fortunes of a party that has lost support under Starmer, whose popularity ratings have sunk to the lowest for any British leader.
Starmer thanked his colleagues for their support, his voice cracking with emotion as he also paid tribute to his wife and children.
The pound and British government bonds were steady in the immediate aftermath of Starmer's announcement, which investors had widely expected.
Despite the attempt at a smooth handover, the change is not without risk.
Beyond saying that the country needs fundamental change and to bring down the cost of living, Burnham has yet to make clear his approach to foreign affairs, the economy and defence.
Like Starmer, he could find he has little room to manoeuvre, hemmed in by bond market investors opposed to any additional borrowing, and confronted by an angry electorate who believe the country is not working properly.
Britain already has the highest borrowing costs in the Group of Seven wealthy nations due to its high debt and interest payments, years of anaemic economic growth, its struggles to cut spending and the need to invest in areas such as defence.
Investors spoken to by Reuters were divided over whether Burnham, who said last September that Britain had to get "beyond this thing of being in hock to the bond markets", would respect the need to reassure markets.
He has since said he was misrepresented.
"In our view, a Burnham premiership would inherit a precarious fiscal situation with few tools to deliver meaningful change," economists at Citibank said on Friday.
Whoever replaces Starmer will become Britain's seventh prime minister since the Brexit vote to leave the European Union, which took place 10 years ago this week.
That level of turnover - the highest in Britain in almost two centuries - underlines the struggle of maintaining the support of voters angry at successive failures to improve living standards, public services and tackle illegal immigration.
6°C