At a meeting of his cabinet on Tuesday, Starmer repeated that, while he took responsibility for one of his Labour Party's worst election defeats, there had been no official move to trigger a leadership contest.
Four ministers expressed their support for him.
It was the latest pledge from Starmer to press on with a premiership that has been dogged by scandal and policy U-turns since he won a large majority at a national election in 2024.
On Monday, he promised to be bolder in tackling the problems besetting Britain to try to shore up his political future.
In a nod to an increase in borrowing costs on the markets over fears of another bout of political instability in Britain, Starmer said the "past 48 hours have been destabilising for government and that has a real economic cost for our country and for families".
"The Labour Party has a process for challenging a leader and that has not been triggered," Starmer told his cabinet, according to his Downing Street office.
"The country expects us to get on with governing. That is what I am doing and what we must do as a cabinet."
Leaving the meeting, four senior ministers offered Starmer their support, with pensions minister Pat McFadden telling reporters that no one had challenged the prime minister at cabinet.
Starmer's defiance was in marked contrast with the feelings of many in the wider Labour Party.
On Tuesday, a junior minister resigned after a handful of ministerial aides also quit the government.
More than 80 Labour MPs have publicly called for him to set a resignation date so the party can install a new leader in an orderly manner.
It was a long way from when Starmer first became Labour leader in 2020, inheriting the party after its worst national election showing since 1935 under his predecessor, veteran left-winger Jeremy Corbyn.
He was then seen as a safe pair of hands able to drag Labour more towards the centre ground.
At the 2024 election, he won one of the largest majorities in modern British history for Labour with an offer of stability after years of chaos under the Conservatives, who oversaw five prime ministers in eight years and left, what his government called, a "black hole" of unfunded spending commitments in the public finances.
Now, he is fighting for his political survival.
On Monday, Starmer had again vowed to stay the course, saying succumbing to calls for him to go would bring in the type of chaos that has dogged Britain since the nation narrowly voted in favour of Brexit in 2016.
It is generally harder for Labour MPs to remove a prime minister than the opposition Conservative Party.
While dozens of Labour MPs might have expressed their dissatisfaction with Starmer, 81 of them need to rally behind one single candidate to trigger a contest.
Removing Starmer now - or forcing him to set a departure date - would likely favour health minister Wes Streeting, who is in a position to move first.
Other possible challengers, Greater Manchester mayor Andy Burnham and former deputy prime minister Angela Rayner face obstacles to running, both seen as favourites of the moderate left of the party.
Burnham does not have the seat in parliament and Rayner has yet to fully resolve the tax issues that prompted her resignation from office in 2025.
