Harry made a U-turn and requested to stay for just one night at Buckingham Palace, only to be told he had accepted King Charles' offer too late.
He was also told his presence could compromise his father's constitutional position, with judgment due in the duke's High Court case against Associated Newspapers on Tuesday.
Harry arrived back in the UK on Monday, his spokesman confirmed.
His spokesman earlier said it was "disappointing" the offer from the King had been "withdrawn at the last moment," with the legal fight given as the reason.
Questions have been raised on the Sussex side as to whether the offer of a bed at a royal residence was ever intended to be available once Harry had accepted it.
But is understood royal accommodation will be made available to Harry and his family for future visits.
The duke is believed to have arrived in London solo, without his wife Meghan or their children seven-year-old Prince Archie and five-year-old Princess Lilibet, after they pulled out of a visit to the capital because of security concerns.
As exasperation appeared to grow in both camps, it is understood Harry on Saturday initially turned down the offer for himself and his family to stay at the palace, before switching later in the day and asking to spend the night there himself.
The appropriate hospitality and staffing provision was said to be no longer available, with Buckingham Palace also believing the longstanding legal case had complicated the matter and could compromise the King's constitutional position.
It is said that while every effort was made to facilitate Harry's stay, as a courtesy to staff and others involved, the Royal Household requires a minimum level of notice to ensure he could be hosted appropriately at a royal residence.
Harry and a group of other prominent names are expected to find out on Tuesday whether they have won their High Court cases against the Daily Mail's publisher over allegations of voicemail interception, landline tapping and obtaining information by deception, which Associated Newspapers denies.
Harry is hoping to reunite Archie and Lilibet with their grandfather the King, who is still undergoing treatment for cancer, for the first time in four years, but it is not clear whether this will go ahead.
Meghan, Archie and Lilibet could join Harry outside of London, with reports he wants to take them to see the final resting place of his mother Diana, Princess of Wales at Althorp in Northamptonshire.
It is also not yet known when or whether Harry will see his father during his trip.
The King has a busy run of engagements this week while Harry is undertaking five days of appearances in London and Birmingham, including marking the one year countdown to the 2027 Invictus Games.
Private conversations are said to have taken place between Harry and the King.
Harry and his team are understood to be frustrated that the reasons given for the withdrawal of the offer shifted from not accepting the accommodation to the forthcoming Associated Newspapers legal judgment.
A royal source said Charles has a constitutional role in the legal system that would make it difficult for him to be linked too closely to individual lawsuits.
A spokesman for the duke said it was "disappointing that the offer (of accomodation) has now been withdrawn, with Tuesday's judgment in the Associated Newspapers Limited case cited as the reason.
"Buckingham Palace has, however, been aware of that judgment since last Thursday. It is therefore unclear why, having formally accepted the accommodation offer, it has now been withdrawn at the last moment."
The Sussex family were not eligible to receive taxpayer-funded protection while in the UK, other than when they were within royal residences.
With the family trip off and the failure to agree on where Harry - who has lived in California since 2020 with Meghan - will stay, it appears Harry's stated desire to reconcile with the royal family has made little progress.
with PA
6°C