The Farrer by-election was triggered by the resignation of former Liberal leader Sussan Ley, ousted by Angus Taylor during a leadership spill in February.
The Liberal Party had held the seat for 25 years but voters were not convinced by candidate Raissa Butkowski.
Within two hours of polls closing on Saturday, results from across the sprawling electorate showed the Liberals suffering a swing against them of more than 30 per cent.
The coalition combined had about 20 per cent of the vote shortly before 8pm AEST.
One Nation's David Farley had just over 42 per cent of votes with 63 of 94 polling places returned.
The win is a first for Pauline Hanson's conservative populist party in the lower house, with Mr Farley joining Barnaby Joyce who defected from the Nationals in December.
"We listened, you listened. You agreed and you spoke tonight," Mr Farley told the party faithful in Albury.
"In the Australian democracy, One Nation has reached the end of its beginning.
"We're going through the ceiling."
Ms Hanson called the result "not just a win for Farrer, it's a win for the rest of Australia".
She said she would release the party's policy on the nation's gas resources in the coming weeks but her comments on immigration drew the biggest applause.
"We have a plan for the future of this nation and we want people that want to be Australians," she said.
She also took the opportunity, as she has in the past, to declare: "I don't want Sharia law in this country."
Ms Hanson earlier said the party would tackle water buybacks if successful in Farrer.
"(Voters) realised that we are the last hope of changing things in this country, to get it back to the country that we used to be," she told Sky News.
"People have fear for the future generations, no hope of owning their own home and the cost of living is just destroying families."
More time to prepare for the contest might have made a difference, according to Mr Taylor.
"This by-election was always going to be a mountain to climb for the Liberal Party," he told supporters at the Liberals' function.
"We have to take away some hard lessons from this and what we have seen since the last (federal) election and from the last election.
For too long we have been a party of convenience, not of conviction, and that must change."
The most likely alternative to One Nation was independent Michelle Milthorpe, but the 25 per cent of the vote she secured was not enough.
Labor did not field a candidate.
Earlier, RedBridge pollster Tony Barry said that while Mr Farley and Ms Milthorpe appeared close in lead-up polls, preferences from coalition voters would likely give the One Nation candidate an edge.
"If they cannibalise the Liberal and National Party vote as expected, then it's a very safe conclusion to say that we're going to see similar voting behaviours and patterns in other regional seats," he told AAP.
Farrer has been held by either the Liberal or National party since its inception in 1949 and by Ms Ley from 2001 until she was ousted in February.
She congratulated Mr Farley on his win.
"Serving the people of Farrer for 25 years, having been endorsed by locals at nine elections, was the privilege of my professional life," she said in a statement.
"I know David will feel the same sense of honour and responsibility."
12°C