Veterans Affairs Minister Michael McCormack is frontrunner to replace Mr Joyce when a leadership ballot is held in Canberra on Monday morning.
A scandal that erupted last week in Australia over an affair between Deputy Prime Minister Barnaby Joyce and one of his female staffers has devolved into an open conflict between Joyce and Prime Minister Malcolm Turnbull that threatens to take down the coalition government.
Mr Joyce, who lives in north-west New South Wales and wears a cowboy hat, is perhaps best-known internationally for his threat in 2015 to kill Pistol and Boo, the pet dogs of actors Johnny Depp and Amber Heard. "I never tried to be anything else". The Australian news media was reluctant to report on the relationship with the woman, Vikki Campion, until it was discovered that the couple were living rent-free in an apartment donated by a supporter.
Mr Joyce has declined to publicly back any successor.
"They should have been referred to police if they had substance", he told the Nine Network on Friday.
The decision to quit comes less than 24 hours after backbench Nationals MP Andrew Broad called on him to quit, and two days after Mr Joyce and Ms Campion granted an exclusive interview to Fairfax Media in which the pair appealed to politicians and members of the public to "move on" from the scandal.
Mr Turnbull said he would be reluctant to wade into the United States gun debate.
Joyce revealed in December he had split from his wife in a statement to the Australian Parliament.
Joyce's resignation will potentially heal the rift between the Liberal and National parties, a political alliance that has existed for almost 100 years.
Barnaby Joyce, the leader of the rural-based Nationals Party, told reporters that a sexual harassment complaint against him that emerged in the media the previous day had convinced him that his position was politically untenable.
Mr Turnbull accused his deputy of making a "shocking error of judgment" by having an office affair.
Australian Twitter users have been making fun of Barnaby Joyce, particularly his role in the "War On Terrier", after he announced he will resign as the country's deputy prime minister.
Joyce also said that he will carry on as a member of parliament, moving to the back bench.
Cabinet minister Matt Canavan described Joyce as a mate who had helped save the coalition Government from a first-term defeat.
"I understand that you are frustrated that we don't intend to answer any further questions but that is all I have to say on the matter". Meanwhile John McVeigh, a Queensland Liberal who is minister for regional development, will act in Joyce's infrastructure portfolio.