Prime Minister Naoto Kan announced his resignation on Friday.
He has been criticised for his handling of the quake aftermath.
Mr Noda, 54, is expected to be confirmed by parliament as prime minister on Tuesday.
He secured 215 votes in the ballot against Mr Kaieda's 177 votes.
Supporters of public favourite Seiji Maehara - the former foreign minister who was eliminated in first-round voting - threw their support behind Mr Noda in the second ballot.
Mr Kaieda was backed by scandal-hit party heavyweight Ichiro Ozawa, who controls the DPJ's biggest faction.
Divided party Correspondents say the new prime minister will face a daunting agenda, including trying to unify a divided party.
- Served under Naoto Kan as finance minister
- Wants to double sales tax to 10% to meet social security spending commitments
- Has not backed Naoto Kan's call for a nuclear-free Japan
- Has stressed importance of Japan-US security alliance
He has said in the past that he favours raising funds through increased taxation - including a doubling of Japan's sales tax, which currently stands at 5% - to cut debt and meet social security commitments.
Unlike Mr Kan, he wants Japan's halted nuclear reactors to be restarted and has not backed his call for a nuclear-free Japan.
"Let us sweat together for the sake of the people," he said after the vote. "This is my heartfelt wish."
- Naoto Kan: (June 10-Aug 11) Criticised for handling of quake crisis
- Yukio Hatoyama (Sep 09-Jun 10) Failed to keep election promise on Okinawa US base
- Taro Aso (Sep 08-Sep 09) Famed for verbal gaffes, lost election
- Yasuo Fukuda (Sep 07-Sep 08) Struggled to pass laws, after DPJ took control of upper house
- Shinzo Abe (Sep 06-Sep 07) Unpopular government hit by scandal
It has also been hit by in-fighting, with the leadership race turning into a bitter factional battle between supporters and enemies of tainted king-maker Ichiro Ozawa.
Mr Ozawa - who is known as the Shadow Shogun - commands the loyalty of around 130 lawmakers, despite awaiting trial on charges of misreporting political donations. He is currently suspended from the DPJ.
The public's favourite candidate, Mr Maehara, would have become Japan's youngest leader since World War II if elected.
Farm Minister Michihiko Kano and former Transport Minister Sumio Mabuchi were also in the leadership contest.