South Sudan became independent last July, but since then relations have deteriorated with numerous clashes along their disputed border.
It has also halted oil production amid a dispute over sharing revenues.
The South Sudanese opted to split from the north last year, following a peace deal in 2005 which ended decades of civil war in which some 1.5 million people died.
But the two never agreed on the transit fees that the South Sudanese government in Juba should pay Khartoum for pumping oil through its pipelines and using Sudan's oil export infrastructure.
The countries have also failed to reach agreement on borders and accuse each other of backing militia groups.
Outstanding secession issues• Blue Nile and South Kordofan have not had popular consultations about their future, due before the split
• Abyei has not held a referendum on whether to join north or south, due before the split
• Sharing oil revenues
• Exact border demarcation
Earlier, UN Secretary General Ban Ki-moon had voiced his concern about the slow progress of the negotiations.
"The moment has come for the leaders of both countries to make the necessary compromises, once again, that will guarantee a peaceful and prosperous future for both nations," he said a statement.
Before halting production, South Sudan accused Sudan of stealing oil worth $815m (£518m).
Correspondents say the shutdown is hurting both economies - 98% of Juba's budget depends on oil while Khartoum needs transit fees to make up a 36% hole in its budget because of South Sudan's secession.
Friday's agreement aims to establish a monitoring mechanism to allow the two sides to lodge complaints if border disputes erupt.
The negotiations are expected to continue on Saturday, focusing on oil.