Abul Qaqa, who claims to speak for Boko Haram, said in an emailed statement that the sect carried out Saturday's attacks to protest the "continued arrests" of its members.
"We are responsible for the attacks in Gaidam and such attacks will continue as long as our members arrested by the Nigerian authorities remain in detention," he said.
Witnesses said sect members armed with Kalashnikov rifles hurled explosives at Geidam police station, freed suspects and stole arms.
The gunmen then rampaged through town attacking six churches, a high court, a shopping complex, a local government secretariat, and robbed a bank.
They also bombed an abandoned beer parlour and looted shops, according to a Nigerian Security and Civil Defence official in the town.
At least three policemen and a civilian were injured in the attacks, according to a security source.
Geidam is the hometown of the Yobe state governor Ibrahim Geidam, and is close to the border with Nigeria's northern neighbour, Niger.
Earlier this month the sect launched multiple suicide and bomb attacks on police stations and churches in Yobe's state capital Damaturu, killing at least 150 people.
Police have not given the total number of suspects arrested in connection with recent attacks in Yobe.
"Until all our members in detention are released and the Yobe state government stops intimidating and harassing our members we will continue carrying out attacks in the state," warned Qaqa.
Yobe is also the home state of slain sect leader Mohammed Yusuf, killed during a Boko Haram armed uprising in 2009.