Restrictions introduced two years ago were watered down after lobbying from the catering trade and tobacco firms.
With returns from nearly all 26 cantons counted, the full ban seemed to have been rejected convincingly.
Zurich newspaper Neue Zuercher Zeitung suggested voters had rejected the ban by nearly two-thirds.
In some cantons, more than 70% of voters rejected the ban, according to Geneva newspaper La Tribune de Geneve. Geneva itself bucked the trend by supporting the ban by 52% to 48%.
Geneva and seven other cantons have already imposed their own comprehensive bans on indoor smoking in places of employment while the remaining, smaller cantons have been less restrictive.
Jean-Charles Rielle, a doctor and member of the committee behind the proposal, told AFP news agency before the vote that they wanted to clear up the confusion.
"In the cantons where these laws [banning smoking rooms] are already in effect, we saw immediately... a 20% drop in hospitalisation due to cardiovascular incidents, heart attacks and these kinds of problems," he said.
However, Laurent Terlinchamp, president of Geneva's association of cafe owners, restaurateurs and hoteliers, said the proposed measures were extreme.
"In Geneva, where the law came in two years ago, we were told that a new clientele would start to come back to establishments," he said.
"But it's not the case today because profits are down 10% to 30% depending on the type of business."
La Tribune de Geneve suggests voters rejected a full ban because they did not want to force the smaller cantons into changing their local laws, and because of resentment at perceived state interference in people's lives.