Nearly 15 young people with face masks, machetes and axes stormed the mosque at 8pm local time (1700 GMT) on Thursday evening, said Mwanza Police Commander Ahmed Msangi during the press conference.
Msangi stated that the attackers targeted worshipers in the mosque.
A motive or the reason behind the attacks and killings is not yet know.
Police officers have arrested three people in connection with the attack.
Imam Ferouz Imail Elias, who leads prayers in the mosque was also among the victims, along with two others.
Several people were also wounded in the attack.
Although attacks on mosques and churches have risen in recent years, there are no reports of inter-religious violence in the East African country.
Approximately 35 percent of Tanzania’s 51 million population is estimated to be Muslim, although there are no official figures.