The protest was called after the city's main mosque was attacked with flares on Saturday by a handful of members from a far-right group, Hogar Social, who also unfurled a Spanish flag outside building, according to images on Spanish TV.
Muslim groups have reported a jump in anti-Islamic hate incidents since the twin vehicle attacks in Barcelona and the nearby seaside resort of Cambrils that killed 15 people last Thursday. Those attacks were claimed by the Islamic State group.
"It is very important for us to affirm our values as Muslims which are the same as those of all other human beings on this planet," said Nizar Liemlahi, the director the Dar Loughat Muslim cultural association, one of the co-organisers of the rally.
Only 0.1 percent of the world's Muslims commit attacks and these people "do not represent our values or our principles," added Liemlahi, a psychologist who moved to Spain from Morocco in 2010.
The protesters brandished banners with slogans such as "I am Muslim and I want to live in peace" and "We also are victims" as they gathered outside Granada city hall.
Mounir Benjell, head of the Spanish Federation of Islamic Religious Entities (FEERI), said the increase in anti-Islamic incidents since the attacks was "unbearable".
"Unfortunately, after what happened, there is an avalanche of cases. In my 25 years in Spain I have not seen anything like it,"
Vandals have defaced mosques with racist graffiti and some people have posted videos on social media sites calling for the expulsion of all Muslims, he said.
About 1,000 Muslims marched in Barcelona on Monday against terrorism.