The Friday evening tragedy unfolded in a remote, mountainous area of southern Kerala as pilgrims made their way home from an annual ceremony at the hill shrine of Sabarimala that draws three to four million people each year.
Kerala Home Secretary Jaya Kumar told AFP that 104 people had been confirmed dead and dozens more injured, some of them seriously.
Police officials said a packed jeep had lost control and ploughed into a crowd of devotees packed onto a narrow road in a hilly and densely forested area 10 kilometres (six miles) from the shrine.
"The accident caused a mass panic and triggered a stampede on the hillside," said Special Police Commissioner Rajendra Nair.
The search for bodies and survivors had been hampered by the remote location, heavy mist and the thick forest terrain.
Indian television showed pictures of casualties being passed over the heads of tightly packed crowds of pilgrims in a rescue effort that stretched deep into the night.
Defence Minister A.K. Antony called the incident "a tragedy beyond anyone's imagination" and said local military units had been put on standby if needed.
The stampede occurred on the final day of the pilgrimage at the Sabarimala shrine, located in Idukki district, about 200 kilometres from the state capital Thiruvananthapuram
The governor of Kerala, a popular holiday destination and spice-growing region with sandy beaches and lush green mountains, said he was "shocked and saddened" at the loss of life.
Prime Minister Manmohan Singh said his thoughts were with the relatives of the victims and announced compensation payments of 100,000 rupees (2,200 dollars) to the dead's next of kin.
Anxious pilgrims with missing relatives thronged local hospitals on Saturday morning, as officials tried to identify the bodies brought down from the stampede site.
Most of the victims were from the neighbouring states of Tamil Nadu, Andhra Pradesh and Karnataka.
Under the customs of the pilgrimage, hundreds of thousands of men and women set off on foot in groups for the Sabarimala temple, each carrying a cloth bundle containing traditional offerings.
But many of the elderly, or those short of time, opt to cram into overloaded buses and jeeps to travel as close as possible to the temple, which is believed to be where the god Ayyappa meditated.
The shrine is packed with devotees throughout the pilgrimage season from November to January.