Meanwhile, hosts United Arab Emirates cemented their place as Group A winners after a 1-1 draw with Thailand in Al Ain that also saw the Thais progress.
India were moments away from reaching the knockout phase for the first time before Jamal Rashid converted a dramatic penalty to shatter their dreams and sneak Bahrain through.
After the defeat, Constantine announced he would step down after completing his second spell in charge.
"Stephen Constantine is stepping down after this game," said the Englishman, speaking in the third person.
"I've been here for four years. My objective from day one was to qualify for the Asian Cup. I've done that and we've broken a few records. I'm exceptionally proud of the players for everything that they've given.
"I think my cycle is finished," added the 56-year-old, who rejoined India for a second spell in 2015.
"I did what I was asked and quite a bit more and it's time for me to move on."
India stunned Thailand 4-1 in their opening game -- their first Asian Cup win in more than 50 years -- but a 2-0 defeat by UAE made for a nervous encounter in Sharjah.
Television showed cut-aways of India's substitutes biting their fingernails throughout the match, while Constantine prowled the touchline in dark suit and sunglasses barking instructions to his players.
India were indebted to centre-back Sandesh Jhingan, who was being compared on Twitter to Liverpool's Virgil van Dijk by the legions of fans watching on TV back home.
The giant, bushy-bearded, defender was immense all game to keep India's clean sheet intact.
But the Blue Tigers survived a scare when substitute Mohamed Jasim Marhoon's header hit the post after 71 minutes.
- Clinging on -
India were clinging by the end, at one point packing all 11 players on their own goal line to defend an indirect free kick.
Veteran striker Sunil Chhetri, who overtook Lionel Messi in international goals during the win over Thailand, flew out to get the crucial block.
Just as it looked like India would hold on for a famous point, Pronay Halder committed a rash foul and Rashed stepped up to smash the ball down the middle to knock India to the bottom of the Group A and out of the tournament.
"Obviously we are bitterly disappointed," said Constantine. "The boys are sick, as you can imagine. I don't think we deserved it but it's the stark reality.
"It's unlucky we couldn't hold out for another three or four minutes," he added. "But I'm pretty sure no one expected us to qualify for the Asian Cup and the boys have exceeded expectations."
"That is the first time in this tournament I have seen my real Bahrain team show up," said Bahrain coach Miroslav Soukup.
"India defended amazingly but I felt that they were thinking maybe a little bit too much about a draw."
It looked like being an easy night for the Emirates in Al Ain when striker Ali Mabkhout, top-scorer at the 2015 tournament, netted after just seven minutes.
As the white shirts poured forward, Ismail Al Hammadi's chipped shot came back off the bar and Mabkhout perfectly placed to nod home the rebound.
But four minutes before half-time, "Thai Messi" Chanathip Songkrasin's ball into the box triggered panic, leaving Thitiphan Puangjan with a simple tap-in.
"Today I was expecting a better performance. We controlled the game, scored an early goal but then we conceded a goal and we lost concentration," said UAE coach Alberto Zaccheroni.