Lyon midfielder Henry struck in the 107th minute after Thaisa's equaliser for Brazil, following Valerie Gauvin's opener, had forced a 1-1 draw in 90 minutes.
France will face either reigning champions the United States or Spain in the last eight in Paris on Friday.
Pre-tournament favourites France will be playing in the quarter-finals for the third straight World Cup, while Brazil have now suffered back-to-back last-16 defeats after their 2015 loss to Australia.
After the chaotic scenes of the earlier last-16 tie on Sunday which saw an ill-tempered Cameroon refuse to play for several minutes during their loss to England -- behaviour Lionesses boss Phil Neville described as "unacceptable" -- this was another hard-fought affair, but without serious controversy.
The 41-year-old Formiga returned from suspension for Brazil to replace Andressinha in midfield alongside Marta, who broke the all-time World Cup goalscoring record with her 17th strike in the previous group-stage win over Italy.
The first sight of goal saw Marta create herself some space with clever footwork outside the box but fire wide, before Debinha saw an effort deflected past the post by the boot of Wendie Renard.
Although France were struggling to put Brazil under any pressure early on despite being roared on by a partisan crowd at the Stade Oceane, they thought they had taken the lead in the 23rd minute when Gauvin bundled Kadidiatou Diani's cross over the line.
Both Gauvin and Brazilian goalkeeper Barbara were left needing medical attention after colliding while going for the ball, and a VAR review unsurprisingly saw the goal ruled out for a foul by the France striker.
- Gauvin strikes -
==================
France were much-improved after the half-time interval, and grabbed the lead with their first shot on target in the 52nd minute.
It was Diani who caused all the problems for the Brazil defence, powering into the box and squaring for Gauvin to slide home her seventh goal in nine internationals.
Cristiane almost replied immediately for Brazil, but goalkeeper Sarah Bouhaddi tipped her header onto the crossbar.
The South Americans did hit back in the 64th minute, as Thaisa slammed home, with the equaliser eventually being given having originally been disallowed for an offside against Debinha.
There were no further twists in the regulation 90 minutes, though, as Eugenie Le Sommer made a total mess of a free header and Tamires had a goal ruled out for offside.
France piled on the pressure in the opening half of extra time, but were almost caught on the counter-attack as Debinha broke free, only to be denied by an excellent goal-saving block by Griedge Mbock Bathy.
And the winning goal came two minutes into the second period, as Henry escaped the attentions of Monica to meet Amel Majri's free-kick with a sliding volley into the bottom corner.