Skipper Ryan Giggs headed the opener and Tom Cleverley hit the post as GB led at the break.
Rashed Eisa equalised with a composed finish in front of an increasingly anxious Wembley crowd.
But substitute Scott Sinclair struck a minute after coming on and Daniel Sturridge's deft chip sealed the win.
Britain's first Olympics victory in 52 years also marked a return to winning ways at Wembley, where its last appearance in the tournament was marked by a 5-3 loss to Denmark in the 1948 semifinals.
“We showed a lot of character,” Giggs said. “For a team that has not been together too long we pulled together and to get goals from two subs also is very pleasing.”
As David Beckham and British royalty watched, the hosts joined Senegal atop Group A with four points to eliminate the UAE. Uruguay has three points after losing to the Africans 2-0 at Wembley earlier on Sunday.
Britain now has a tough match against Uruguay in Cardiff on Wednesday, needing to avoid defeat to progress to the last eight.
“It's still all to play for,” Britain Coach Stuart Pearce said. “Each game is a cup final in the group stage and we felt going into this game it was a must-win. And it's the same with Uruguay now.”