The result cuts short Tsipras's bid to lead Greece on into the post-bailout era after presiding over its fraught rescue by international creditors at the height of the European debt crisis.
Leftist Tsipras called New Democracy leader Kyriakos Mitsotakis to congratulate him, a source in the prime minister's office told AFP.
Early results from 40 percent of polling stations showed New Democracy scoring a crushing victory by nearly 40 percent to over 31 percent for Tsipras's leftist Syriza party.
"The result has been determined... but we'll be back," outgoing finance minister Euclid Tsakalotos told state TV ERT.
If the results are confirmed, the 51-year-old Harvard graduate and former McKinsey consultant Mitsotakis will have a majority of 158 lawmakers in the 300-seat parliament. Tsipras's party will have 86 seats.
The final number will depend on how smaller parties fare. They need at least 3.0 percent of the vote to enter parliament.
The contents of at least one ballot box will likely be unaccounted for, after it was stolen by suspected anarchists in Athens, police said.
- New parties -
===============
New arrivals fighting to secure representation are Greek Solution, a nationalist party formed by TV salesman Kyriakos Velopoulos, and MeRA25, an anti-austerity party formed by maverick economist and former Greek finance minister Yanis Varoufakis.
According to the exit polls, Varoufakis's party could elect nine lawmakers.
Greek Solution could end up with 10 deputies, while neo-Nazi party Golden Dawn looks likely to be shut out of parliament for the first time since 2012.
Golden Dawn, until recently Greece's third-ranking party, is in steep decline amid an ongoing trial for the 2013 murder of an anti-fascist rapper, allegedly carried out with the knowledge of senior Golden Dawn members.
- A family affair -
===================
New Democracy was last in power in 2014, in coalition with the Greek socialists.
Mitsotakis is a scion of one of Greece's top political families.
He is the son of former prime minister Constantine Mitsotakis, one of the country's longest-serving parliamentarians.
His sister is former minister Dora Bakoyannis, Athens's first female mayor. And new Athens mayor Costas Bakoyannis, elected in May, is his nephew.
Sunday's election is Greece's third in as many months, and the first held in midsummer since 1928.
In May, New Democracy beat Syriza by nearly 9.5 points in European parliament elections. A week later, it completed a near-sweep of Greek regions in local elections.
After that, Tsipras was forced to call an early election. His term was scheduled to end in the autumn.
Greece's youngest premier in more than a century, Tsipras had trailed in the polls for months amid widespread dissatisfaction over high taxes.
In parts of the country, there was also a backlash against a controversial agreement with North Macedonia that ended a bitter 27-year dispute over the country's name.
"Greece is exiting 10 years of crisis and the new government will have the heavy task to give a chance to the country to recover completely or to sink", 36-year-old Aphrodite told AFP, as she cast her vote in the bohemian downtown neighborhood of Exarcheia.
"I hope that from tomorrow we will be able to breathe with relief. To take a deep breath, if Mitsotakis does what he promises," added Athinodoros, a 48-year-old self-employed worker.
Tsipras has accused Mitsotakis -- who was part of a 2012-2014 crisis government -- of "disastrous" mismanagement that brought hundreds of thousands of job losses and business failures.
Mitsotakis has now pledged to create "better" jobs through growth, foreign investment and tax cuts and to "steamroll" obstacles to business.
Tsipras -- who reduced unemployment and raised the minimum wage for the first time since 2012 -- was criticised for campaigning as an anti-austerity crusader before eventually accepting a third EU bailout and the economic cutbacks that entailed.
The election was held a few weeks before the first anniversary of the Mati fire, Greece's deadliest wildfire that killed 102 people near Athens.