Aides said he was "surprised and saddened" by the results.
Both Berlusconi's PDL party and its Northern League allies did badly in the first round of voting on Sunday and Monday in elections in 1,310 towns and 11 provinces. The polls were seen as a key test for the premier midway through his third term.
Berlusconi's centre-right bloc suffered the biggest shock in their stronghold of Milan, trailing a centre-left candidate who forced them into a run-off for the first time in 14 years.
Berlusconi, embroiled in a sex scandal and facing three corruption trials, received 47 percent fewer preference votes than he did at the last election in Milan five years ago.
In contrast to the period before the vote, when he bombarded the airwaves with comments in a round of frenetic campaigning, Berlusconi remained uncharacteristically silent on Tuesday, 24 hours after polls closed in the two-day first round. The media magnate had turned the elections into a vote on him and his national policies, but the strategy spectacularly backfired.
Analysts said that although the centre right could fight back in run-offs in two weeks, the vote had undermined Berlusconi's reputation as Italy's dominant political force for the first time since he stormed to power in 1994.
They widely predicted political instability because the losses damaged an alliance with the anti-immigrant, pro-devolution League, on which his government depends.
Voters sick of political mudslinging also rewarded candidates from outside the big parties.
Massimo Franco, a respected commentator in Corriere della Sera, said the vote had begun "the reassessment of a leader who after presenting himself and being considered by allies as a god, now risks becoming a scapegoat".
Milan's centre-right mayor Letizia Moratti, Berlusconi's candidate, won 41.6 percent of the vote against 48 percent for her rival Giuliano Pisapia, giving the left a chance to win Italy's financial capital for the first time in nearly 20 years.
Her clumsy attempt to tar Pisapia as a dangerous leftist misfired with electors, as did Berlusconi's constant attacks on magistrates who have charged him in four concurrent trials.
Milan is Berlusconi's hometown, where he built his business fortune and launched his political career, making the setback there all the more significant.
The vote confirmed indications from opinion polls showing his popularity has been undermined by a sex scandal, three corruption and tax fraud trials and a faltering economy.
"He is saddened, surprised and saddened. He did not expect a result like this," aides told Italian news agency ANSA. The Northern League fared worse than expected in its heartland and media reported its leader, Umberto Bossi, blamed Berlusconi and his attempts to pass laws to evade a string of prosecutions.
The centre left swept up Turin and won a first round victory in its stronghold of Bologna, while Berlusconi's PDL party led in Naples, which is also set for a run-off in two weeks.
"The wind in the north is blowing against the PDL and the League," said Pierluigi Bersani, leader of the largest opposition party, although it paid for its notorious internal squabbles in Naples, where its candidate was eliminated.
On Tuesday night Berlusconis' lawyers filed a suit with the Constitutional Court arguing that the Milan court trying him on prostitution charges is not competent and that he should be tried by a Rome tribunal that hears cases involving ministers.
The premier denies all the charges and says politically biased leftist magistrates are hounding him.
Berlusconi is also taking the heat for failing to revive Italy's chronically low growth. The economy expanded just 0.1 percent in the first three months of the year, well below rates in Germany, France and even crisis-hit Greece.
The League, which is vital for Berlusconi's survival after a split in the ruling PDL party last year, has marked its distance from the premier on several issues in recent weeks, notably by opposing Italy's involvement in the NATO bombing of Libya.
It had hoped to cash in on Berlusconi's weakness, but it failed to boost its share of the vote in the big cities. In Milan and Turin it won less than 10 percent support.