While United have move forward each year under Solskjaer, at least in terms of league position, since his appointment in December 2018, catching Pep Guardiola's Manchester City next season still appears a distant prospect.
Solskjaer's first four semi-finals as United boss ended in defeat, including a 2-1 loss to eventual champions Sevilla in last season's remodelled Europa League.
Solskjaer made 10 changes for United's 2-1 defeat at Old Trafford on Tuesday that moved Leicester nine points ahead of Liverpool in the battle for a top-four finish.
The United manager is expected to recall his first-team regulars at home to Liverpool on Thursday in a fixture that was rearranged due to fan protests at Old Trafford 10 days ago.
"If I think about the players they could play, they have a really strong side available. Midfield, wow. Up front, all good players. They might be young but they are good.
"Of course we will never give up," Solskjaer told a news conference. "Man United will never, ever give up. This team, this club has had too many setbacks, too many great comebacks to have that in our DNA.
"You have got to make progress and that is the reality," he said. "You want to finish the season strongly and if we manage second then it is progress.
The game became increasingly open in the final minutes. De Gea made a fine save before an own goal from Jesus Vallejo sealed a 2-0 second-leg victory for United.
The row erupted following Sunday's 3-1 win in the Premier League for United when Solskjaer senior criticised Spurs attacker Son Heung-min for exaggerating a blow to the face.
"I always get fed, I can assure you of that," the 20-year-old midfielder told the paper with a smile, adding that his Norwegian team mates had teased him over the incident when he arrived for training on Monday, asking if he'd had breakfast.
France international Anthony Martial was sent off in the 28th minute in the humiliating October defeat at Old Trafford after retaliating to an elbow from Erik Lamela with a push in the Argentine's face.
"We didn't look anything like (them) and we got conned into a red card and that's something we have to be aware of," Solskjaer told a news conference.
Rashford missed United's FA Cup loss to Leicester City with a foot issue and was ruled out of England's World Cup qualifiers against San Marino, Albania and Poland but the 23-year-old has returned to training.
"I don't know about Anthony, he didn't join in training this morning... but Marcus and Mason trained and hopefully they can get through that with no reaction," Solskjaer told reporters.
De Gea has missed United's last three games after he was given time off to travel to Spain for the birth of his first child.
"He is touch and go for Sunday I think because I can't really know when he came back, so I'm not really sure if it was Monday or Tuesday that he arrived back in Manchester," Solskjaer told reporters.
Uruguay international Cavani has missed United's last four games in all competitions with a muscle injury but trained on Tuesday ahead of the trip to Selhurst Park.
"We're looking OK. There's a couple of doubts for tomorrow that we have to give as much time as possible," Solskjaer told a news conference on Tuesday.