Eliot Engel, who heads the House Foreign Affairs Committee, said he had learned that Linick had opened an investigation into Pompeo.
"Mr. Linick's firing amid such a probe strongly suggests that this is an unlawful act of retaliation," the congressman said in a statement.
A Democratic congressional aide, speaking on condition of anonymity, said Linick was probing complaints that Pompeo misused a political appointee to perform personal tasks for himself and his wife.
Washington's top diplomat has raised eyebrows for frequently traveling the world on his government plane with his wife Susan Pompeo, who has no official role.
CNN reported last year that a whistleblower had complained that Diplomatic Security, which guards US missions overseas as well as the secretary of state, had been assigned questionable tasks for the Pompeos such as picking up the family dog and takeout food.
The State Department confirmed Linick's firing but did not comment on the reason or on whether Pompeo was under investigation.
A State Department spokesperson said the new inspector general will be Stephen Akard, a former aide to Vice President Mike Pence from his home state of Indiana.
Akard since last year has led the State Department's Office of Foreign Missions, which handles relations with diplomats in the United States.
House Speaker Nancy Pelosi said Linick was "punished for honorably performing his duty to protect the Constitution and our national security."
"The president must cease his pattern of reprisal and retaliation against the public servants who are working to keep Americans safe, particularly during this time of global emergency," Pelosi said.
Pompeo is one of Trump's most trusted aides - and a rare one never to come publicly into the crosshairs of the mercurial president. In recent months Pompeo has moved US foreign policy forcefully to the right - encouraging a drone strike that killed a top Iranian general and promoting a theory, discounted by mainstream scientists, that the COVID-19 pandemic originated in a Chinese laboratory.
Pompeo, 56, is widely seen as entertaining presidential ambitions of his own. But he has frustrated some Republican colleagues who want him to quit and return to his state of Kansas to run in November for a Senate seat that is looking surprisingly vulnerable for the party.
Linick, a longtime prosecutor, was appointed in 2013 by Trump's predecessor Barack Obama to oversee the $70 billion juggernaut of US diplomacy.