Pellegrino, who took over in June 2018 after being sacked by Southampton, led Leganes to a 13th-place finish in La Liga last season.
But they are yet to win this term and sit bottom of the table after seven defeats and two draws.
A club statement read: "Leganes and Mauricio Pellegrino have decided, by mutual agreement, to part ways."
"The Argentine coach, who managed last season to keep the team for one more year in the top flight, will not manage the team next weekend against Mallorca."
Luis Cembranos and Carlos Martinez, who were previously the coaches of Leganes B, have been put in temporary charge.