Bayelsa has been ruled by the opposition Peoples Democratic Party since Nigeria returned to civilian rule in 1999.
It is a key political battleground because of the lucrative oil and gas sector that is largely based in the southern Delta region.
David Lyon of Buhari's All Progressive Congress (APC) won more than twice the number of votes as his closest rival Douye Diri of the PDP, said the Independent National Electoral Commission.
APC supporters celebrated across the state when the results came in, dancing and singing. Major roads in the capital were blocked by jubilant APC sympathisers, AFP correspondents saw.
In a statement on Monday, the president congratulated the new governor, while deploring the violence that characterised the vote.
"Violence during elections vitiates our commitment to demonstrate to the world and upcoming generation that we are a people capable of electing leaders in a peaceful and orderly manner," he said.
Those unhappy with the results should seek legal redress rather than resort to violence, he added.
Political analysts see the APC's victory in an opposition stronghold as a sign of Buhari's growing influence.
"The victory is a big boost for APC and Buhari," said Dapo Thomas, a politics and history lecturer at Lagos State University.