A series of graft scandals erupted last year, including a massive telecoms licences scam that has undermined Prime Minister Manmohan Singh and a probe into corruption related to the Commonwealth Games.
The open letter, which made the front page of the Times of India, was signed by 14 people including Azim Premji, the founder of software giant Wipro, who is well known for his business ethics and philanthropic work.
Also among the signatories were the heads of carmaker Mahindra & Mahindra, leading conglomerate Godrej & Boyce and HDFC bank, as well as senior judges and former governor of India's central bank Bimal Jalan.
They expressed concern that India's rapid growth was being derailed by corruption, which they called "the biggest issue corroding the fabric of our nation" and one that needed to be tackled "on a war footing".
Bribes, extortion and fraud are part of everyday life in India, ranging from small "sweeteners" paid to police to avoid traffic offenses to the recent telecoms scandal that may have cost the government $40 billion in revenue.
The signatories recommended that investigative agencies be free from political interference and said that India needed "genuinely independent" regulatory bodies in addition to the courts.
They also took a swipe at the opposition for causing weeks of parliamentary deadlock by protesting over the telecoms scandal, in which second-generation mobile phone licences were sold off at bargain prices.
"It is imperative to establish ways and means, for every Indian, to feel that they have genuine equal opportunities, access to decision makers and a stake in India's progress," the letter said.