The benchmark Nikkei 225 index fell 0.32 percent or 65.99 points to 20,519.32 in early trade while the Topix index was down 0.30 percent or 4.44 points at 1,494.79.
Investor sentiment remained in the doldrums as the yen held up on safe-haven buying amid worries about the trade war between the world's two largest economies, analysts said.
A strong yen clouds the outlook for Japanese exporters as it reduces their profits when they are repatriated.
"The yen remains strong but its strength comes from uncertainty over dollar-yuan rates," said Seiichi Suzuki, senior market analyst at Tokai Tokyo Research Institute.
Market players "don't like this unpredictability," he told AFP, adding they would be watching closely how the yen moves on Wednesday.
The dollar fell to 106.15 yen early Wednesday from 106.49 yen in New York Tuesday afternoon.
Tensions have risen sharply since last week when US President Donald Trump announced fresh tariffs on Chinese goods from September 1. The yuan then slumped in a move seen as a retaliation by Beijing.
On Wall Street on Tuesday, major US indices climbed more than one percent after China moved to stabilise its currency, ending a five-day losing streak for the Dow and a six-day skid for the S&P 500 and Nasdaq.
In early Wednesday Tokyo trade, SoftBank Group slipped 0.40 percent to 5,179 yen before the IT investor announces its April-June results.
Fast Retailing, which operates clothing chain Uniqlo, dropped 0.54 percent to 63,380 yen while Sony rose 1.31 percent to 5,874 yen.