The benchmark Nikkei 225 index fell 0.33 percent or 68.75 points to 20,516.56 while the broader Topix index ended slightly higher at 0.05 percent or 0.70 points at 1,499.93.
Investor sentiment remained in the doldrums as the yen held up on safe-haven buying amid worries about the trade war between the world's two largest economies, analysts said.
A strong yen clouds the outlook for Japanese exporters as it reduces their profits when they are repatriated.
"The yen remains strong but its strength comes from uncertainty over dollar-yuan rates," said Seiichi Suzuki, senior market analyst at Tokai Tokyo Research Institute.
Market players "don't like this unpredictability," he told AFP.
The dollar fell to 106.21 yen from 106.49 yen in New York Tuesday afternoon.
Tensions have risen sharply since last week when US President Donald Trump announced fresh tariffs on Chinese goods from September 1. The yuan then slumped in a move seen as a retaliation by Beijing.
On Wall Street on Tuesday, however, major US indices climbed more than one percent after China moved to stabilise its currency, ending a five-day losing streak for the Dow and a six-day skid for the S&P 500 and Nasdaq.
China's central bank "conclusively demonstrated its capacity to steer global markets while effortlessly neutralising the 'currency manipulator' tag that could have mushroomed into overblown proportions", said Stephen Innes, managing partner at VM Markets.
In Wednesday Tokyo trade, Fast Retailing, which operates clothing chain Uniqlo, dropped 0.65 percent to 63,310 yen while Sony rose 1.94 percent to 5,911 yen.
SoftBank Group closed down 0.23 percent at 5,188 yen just before the IT investor announced its April-June net profit more than tripled thanks to exceptional gains related to the sale of shares in Chinese e-commerce giant Alibaba.