Tokyo's Nikkei average fell 0.78 percent on Thursday after concerns about the impact of record high oil prices on the global economy encouraged investors to take profits in recent winners including retailer Aeon.
Bank shares, which had drawn buyers betting on a sustained recovery by Japan's economy, rang up losses but auto stocks gained after trade figures showed signs of a pick-up in exports.
Analysts said the high oil prices and a fall in US stocks put a dampener on the Japanese market, which looks ripe for a correction after recent rapid gains.
"The Japanese market can hold up under profit-taking pressure as long as US markets stay firm," said Yumi Nishimura, manager of the investment advisory section at Daiwa Securities SMBC Co. "But we have to be careful."
The Nikkei lost 97.10 points to 12,405.16, after closing above 12,500 on Wednesday for the first time since July 5, 2001.
The broader TOPIX index slipped 0.59 percent to 1,268.06.
Aeon, Asia's largest retailer, lost 2 percent to 1,985 yen, after hitting a high for the year of 2,080 yen on Tuesday.
Ito-Yokado Co shed 3.9 percent to 3,940 yen ahead of its delisting on Friday before it forms a holding company with Seven-Eleven Japan Co and Denny's Japan Co.
Seven-Eleven fell 4.1 percent to 3,270 yen and Denny's lost 3.1 percent to 2,160 yen.
Casual wear retailer Fast Retailing Co, which will be added to the Nikkei average in place of Ito-Yokado, jumped 9.8 percent to 9,000 yen.
Ryohin Keikaku Co Ltd, which sells Muji brand clothing and household goods, rose 6.4 percent to 6,530 yen after Goldman Sachs raised its rating to "outperform" from "in-line".
Bank shares came under selling pressure, with second-ranked Mitsubishi Tokyo Financial Group down 3.5 percent at 1.11 million yen and No 3 lender Sumitomo Mitsui Financial Group losing 3 percent to 873,000 yen.
Japan's largest lender, Mizuho Financial Group, ended down 2 percent at 602,000 yen.
Government data showing that exports rose 4.3 percent in July from a year earlier boosted some exporters. Honda Motor Co was up 0.9 percent at 5,900 yen and Toyota Motor Corp rose 1.1 percent to 4,540 yen
Some oil-related shares gave up early gains as oil prices came off a record $68 a barrel and after Japanese refiners suspended some operations due to the approach of a powerful typhoon.
Top refiner Nippon Oil Co closed down 1.4 percent at 852 yen and Teikoku Oil fell 0.4 percent to 907 yen.
Despite missing out on rises in the stock market, high-tech shares were unlikely to entice investors because of a clouded outlook for the sector, analysts said.
Profit prospects for technology stocks look dim, said Hideaki Saito, a senior strategist at Nikko Asset Management.
Technology stocks have peaked and are entering a correction phase, Deutsche Securities analyst Fumiaki Sato said in a report on Thursday, pointing to weak personal computer sales and semiconductor shipments.
Trade volume stood at 1.78 billion shares, slightly less than Wednesday's 1.69 billion shares. Decliners outnumbered advancers 1,135 to 394.