The kiwi fell as much as 1.6 percent. The Australian dollar also weakened, dropping half a percent.
"The market was taken by surprise by the dovish tone (of the Reserve Bank of New Zealand)," Thu Lan Nguyen, an analyst at Commerzbank said. "Most central banks have turned dovish. Even those that hiked interest rates did it with a very cautious outlook on rates."
The US Federal Reserve abruptly ended three years of monetary policy tightening last week amid signs of an economic slowdown. The European Central Bank, Reserve Bank of Australia and the Bank of Japan have all turned dovish since.
The kiwi fell to $0.6794 after the RBNZ left rates unchanged and said its next move would probably be lower. The Australian dollar weakened to $0.7103.
Major currency markets were quiet elsewhere as risk appetite remained fragile after a selloff last week triggered by fears of slowing global growth.
The US dollar gave up earlier gains but held firm after US government bond yields bounced off recent lows. Against a basket of currencies, the dollar index was flat at 96.736 .
The euro rose to $1.12758.
ECB President Mario Draghi said on Wednesday that policymakers could delay an interest rate increase again if necessary, but such dovishness has already been priced into the single currency, analysts said.
"Despite the dovish Fed last week, the trade-weighted dollar continues grinding higher as investors retain concerns about global growth and other central banks retain or shift towards a more dovish bias," ING analysts said in a note.
Emerging markets were mostly weaker. The Turkish lira fell another 1.4 percent to 5.4028 lira per dollar after gaining earlier in the week, when local banks acted to support the currency.
The pound lost 0.3 percent to $1.3167 before steadying near $1.32 as Britain's parliament prepared to vote on various Brexit options.
Parliament has taken control of the Brexit process from Prime Minister Theresa May for a day, and the lawmakers will vote on the Brexit options as they look for a way to break its deadlock over how to leave the European Union. GBP/
The Japanese yen rose 0.3 percent, trading at 110.31 yen per dollar.