The S&P/ASX 200 index closed down 0.4% or 24.60 points to 6,793.40, but the benchmark rose 0.6% on Thursday and gained 1.4% this week.
Markets were jittery after the ECB unexpectedly held rates steady as central bank President Mario Draghi cautioned about pulling the trigger too quickly on policy easing.
Financials lost 0.7% with the biggest lenders Commonwealth Bank of Australia and Westpac Banking Corp falling 0.5% and 1%, respectively. The sub-index, however, gained 1.8% for the week.
The focus is now squarely on what the Fed decides at its July 30-31 policy committee meeting, with expectations running high for a cut in rates.
However, strong jobs data released on Thursday showed sustained strength in the U.S. labour market which has left investors concerned the Fed may choose to wait to cut another day.
"Markets are getting nervous that the Fed could disappoint next week," Edward Moya, senior market analyst at OANDA said in a note.
Tech stocks fell 0.9%, in step with their Wall Street peers that dropped after weak results from chipmakers and Facebook that flagged weaker revenue growth next year.
Limiting the benchmark's losses, mining stocks gained as iron ore prices firmed on a strong demand outlook, but posted its biggest weekly decline since May.
Mining behemoths BHP Group rose 0.6% and Rio Tinto gained 2.1%.
Energy firms added 4% this week, its best since mid-February, after rallying for six straight sessions as tensions in the Middle East supported oil prices.
New Zealand's benchmark S&P/NZX 50 index closed down 0.83% or 90.67 points to 10,807.61.
Auckland International Airport slipped 2.2% and a2 Milk Company dropped 1.4%.