The tourists went to lunch at 133-3, still 41 runs behind Sri Lanka's meagre first innings total of 174 with seven wickets in hand.
Australia lost Shane Watson before they had added to their overnight score of 60 for no loss, and the other opener Phil Hughes fell shortly after the drinks break.
Watson had made 36 when he left alone a delivery from seamer Suranga Lakmal that dipped in to uproot the off-stump.
Left-handed Hughes carried his overnight score of 23 to 36 when he was snapped up at forward short-leg by Tharanga Paranavitana off Suraj Randiv.
It soon became 116-3 as skipper Michael Clarke (13) attempted to drive seamer Chanaka Welegedara outside the off-stump and edged a low catch to Mahela Jayawardene at second slip.
Shaun Marsh, making his Test debut in place of former captain Ricky Ponting, hung around for two hours to make an unbeaten 26.
The left-hander hopes to get a big score and cement his place in the side once Ponting returns from home for the third and final Test in Colombo following the birth of his second child.
Michael Hussey was the other unbeaten batsman at the crease on 15.
Australia lead 1-0 in the three-match series, having won the first Test in Galle by 125 runs last week.