Seamer Dhammika Prasad removed both openers by the third over and left-arm spinner Rangana Herath also chipped in with a wicket as the tourists plodded to 38 for three by tea.
Prasad had Mohammad Hafeez caught at third slip off his sixth delivery and then trapped Ahmed Shehzad leg-before in his next over to reduce Pakistan to 11 for two.
It became 35-3 when Herath, brought in to the attack in the sixth over, had Azhar Ali leg-before in his third over.
The collapse left the two veterans, Younis Khan (17) and skipper Misbah-ul Haq (two), unbeaten at tea with their team still 262 runs behind.
Sri Lanka's innings was built around a painstaking century from opener Kaushal Silva, who defended solidly at one end even as Pakistan ran through the batting at the other end.
The diminutive right-hander lit up the rain-hit Test with a stubborn 125, his second Test century after making 139 against Bangladesh in Dhaka in January last year
Silva, however, failed to carry his bat through the innings when he was ninth out from the 300th delivery he faced, caught behind off left-arm spinner Zulfiqar Babar.
Silva, who began the day on 80, had reached his century 30 minutes before lunch when he drove Babar through the covers for his 15th boundary.
Bad weather had washed out the entire first day's play on Wednesday and just 64 overs were bowled on the second day due to a wet outfield at the Galle International Stadium.
Sri Lanka added 11 runs to their overnight score of 178-3 before they lost the wicket of skipper Angelo Mathews, who was bowled by left-arm seamer Wahab Riaz for 19.
Dinesh Chandimal contributed 23 in a fifth-wicket stand of 37 with Silva when he was bowled by Babar, who was surprisingly handed the second new ball which was claimed after 80 overs.
Kithuruwan Vithanage was the third batsman to be dismissed in the morning session when, on 18, he gave Hafeez an easy return catch.
Sri Lanka went to lunch at 273-6, but leg-spinner Yasir Shah dismissed Dilruwan Perera off the second ball after resumption, wicket-keeper Sarfraz Ahmed taking a catch.
Babar removed Prasad and Silva in successive overs before Yasir held a skier off his own bowling to dismiss last man Nuwan Pradeep.
Wahab and Babar finished with three wickets each, while Hafeez and Shah took two each.
The three-Test series will be followed by five one-day internationals and two Twenty20 matches. – AFP
Original source: Pakistan lose wickets cheaply