The latest move sent the rand up almost 5% on Sunday night.
Mr Gordhan was widely respected when he served as South Africa's finance minister from 2009 until 2014.
However, Mohammed Nalla, head of research at Nedbank Capital, said having a finance minister serve just two days did not bode well for South Africa's reputation.
"International investors are probably thinking: why didn't the president make a much more considered decision in the first place?" he said.
The leader of the opposition Democratic Alliance party, Mmusi Maimane, said: "This is reckless by President Zuma - he is playing Russian roulette with the South African economy."
A statement from Mr Zuma's office said he had "received many representations" to reconsider his decision to appoint Mr van Rooyen.
"As a democratic government, we emphasise the importance of listening to the people and to respond to their views," it added.
Credit agency Fitch downgraded South Africa earlier this month, leaving South Africa just one notch above "junk" status. It said on Thursday that Mr Nene's sacking "raised more negative than positive questions".
Mr Nene's reluctance to approve a plan to build several nuclear power stations at a cost of up to $100bn is thought to have contributed to his removal as finance minister.
Mr van Rooyen will take over from Mr Gordhan as minister of co-operative governance and traditional affairs.
Marches to call for Mr Zuma's removal as president are being planned for five cities in South Africa on Wednesday - a public holiday.
The US Federal Reserve is expected to raise interest rates on the same day in a move that could put economies in countries like South Africa under further pressure.
Former Health Minister Barbara Hogan on Friday called on Mr Zuma to resign. The highest-profile ANC member to oppose Mr Nene's removal, she said that the president had crossed a line and needed to be held to account.
Razia Khan, an analyst with Standard Chartered bank, said the week's turmoil was "perhaps the first instance since 2007 that Zuma has come under severe pressure within the party".