As a capital, Panama City has no shortage of places to access the internet or make a phone call. Chances are, your hostel or hotel will have facilities for both. Lavanderias (laundromats) are also widespread and relatively cheap.
ATMs are plentiful, especially around El Cangrejo's banking district. Banco Nacional, on Plaza Concordia, and Banistmo, on Via España, both change traveler's checks. Changing currency can be a bit trickier. If you have Costa Rican money you're best off changing it before you enter Panama. The international airport has a Banco Nacional that changes money. Pana cambios on Via España can also help you.
Most embassies are in the Bella Vista district.