The city is totally centered on Plaza 25 de Mayo and from there you can walk to pretty much anywhere. Exceptions include heading out to the nightclubs on Avenida Sarmiento, for which you must take a bus marked Chaco/Corrientes and the Omnibus terminal for which you must take bus numbers 3 or 10 from the corner of Santa Maria de Oro and Juan D. Peron.
If you want to check out the city of Corrientes, Remises (unofficial taxis) leave from in front of Calle Frondizi 151 and it is possible to share the fare with other travelers heading in the same direction. The Remises usually won't leave until they are full, but with four people in the car, the fare to Corrientes is only $.75 and the trip takes about 15 minutes.