The Undavalli Caves are located at a distance of 6 km away from Vijayawada. It is to the south west of Vijayawada. The caves have been cut out of a hill of Sandstone. The caves date back to the 4th and 5th century AD. The cave is four-storey tall and houses a statue of Lord Vishnu. The statue has been sculptured out of a single block of granite rock. There are also other caves located at the site which are dedicated to other Gods.
The caves are also designed as Buddhist monasteries. The monks have used it as a rest house during the rough monsoon season. The caves face the Krishna River.