When dealing with large amounts of data, you cannot rely on being able to map the responses to an object. For XML, use the SAXParser API and write a handler that can process smaller amounts of data at a time without loading the entire response into memory. Jackson and GSON provide the equivalent APIs for JSON. CSV data can be processed one row at a time. Some state must be maintained to track what kind of a row you are processing.
Additionally, if the response is going to take a long time to be streamed, timeouts on the client need to be set appropriately.
You may also need to code your application to be robust to varying network conditions. For example, if you don't receive the whole response from the server, you may not want to be able to consume a partial response.