We get nightly data extracts from the databases hosted at different service provider sites through FTP in flat file format.
Existing system can be unstable during FTP downloads at times and data quality is also questionable.
Now we have an opportunity to modify the whole process.
Would you be able to share your best practices for data integration in batch format. Would it be an overkill to have a VPN connection with an API. How about using XML file to improve data quality.
Thanks for sharing your ideas...