Transfer TB of data from Mainframe to x86 platform / Azure

A_programmers · New User Joined: 24 Mar 2010 Posts: 19 Location: USA

Dear Friends,

We have requirement to send 100 TB+ data from Mainframe to Azure Cloud storage (Or even Windows Server will work). Migrated Data on cloud storage (after EBCDIC to ASCII conversion) will be used for reporting purpose. I have tried sftp to transfer files from Mainframe to Azure blob however performance is very slow i.e. 2 GB data movement is taking 20 minutes. Both Mainframe and Azure are currently present in same US region.

Any advice on how to migrate huge data from Mainframe to x86 platform / Azure / AWS ?

Thanks
John

Robert Sample · Posted: Fri Mar 18, 2022 7:03 pm

2 GB in 20 minutes is about 1.6+ megabytes per second -- not slow but not especially fast. You might want to contract with IBM or some other company to do the data migration for you; they have tools that can speed up the transfer. If nothing else, talk to your systems people and see what can be done to optimize the transfer speed.

A_programmers · New User Joined: 24 Mar 2010 Posts: 19 Location: USA

Thanks Robert for your comments. Due to budget constraints; I thought of exploring in house options and then if nothing works I will involve 3rd party. Regarding optimization; I made sure that target storage is near to Mainframe, tried with multiple buffer size to get the sweet spot buffer size and used SSD storage in target. I will explore further options on Mainframe and will also get in touch with Mainframe admin.