Easily move data from the mainframe to the cloud with Mainframe Connector

Cloud migration is a necessity for many businesses that process and store large amounts of data. The procedure leads to data being stored and applications being hosted in an environment that is more agile than a traditional mainframe. Those who do not move to the cloud are unable to harness the power of systems and applications that cannot be run locally. Protecting data is top of mind. In fact, 65 percent of 200 executives polled across the globe cited moving sensitive data as their top worry when migrating to the cloud. Such concern can lead businesses to hesitate in migrating to the cloud. Ultimately, this hesitation can result in missing out on the opportunity to get the most of their data.

Fortunately, there is now an open-sourced Google Cloud Mainframe Connector that simplifies data migration and eliminates some of the concerns that plague the process.

SoftServe engineers, working as part of Google Professional Services team, helped create and bring Mainframe Connector to market. Accordingly, SoftServe associates are uniquely qualified to make adjustments and identify ways in which the tool can be enhanced to meet specific business needs.

Mainframe Connector brings the power of Google Cloud and BigQuery to the Mainframe

Essential to cloud migration, the ELT pipeline consists of three parts:

Mainframe Connector allows users to easily move through all these steps in the mainframe operating system (z/OS). This is done by using job control language (JCL), which also allows Google Cloud users to analyze data in BigQuery from batch jobs on the mainframe.

The connector transcodes data into Optimized Row Columnar (ORC) files, which are compatible with BigQuery and optimize the task of processing data. As a result, moving data between the mainframe, Cloud Storage, and BigQuery is simplified, and tasks performed on the mainframe can benefit from cloud services.

A cloud migration solution for large data sets

When dealing with massive amounts of data, such as that of an enterprise client, cloud migration efficiency is vital. Mainframe Connector was, in part, developed to accelerate mainframe Teradata migration to BigQuery on Google Cloud for a global CPG organization, which not only has such quantities of data, but relies on it for demand forecasting, insight into workflow, and knowledge of how distribution centers are functioning. Information gleaned from such data analysis has only become more important as supply chain issues continue to create headwinds in the sector.

Mainframe Connector can handle large data transfers without concerns by operating in a Java Virtual Machine (JVM), which helps manage workloads in the mainframe processor. Additionally, the gRPC (Google Remote Procedure Call) server included in the connector further helps with processing by performing big computation operations in the cloud. The solution accelerates mainframe Teradata migration to BigQuery on Google Cloud.

Providing a solution across verticals

Mainframe Connector works for more than simply enterprises looking to migrate enormous amounts of data to the cloud. The simple process of connecting to Google Cloud Storage using the tool is largely the same, regardless of the business:

1. Read the dataset on the mainframe.
2. Convert encoded dataset to ORC.
3. Upload ORC to the cloud.
4. Register ORC as a table.
5. Run MERGE DML to load new data into the table.

However, there are likely to be unique requirements for different sectors and verticals. In such cases, fine tuning and changes to the migration process can be made.

Let’s talk about how SoftServe can provide the experience and expertise to make cloud migration seamless.