Legacy did not evolve in a structured manageable manner

There are several reasons for these challenges, of course hardly any of which can be attributed to any errors in their management, but rather to the nature of evolution of these legacy systems. For example,

• Heterogeneity of the systems and workloads across business functions: Typical examples are proprietary ETLs such as Informatica or DataStage, Databases and Data Warehouse such as Teradata or Netezza, scripts such as BTEQ scripts, specialized jobs, stored procedures, proprietary reporting scripts etc.

• Isolated and distributed nature of the systems: Most organizations will have their data, reporting and analytics scripts evolve in different systems over different time periods. E.g. sales leads and logistics reports can be in different systems and accessed through different batch processes. This also leads to issues related to non-standardization, disparity of terminologies related to business semantics as well as lineage issues.

• Customizations over the years: On multiple occasions, the systems are customized to suit the needs of the business, for example, business logic is embedded into scripts and stored procedures.

• Lack of documentation of value: More often than naught, either these implementations and customization are out of date on documentation or have documentation that does not really help explain the behavior of the systems

• Non availability of the resources responsible for building these systems: Most resource that implemented these systems are either not with the organization anymore or have been reassigned to other important business within.

• Limitations in accessing new resources who can decipher these systems: The resource issue is not so easy to resolve either, as such resources are not available for hire in this age. For example, engineers or developers who can work on VSAM or ISAM scripts on mainframes.

Concerns migrating an unknown

These challenges pose several difficulties and lead to questions in the minds of the business executives as well as the data architects, viz.

• Would we be able to migrate all of our legacy to the cloud successfully? Can we understand the complexities involved in this migration? What level of success can we expect?

• Would we be able to get some information on each of the workloads to understand the ROI and prioritize which workloads we migrate first?

• Would we be able to optimize the workloads for the new systems for long term savings as we use the migrated workloads in future?

• Can we estimate the time, effort and cost needed to migrate from these legacy systems?

• How would we cost effectively validate that the migration is indeed successful?

• Where would we find the resources needed to perform this migration?

It is not that this spells doom to the migration initiatives that the businesses would like to undertake, though this remains the single most important challenge to address and it surely is the very first one the businesses will come across.