As early as 2005, Informatica started preaching the use of an ETL framework to create a proper data integration methodology. At the time, many organizations relied on an enormous number of scripts to integrate data. Despite their best effort, data remained fragmented in many different silos. It seemed that as users developed more scripts, problems grew more complex and solutions became harder to find. Since then, many data integration tools followed and developed this market.
Over a decade later, it feels good to think about the progress made in data integration tools. Many ‘data minded’ folks have achieved great things, and significant gains have been realized for data practitioners.
The cloud transition
Despite this progress, current ETL-based data integration tools like Informatica or Talend utilize architectures which simply do not address the newest and most important challenge involving data silos: the transition to the cloud.
Today, in a vastly hybrid cloud model, IT loses productivity due to siloed data and incompatible application infrastructures. Traditional data integration tools require a ‘project’ context created in a careful methodology involving the line-of-business owners and IT. The Velocity methodology from Informatica stands out an excellent example. In these cumbersome methodologies, users complain most about the time required between logging a request and receiving data.
One of the main benefits of cloud is agility. However, agility is the first reason why traditional data integration tools cannot be the primary strategy for cloud migration. The issues are many. They use a heavy, lengthy, and project-oriented methodology, a misfit for the cloud environment where requests for data are frequent. In the cloud, users require data copies quickly for a myriad of reasons. Data copies frequently populate geographically-distributed production environments, QA test/dev sandboxes, and analytic data warehouses.
Repackaging legacy tools
Many data integration tools have simply repackaged old technology, but they still rely on the same algorithms and principles. Or, they work on a project basis, meaning they are not designed to respond quickly to new data requests common in a cloud environment. David Linthicum, Chief Cloud Strategy Officer at Deloitte consulting may have said it best:
“The legacy data integration tools of the past no longer cut it, and that results in less value for the business. As we move more workloads into clouds, as well as add stand up IoT and mobile devices, this data integration problem will only get worse”
Many of the options for responding to the new cloud-based data integration demands lead to disappointment. Custom scripts which leverage open source utilities can become extremely difficult and expensive to maintain. Point-to-point solutions specific for a single cloud or migration scenario often lead to vendor lock-in and deepen data silos further. Traditional replication products not designed for the cloud cost too much in large up-front investments. Combining several traditional data integration tools causes cost and complexity to grow out of control and creates unsustainable support burdens.
Unlike legacy data integration tools, Griddable fits the cloud environment. It features a cloud-first architecture and a streamlined user experience that automates cloud operations. By combining many tools into one platform, Griddable flexibly migrates data to heterogeneous databases on multiple public and private clouds. Better still, it supports all types of source and target database topologies.
Griddable’s modern architecture brings scale, robustness, versatility and transactional integrity. You will understand it perfectly with this 2-minute video:
Additionally, it is hard to beat Griddable on its ease of use. Setup takes 10-15 minutes before migrating and synchronizing data in real time between a source and a target. This video illustrates how, without any training or particular set up, a user can start moving data.
To see Griddable in action, click the “Demo Now” button for a 10 minute, no-obligation tour.
Best to you with your transition to the cloud!