Talend has built a native integration of Stitch into Amazon Redshift Console. Customers using the Stitch cloud-based ETL tool can now see how effectively their data is being imported into Amazon’s data warehouse. Talend also claims that it will deliver an accelerated setup for ETL processes. Stitch already connects to a large number of SaaS data sources that can be brought into Amazon Redshift.
Chris Merrick, Vice President, Stitch Product and Engineering, Talend, said: “Talend is dedicated to serve as an innovative and strategic business partner to help our customers achieve their technology goals.
“As one of the first companies to provide native integration support for Amazon Redshift Console, we are focused on leveraging the agility, breadth of services, and pace of innovation that AWS provides.”
Talend accelerating the ingesting of data
One of the challenges for all ETL projects is the speed with which data can be ingested. Many organisations still ingest data through batch processing. It means that data is rarely real-time or even near real-time. One reason for using batches is that moving to stream processing requires greater management and visibility of data sources and how that data is ingested.
When data comes from large numbers of sources, ingesting also becomes complex. Importing data in the right sequence to ensure it is clean and properly integrated is not easy. One solution is to use data pipelines, and this is where Talend Stitch comes in. It allows users to define data throughput, reliability and latency to ensure data moves as fast as possible.
By integrating into large numbers of SaaS data sources, Stitch also makes it easy for data engineers to control how those pipelines intersect. Where there are dependencies between the different data sets, it can allow data to be merged before it goes further. Where data needs to be cleaned or enhanced, it can bring other controls to bear. Ultimately, all of this is about automating the data ingesting as far as possible.
Stitch and Redshift Console low touch, high speed
Integrating Stitch into the Redshift Console is about more than just smoothing the integration of data. It means that Talend customers will be able to take advantage of the wider Amazon Redshift marketplace. For example, the integration means users can sign-up to import data from any Redshift Partner in just a few clicks.
That sign-up process also creates a connection to the data source. It, in turn, means there is no need for any additional coding of data pipelines. Instead, the connections will be provisioned as part of the process. It means that users can sign-up rather than waiting for data engineering teams to find time to add a new connection.
It also provides a single place to see all connections to all partners and the state of data ingesting. While this will help all users, those doing stream processing should see a significant gain. They will be able to spot problems with connections and data imports quickly. They can even set up alerts for any critical data link to ensure that it doesn’t impact their overall data strategy.
Enterprise Times: What does this mean
Data silos are not going away any time soon. That means that as organisations look to make the most of their data, they need to continue the ETL process and add data cleaning and enhancement. Doing so takes time, effort and no small amount of coding.
Talend is building on the ease of creating data pipelines that Stitch offers by integrating with the Amazon Redshift Console. Now users can extend their data warehouse without having to wait for time from the data engineers. Having the provisioning of the connection links between data sources automated reduces workload and errors. It is a win-win for data engineers and users.
