Hello! It’s our pleasure to share the highlights of the latest 8.9 release from Designer Cloud (formerly Trifacta). New connectors, Custom SQL, Collaboration, and more – we have an exciting lineup of capabilities for you.
Full Execution on Google BigQuery with Custom SQL sources
Aligning with our vision for data modernization, we deliver efficient transformations with intelligent ELT approaches. In this journey, we leverage the processing power of modern cloud data warehouses such as Google BigQuery and execute the transformations within the data warehouse. Our goal since we launched Pushdown Optimization with the 8.3 release is to continuously add support for all transformations and functions. With the 8.9 release, we now support custom SQL queries with Pushdown Optimization for flows where the source and destination are both within BigQuery.
This capability allows you to run jobs on BigQuery that are sourced from imported datasets created with custom SQL. Further, this allows custom SQL entered by users to be included in the pushdown plan for BigQuery Pushdown. The custom SQLs will execute along with your recipe on BigQuery. You can learn all about job execution on BigQuery in this helpful article.
Flexibility with dataset sample sizes for data transformations
Designer Cloud enables the design of the data transformation steps through a “recipe,” based on a sample of your complete dataset. By default, these samples are 10MB in size. Based on customer feedback, we now allow you to change the size of the sample in the transformer interface, with an easy-to-use slider. This flexibility is helpful in cases where complex transformations in your recipe might extend the in-memory sample and slow down the processing within the browser. Adjusting the dataset sample size helps improve the performance of Trifacta in your browser and addresses any low-memory conditions.
The updated size of the samples is saved for each recipe and you can set different sample sizes for each recipe. If you are collaborating with others on your recipe, they can leverage the same updated setting.
Click here to learn more.
Better control with Data Pipelines
You now have additional options and better control with your data pipelines. Designer Cloud allows you to create tasks to delete existing files or folders. This helps in removing files that were generated as part of intermediate steps in the execution of your pipeline, which are no longer needed.
You can learn more from this article feature on our community site, detailing the steps to achieve this functionality.
Easier collaboration on flows
Designer Cloud makes it easy to collaborate on data engineering tasks and we pride ourselves on our collaborative capabilities. With the 8.9 release, we now have a clear collaboration icon indicating that a particular flow has been shared with others. You can find the list of these collaborators when you hover over the icon. This not only enables teamwork and collaboration but also leverages the collective intelligence of the organization to make data tasks better and more efficient.
Click here to learn more about sharing flows and collaboration.
Additional connectors for your data sources
As part of our endeavor to help you connect to a wide range of data sources, we continue to extend our configurable JDBC framework for new connectors. With the 8.9 release, we have 3 new connectors for your use cases.
- Google Ads: With this new connector, you will be able to reach out to audiences interested in your products and services through an online advertising platform.
- Microsoft Advertising Connections: You can now leverage this service that provides pay-per-click advertising on Bing, Yahoo, and other search engines.
- Presto: Using this high-performance, distributed SQL engine for running interactive analytic queries for any scale of data, you can query various data sources such as Hadoop, MySQL, MongoDB, and Teradata.
We believe access to data sources is critical for businesses to gain a competitive advantage, and these connections will help you gain that benefit. You can learn all about our connectivity updates here.
We hope you have as much fun working with these new capabilities from Designer Cloud as much as we had in building them and bringing them to you. Till next time, onwards and upwards!