What’s New in 9.5
We’re excited to share our latest capabilities from the Designer Cloud and Google Cloud Dataprep 9.5 release. As always, there’s a wide range of new features to discuss:
Introducing Designer Cloud!
For AWS and Azure users, Designer Cloud 9.5 officially introduces a new product name and an updated product look. For more information about this change, be sure to read our full blog post on the rebrand.
Snowflake Enhancements
Our 9.5 update brings several enhancements to Designer Cloud’s Snowflake integration:
- Snowflake Pushdown Support for Sampling: For AWS users, pushdown processing can now be used when creating samples from data stored in Snowflake tables. This makes the data preparation process even more seamless by greatly decreasing the time needed to create a data sample based on full scans of a dataset. With 9.5, this pushdown support is available for all sampling techniques other than Clustering and Stratified samples.
- Snowflake Pushdown Support for S3 Data Sources: Users working with S3 data files can now process their workflows up to 12x faster by leveraging Snowflake pushdown. Previously, Snowflake pushdown processing was only available when working with data sourced from Snowflake tables. With 9.5, Snowflake pushdown processing can now be used with data sourced from S3 buckets that is being written to Snowflake tables. This allows for ELT from S3 to Snowflake, and can greatly increase the run time efficiency of workflows for those using S3 data sources.
- Upsert Support for Snowflake Publishing: Upserts can now be used when publishing to Snowflake, allowing users to add individual rows without processing and replacing entire tables.
- Snowflake JDBC Connector (Private Preview): We’ve implemented a common JDBC framework so that users can connect to Snowflake wherever it is located (either in AWS, Azure, or another cloud deployment). This allows users to ingest data from Snowflake on any cloud, as well as making it possible to write data to Snowflake on Azure (using pushdown processing!). This feature is still in private preview. Reach out to your Customer Success Manager or Account Executive to get access.
Individual Asset Transfer Between Users
Previously, in cases such as a user leaving a company, admins could bulk transfer all assets from one user to another via API.
In 9.5, the ability to transfer asset ownership has been expanded from admins to include individual users. It’s also now possible to transfer the ownership of individual assets between users (as opposed to the bulk transfer of all assets) – including all 1st class objects such as flows, connections, imported datasets, plans, macros, and UDFs. This allows for helpful use cases, such as transferring developed assets to a centralized operations account for scheduling.
We’ve also made asset transfer accessible via dropdowns in the UI, making it possible for non admins to transfer assets without having to write code in an API. To top it off, we’ve added a table to record the transfer history of assets, giving admins a record of who has owned which assets over time.
Edit Recipes With Datagrid Disabled
In 9.5, users now have the option to launch the transformer page (recipe view) with the datagrid disabled. This allows users to edit recipe steps without waiting for data samples to load, allowing for faster edits when moving in and out of individual recipes. This is particularly useful when working with large datasets/recipes, or in cases where users find themselves in environments that have poor internet connectivity, causing slower sample loading times.
New Flow Parameter Type – Selector
In 9.5, we’ve added a new flow parameter type – Selector. The Selector Flow Parameter allows users to define a parameter based on an enumerated list of values with a single selection option. This can be used to define an override key, where an overridden value applies to all references of the parameter within a flow.
Flow Import and API Connection Mapping
When importing flows to a new environment or workspace, users can now specify connections, allowing for more plug-and-play usability. Using this feature, users can simply change their connections, and datasets will be replaced properly without any additional steps.
Refresh Excel, PDF, and Google Sheets Files
In 9.5, dataset refresh has been further expanded to include Excel files, PDFs, and Google Sheets. This builds on existing dataset refresh support for relational, delimited, schematized, and JSON files. When the underlying schema for a supported dataset changes, dataset refresh allows users to upload fresh data and refresh their datasets without the need to create a new dataset object and replace it in the flow. Dataset refreshes can be used to address schema changes, or to add or remove columns of data from a dataset. This makes your datasets more durable, reusable objects and helps to avoid workspace clutter and versioning issues.
Speeding Up the Job History Page
To speed up performance on the job history page, admins now have the option to change the default number of days displayed (180 days / 120 days / 60 days). Changing this default can reduce page rendering time by up to 20%, providing even faster performance.
Enable OAuth for Sharepoint
For enhanced security, users can now leverage OAuth 2.0 connectivity to access Sharepoint lists.
Google Cloud Dataprep Specific Releases:
In 9.5, we have some additional updates specific to Google Cloud Dataprep:
- Enable Sort Transform: Dataprep users can now sort dataset samples in the transformer grid. Samples can be sorted based on columns in ascending or descending order or based on the order of rows when the dataset was created. This was an existing feature in Designer Cloud on AWS that has been brought to Google Cloud Dataprep.
- Enable Service Accounts for In-VPC Batch Jobs and BigQuery Execution: For Dataprep users, Service accounts can now be used to execute transformation jobs within your VPC and within BigQuery. This enhanced security measure removes calls to the Trifacta VPC for credentials and reduces timeouts on longer-running jobs.
New connectors with 9.5 Release
We continue our journey to help you connect to any data source, enabling additional use cases. With our 9.5 release, we support the following new early-preview/read-only connectors:
- Workday
- Google Calendar
- QuickBooks
For more information, see Early Preview Connection Types. You can learn all about our connectivity updates here.
If you haven’t done it already, it’s a great time to sign up for a free trial. Join us today on our journey to the cloud.