It’s been a whirlwind of a New Year for us at Alteryx (formerly Trifacta) with all our big announcements and updates. On that note, our delivery of new capabilities showcasing our innovation in the data engineering space continues at a frenetic pace. 2022 is here and it’s time to get on board the new 9.x software release train. We have some innovative capabilities on the first release of this train, and we just got started on what is going to be an exciting journey ahead. Let’s dive right in!
General Availability of SQL-based ELT on Snowflake, the Data Cloud
Last year, we launched the private preview of SQL-based ELT on Snowflake with our 8.10 release at AWS re:Invent. We’re now excited to announce this capability is now generally available to all our customers.
With full pushdown on Snowflake, the data transformation logic also known as the data wrangling logic is converted into SQL, and the transformations are directly executed on Snowflake. During transformations, the data stays within Snowflake, resulting in a secure solution that efficiently uses the compute resources in the cloud while delivering a complete data transformation solution within the ELT architecture.
Read our technical blog post to learn more.
Keep your data fresh and updated with ‘Dataset Schema Refresh’
Our newest capability allows you to refresh dataset metadata with the latest source schema. A schema is a skeleton structure representing the logical view of a dataset. This dataset can be a file, a table, or a SQL query in a database. Schemas may apply to relational tables and schematized file formats such as Parquet and Avro.
Dataset schemas often change at different rates, and often columns could be added or removed from your dataset. This could result in incorrect results as headers may not align with the new underlying data. To overcome this undesired outcome, users needed to import new copies of the dataset to keep it in sync. If the same dataset is being used in several flows, users needed to manually replace the dataset on each flow to sync with the new version. This can be highly error-prone.
Now, “Dataset Schema Refresh” enables on-demand updating of your imported dataset schemas to capture changes to columns. This helps reduce the number of duplicate or invalid datasets that are created from the same source. Additionally, this helps reduce the challenges of replacing datasets and retaking samples with changes to dataset schemas. You can initiate multiple refreshes of different datasets concurrently, helping you with increased efficiency. This capability applies to relational schemas, schema files, and delimited files.
Increased flexibility with data transformations using Javascript User-Defined Functions
Today, Designer Cloud supports a wide range of functions to transform your data easily and efficiently. However, there may be scenarios to address specific use cases that need custom functions. To support these use cases, you can now create custom transformations using Javascript User-Defined Functions (UDFs). These are also referred to as custom user-defined functions. These functions can be imported into Designer Cloud to use in your recipes.
Once a Javascript UDF has been created and added by one user, it becomes searchable and available for all users within the familiar Designer Cloud Transformer interface. You can also combine the UDF with other generally available functions in the Transformer formula builder for further expansion.
With the 9.0 release, Javascript UDFs are available in private preview. To enable this capability in your environment, please email [email protected] with details of your use case. We look forward to working with you and enabling your specific use cases.
REST API Connectivity
We now support REST API connections on Designer Cloud that provide a generic interface to relational data. REST APIs have gained popularity as they provide a flexible, lightweight way to integrate applications, and have emerged as a common method to connect endpoints in different architectures.
Here is a high-level refresher on what APIs and REST APIs are. APIs or Application Programming Interfaces are sets of rules that define how applications can connect to, and communicate with each other. REST APIs communicate via HTTP requests to perform standard database functions such as creating, reading, updating, and deleting records within a resource. Using REST API connections within Designer Cloud, you can now create connections to individual endpoints across hundreds of REST-based applications. This is an import-only connection type. REST API connectivity is available in private preview with the 9.0 release.
Click here for more information.
Easy resolution for missing or invalid connections
When a flow containing pre or post-SQL scripts does not have a valid connection associated with it, the scripts will fail. This could happen due to moving flows from one environment to another where connections might not exist. It could also be due to users importing flows containing pre or post-SQL publishing actions using private connections that have not been shared with them.
We now provide users the ability to resolve missing or invalid connections for pre or post-run SQL scripts by alerting the users on these invalid connections on the flow view canvas as well as on the side panel. When the publishing action is opened, you will see that the current connection is invalid and you will have the option to choose another valid database connection that you have access to.
New connectors with Designer 9.0
We continue our journey to help you connect to any data source, enabling additional use cases. With Designer Cloud 9.0, we support the following new connectors:
- Zendesk: A service-first CRM company that builds software designed to improve customer relationships.
- LinkedIn Ads: A paid marketing tool that offers access to LinkedIn social feed through sponsored posts.
You can learn all about our connectivity updates here.
With our recent acquisition by Alteryx, we are forging ahead in our cloud journey for the best of data engineering and analytics. If you have not done already, sign up for a free trial today and join us on this exciting ride. Onwards and upwards!