Autopilot Equals Analytic Freedom - Inspire 2017

Spend less time refreshing and maintaining your analytic apps and more time analyzing, exploring, and expanding your insights. Learn how to automate jobs and easily adjust them on the fly when things change. Find out how to sustain day-to-day business processes and eliminate repetitive tasks so you can free up your team to tackle those analytics projects you've been dreaming.



Video Transcription


Julie Mendelson:
Thank you all for being here. My name is Julie Mendelson. I'm a product donor at Alteryx and I work on our enterprise server team. I'm primarily responsible for taking all of the product vision and roadmap items for the server offering. I work closely with the software development team to bring new features and enhancements to life. I started with Alteryx about seven months ago. Since I started, I've done work in the scheduler, which we'll be talking about today. It's really exciting to be here. This is my first Inspire. What I really love about being here is getting to interact directly with many of you. I love getting to hear how you guys are using the platform, what's working well, maybe what you'd like to see improve in the future going forward. What's amazed me the most as I've been introduced to Alteryx when I started with the company was the sheer number of types of business problems that can be solved. For every problem that can be solved, there is usually more than one way to go about creating your solution.

What I wanted to talk with you all about today is taking it one step further, thinking about how you're currently using the platform and what are ways that it can really be optimized to increase your efficiencies. One of the first things that I think about when I think of trying to increase efficiency and make life easier with Alteryx are the many methods and tool sets that we have available to automate your analytic processes. My definition of automation is taking something like a task or a process and just making it happen automatically. When it comes to repetitive in manual analytic tasks, we all know that automation will help free up your and your business colleague's time to focus on more important projects or things that you have on your plate. What some of you might not be familiar with are some of the different ways that you can go about automating your processes with Alteryx, and so that's what we're going to talk about today.

With Alteryx, you no longer have to wake up in the middle of the night to manually run a workflow that you know someone in an opposite timezone needs. You also don't have to recreate the same or similar report over and over again for request that people make from you for that data. There's no one way to think about automation when it comes to Alteryx. We're going to over just a few examples and a few options on how you might be able to think about that and apply that to what you guys are doing today. Due to the nature of this conversation, I do have to say that if there are any forward-looking statements made about what might potentially happen in the future, things can and often do change. Don't go making purchasing decisions based off about information or anything like that. It's not going to the stepping stone.

Here's what we're going to talk about today. We'll first cover reasons why you want to think about automation. Then we're going to jump in to some specific ways that you can go about doing so. As our customers first get introduced to Alteryx as a platform, we tend to see it where maybe one or two people discover the power and the value of using designer. Then naturally as more and more users start using the platform, it begins to make sense to stop creating your workflows in silos and to start actually collaborating and sharing and getting those insights out to your users. That's where the Alteryx server is a great tool for you to be able to do things like publishing your apps to the gallery. It also comes with some tools that help with automating and help it make you do that in an easy way right out of the box, so things like the scheduler or getting access to the server API and command line interfaces.

The point that I want to make here is that regardless of where you are whether you're a server customer or if you are an organization with some designer users, there are tools that you can take advantage of today.

Why automate? We often hear from our customers about the challenges of keeping up with the demand for data insights within their business organization. It's really important to be able to scale your analytic processes to go as far as possible. The concept of automating seems like an obvious thing to do but without understanding some of the key reasons, you might miss some opportunities within the platform. Some of the reasons that you might want to think about automating things is ensuring your data relevance. If you can remove the need for manual intervention to update and refresh data sets, then you can get as close to real-time data processing as possible.

You can leverage off-hours processing cycles. I think I saw somewhere that we have based on usage data the average amount of time that it takes a workflow to run is something like 24 seconds, but that can range. I've heard of things like several hours for a job to run. If you can be strategic about when you're going to have those jobs run, you will improve your efficiencies and you can be smarter about when you're actually having those kickoff.

All of this is aimed at delivering your insights to your consumers so that they can make their data-driven decisions more quickly. For you to be able to focus less on those repetitive tasks and more on those other analytic projects that maybe you've put on the back burner because you've been in the weeds of doing the same tasks over and over again.

There are few different ways that I like to think about how you might approach looking at what you're currently doing and where you can automate. That would be within your workflows thinking about potentially creating linkages between workflows and then taking it a step back and thinking about Alteryx as a service itself. Maybe it's one link in the chain of your overall process and there are other services that you're using outside of Alteryx and it makes sense to think about automating between services. Within your workflows, you can look at things like the scheduler to be able to schedule your jobs to run automatically. We have a concept of workflow events that we're going to look at where you can do some cool things with the run command tool and some email automation. Between workflows, we're going to talk about some macros that are available to you that can help you link workflows together to kick off automatically. Then when you're looking at Alteryx as a service, we'll talk about the server API and our command line interface.

Are there people in the room who are using the scheduler already? Good. The scheduler for those of you who aren't familiar is a powerful tool that lets you take [pre-built 07:52] workflows. You can set them to run on a custom schedule. You can schedule them to run one time in the future or on a recurring basis. It's a great way to just step those workflows to run without having you manually do so. It's really the easiest way in my opinion to get started with automating with perhaps the largest value add right off for that.

When it comes to scheduling, we have two different offerings that we have that you can take advantage of. One is an add-on that you can get as a designer user. It's the designer with desktop automation. It's a scheduler add-on that lets you schedule frequencies of one-time schedules, every minute, hour, days, weeks, months and you can even create custom schedules. If you know you have a job that you want to run every other month, on the third week of those months, on Tuesdays of those weeks, you can create those types of schedules.

The scheduler with the add-on is very simple to be able to start using within designer directly. We have this create schedule icon. As you're creating your workflows, you can easily click this icon or go to options and select schedule workflow. From here you'll set the location of where you want to schedule those jobs. For this offering, your schedules are going to be running locally on the machine that you're using. You're always going to select My Computer here. The flow will be contained in designer and you'll actually view and manage your schedules in designer as well.

There are couple of different options of where you can have a copy of your workflow run here. You can have it run as a copy stored directly in the scheduler DB. In this case, it's great if you know that the workflow is complete. You just want that schedule to reliably run. You can have it save a copy in the DB. You know nothing is going to happen. It won't accidentally be moved. The schedule will always have access to that. Sometimes you may need some flexibility and you may want to make updates to that schedule and not have to worry about recreate, sorry, updates that workflow and not have to worry about recreating schedules. The option of running the schedule from its original location on disk will allow you to do that. If you have your workflow saves locally, people can make changes. The scheduler will pick up the most up-to-date copy of that workflow and run from there.

The other offering that we have around scheduling is the server scheduler. With the 11.0 Alteryx Release, we came with a new scheduling experience where you can actually manage those schedules directly in the gallery interface. It's a browser-based scheduling experience. You're able to take advantage of increased reliability by running your jobs in a server environment. You're not susceptible to maybe someone who is scheduling on their laptop to closing their machine, shutting down the computer. This is really a way that you know that the schedules can run at any time of day and it's not susceptible to someone going out of town or having it run on someone's computer and all the various ways that that computer can turn off.

You're also getting an increased level of data governance with this. You're able to take advantage of things like our workflow credential feature, which we'll talk a little bit about. We can set schedules with set credentials to run with those and ensure that the jobs are only accessing the data that they're supposed to be accessing.

The experience of using the server scheduler is very similar from designer. You select the same icon or go to options and create schedule for schedule workflow. In this case if it hasn't already been added, you can add your gallery as a location. When you select that as you create the schedule, it will actually automatically launch an instance of your gallery in your browser so you can create the schedule in that interface from there.

We also have the ability with the server scheduler to schedule directly in the gallery itself. As you are looking at your workflows, you will see a new schedule button where you can come in to that same interface and create the same ones recurring or custom frequencies that the designer with automation scheduling feature supports. In this case, your users are going to be viewing and managing their schedules directly in the gallery. They'll have a schedule's tab that they can access and will be able to see their own schedules and manage them from here. They can view their schedules on the schedule name when it was created, the next run time, the last time it ran. Like I mentioned, it supports setting workflow credentials.

The gallery has a feature where you can actually set workflow credentialing options. As the gallery administrator, you can have it so that all of the workflows use the default credentials, which is really just that server system account. You can do things like require your users to enter credential to run jobs, which is the way of making sure that they're only accessing the data that they're supposed to be. In that case if they are required to enter their credentials, the scheduler does allow you to enter your credentials and set it so that each time the job runs the schedule will run with the rates that of credentials.

Another feature that was added with the new scheduling experience is a view in the administration panel for the server administrator. We added a new jobs page where they have a viewing to all of their users' schedules. It's a great way to get a bird's eye view of how your users are scheduling. This is where they can come and edit, delete schedules and manage actually enabling the feature. They can turn it on here and no longer do your users need things like a controller token or the machine name to actually get started with scheduling.

There's also an exciting add-on here where we have the status tab. The status tab lets you view jobs that are running and queued in the server. You can actually delete jobs that may be stuck. If you have a job that's running for a prolonged period of time and it's bogging down your queue, you can actually remove that job and open up the queue so that other jobs can process. If it's a schedule job, it's not going to remove the schedule itself. It will just remove that one instance of the schedule run. Scheduling, you can see it that there's a lot of power here to be able to start creating these schedules, have them kick off automatically, have them run at any time of night and remove the need to manually kick off the jobs yourselves.

Is there anyone who is familiar with workflow events? Workflow events, this is a feature that actually lives in the designer. In the designer, there is a workflow configuration panel. In that panel, there is a tab to create events. You can create events that can kick off at various times or different criteria sets; before your workflow runs, after it runs, after it runs with errors or after it runs without errors. There are two tools in the designer that these events work with. The tools are embedded within this tab, depending on which event you're creating, so that would be the send email tool and the run command tool.

The send email tool is a way for you to be able to automate things like email notifications. If you schedule a job that runs weekly and you want to know that it's kicked off, that it ran, that it happened without having to login and look for that yourself, you can use the event command or the event send email to set up that email and have it automatically trigger much like you're actually placing the email tool in the workflow itself. You don't actually have to drag the tool in. You can create that event and have it kicked off based on when you want to know. If it's after the workflow runs without errors, you can set that up here. The other thing that's nice about this is you can automate sending out reports that get generated with your workflows. If there's an output or report that you know that this user needs after it generates weekly, you can set it up to automatically email that individual with the contents.

The run command tool is a developer tool that is really powerful. It can be used in a lot of different ways. The concept is essentially to allow you to be able to use to... It's like interacting with your command line interface within an Alteryx workflow. You can do things like run external command line programs. You can use batch files. You can run different scripts. You can actually use cURL to transfer data from websites almost like an input tool itself. It's another way that you can actually set up an event to trigger to do things automatically for you. One really cool example I've heard is using the Alteryx service command line interface to actually pass in an add-to-queue request. In this scenario, they set this up so that the event would run after workflow A kicks off after that completes. Then it passes in the add-to-queue request to kick off another workflow. It's one way to start thinking about having a workflow trigger based off of your first workflow run.

Since we're starting to go down that road of talking about linking workflows between one another, we wanted to jump in to talking about the crew macros. The crew macros are available on our community. They're actually created by an Alteryx employee. I know that there are a lot of people who've been contributing to them actively like some of our aces. There are lot of really cool tools available within the macro package that you can download. This is actually a great example of how you can take the Alteryx capabilities and build out your own tools in macros to do things that may be specifically you aren't seeing how they can be done.

A set of the crew macros that I wanted to talk about are the runner macros. The runner macros essentially allow you to run workflows; I think that's why they're called runner macros. We have the runner macro, the conditional runner macro, and then the list runner macro. The runner macro is going to be the simplest one that you would start out with. All you need to do is configure the path to the workflow that you wanted to run. It has a set of two outputs, so either a success or a failure. Based on that, you can build out a process flow whether your workflow runs successfully. You can go off and do other things from there. If it failed, you can have... In this example, I have custom emails set up so that I'm notified based on the success or failure of this macro run.

If I want to start linking additional workflows to run with this, I can make use of the conditional runner macro. This will allow me to... If workflow A successfully runs, I can hook it in to my next conditional runner macro that has an input and set it to run workflow B and so forth. If no input data is received, so if that success is not true in this example, the workflow that set in my next macro will not run.

The list runner macro is a similar concept but it's going to be useful when you have something like a directory of workflows that you just want to run one after the other. In this case, it will have a single output after it runs through those workflows. It will essentially dump logging information into a single field. We recommend using this with the log parser macro to be able to take that information and easily put it into a data table. There are success and failure criteria in here that you can see, but it's another way to be able to kick off multiple workflow runs.

Taking a step further and thinking about Alteryx as a service. Maybe there are things that happened before you're processing an Alteryx or after the processing is done. You want to look at potentially integrating Alteryx with other processes that you guys are already doing. We do have the Alteryx server API. This is a way... It's a REST HTTP web interface where you can programmatically interact with the gallery processes. You can do things like poll analytic app questions. You can queue job executions. The best examples that I see of this are customers who want to take a pre-built analytic app, which essentially lets their users do things like putting parameters to run a custom report. You can embed that directly into an internal website where they can run it directly from. Maybe you have an internal internet site that people are just used to going to. They don't really know what Alteryx is but they can make use of this reporting data.

To get started with the API, it's really simple. Your server administrator would just need to enable API access for you. Then authentication is supported with OAuth and a key and secret will be required. We do have a really cool interactive API documentation site where you can actually take that key and that secret and come to a really simple webinar phase and make sample API calls here. You can view the different parameters that it accepts and actual responses. It's a great way to get started with familiarizing yourself with the API and to check out that documentation here. Just as I mentioned some examples of what you can do with this, you can embed your apps into web pages, mobile apps, really anything that can consume REST API. You can incorporate Alteryx directly into that so that people can make use of the insights that they can get from there.

The server command line is something that lives within AlteryxService.exe or the service that we call it. This is something that actually performs a lot of different operations, including job orchestration and storing information about your workflows and workflow execution. It also supports a command line interface. The add-to-queue piece of this is usually what I see people use most often, when they want to use other external services like Windows task scheduler or control M to actually orchestrate running jobs as a part of other process that they're already doing. There's actually really great community posts by Steve Ahlgren. He is one of our lead server developers that talks a lot about the command line interface. If you're interested in learning more it, I definitely recommend checking that out.

Bringing it all full circle. We talked about three different ways to look at maybe how you may be using Alteryx and what are some approaches that you can look at to improve some of your efficiencies. Looking within your workflow: Are you already scheduling jobs? If not, something that you may want to look into doing, making use of those workflow events to automate your email notifications or to incorporate that run command tool to do other things that you may already be doing manually like file cleanup or even beginning to link your workflows together through that run command to add-to-queue. Creating the links between those workflows: Checking out the crew macro set and those runner macros. Then thinking about Alteryx as a service itself: Are there ways that you can tightly integrate it with things that you already have running with user flows that people are already used to doing going directly to internal sites without having to make them go somewhere else to run these analytic apps?

These are actually just a few different ideas. I know that there is a lot more out there. I definitely encourage you all to check out the community, post questions there. There's a lot of information to be found there. If there are things that you know that you would like to see, it's also a great place to add feature request. As a member of the product team, we look really closely at those. When you see other feature request, you can vote those up and it gets more attention from us that way. We also have online help. Our download page includes a lot of information about our latest releases. If you haven't used Alteryx yet and you're new, you can actually get a free trial download really easily at alteryx.com.

We wanted to plug that we love getting feedback, so definitely encourage you all to please fill out the survey. If you've downloaded the Inspire app, you can go in there and open up your schedule. Find this session and definitely give some feedback on what you've heard today.

That would be it. I do want to open it up for any questions, but also my contact information is up here so you can feel free to reach out to me directly but otherwise if anyone has questions.

Speaker:
Just a second here. For everybody who is going to ask questions, if you would just wait for the mic to make sure everybody can enjoy your question that'd be great. I'll be running around. You could also queue up at the mic here as well.

Crowd question:
On the runner macros, those are macros that we call from another workflow, correct?

Julie Mendelson:
Yes.

Crowd question:
Either you have to have a workflow that you're manually firing to take advantage of them or that is scheduled within the scheduler system?

Julie Mendelson:
Yes.

Crowd question:
On the scheduler system, I didn't catch it but you don't have predecessor functionality, right?

Julie Mendelson:
Meaning like?

Crowd question:
Meaning if I have two jobs that run and gather data for a third job, that third job is scheduled predicated on those other two successfully finishing?

Julie Mendelson:
We don't currently have that built in to the scheduler functionality yet. That's correct.

Crowd question:
[inaudible 29:32].

Julie Mendelson:
The list runner, yes. Within the scheduler itself, we don't have that logic.

Crowd question:
The designer with desktop automation add-on tool that you spoke about, is it in Alteryx 11.0 or is it the previous version as well?

Julie Mendelson:
That's available in the previous version, so if you're using a 10-5. I don't recall when that became available, but it's available with older designer versions.

Crowd question:
If you're looking in designer ones and you have to use my computer, can you use your own network drives or does it have to be on like your C drive?

Julie Mendelson:
You can use your own network drive. As long as you have access to those drives, the scheduler will be able to point to workflows there.

Crowd question:
On the scheduling with the date function on either the gallery or otherwise, is there any way to tell Alteryx when there's an FME date that's not necessarily the end of the month but it's based on the financial month end?

Julie Mendelson:
We do get requests around having things like fiscal calendar options for scheduling. It's not something that we have built in yet. We might be able to talk and look one on one and see if there are any work around that we could do to make that work.

Crowd question:
How would you recommend stringing to other multiple applications? We have questions in different workflows. I don't think the list runner automatically brings up the questions that are being asked when the next workflow takes off.

Julie Mendelson:
I didn't cover it in this session but you can chain analytic apps. That answer is based on the first one. You can conditionally populate with the next set of questions. I know that there are community posts on that or someone in the solution center would probably be able to walk through an example of chaining apps together. We do have support for that functionality.

Crowd question:
Another question I have, is there a way to download the workflows, the frequency and who has posted it from the scheduler probably into CSV or Excel or any format?

Julie Mendelson:
Download the workflow from another user?

Crowd question:
Yes, I mean from scheduler i you want to have download all the workflows. What are the workflows that you're running on the scheduler if I want a list of all that?

Julie Mendelson:
Are you using the designer-based scheduling?

Crowd question:
Yes.

Julie Mendelson:
You can see all of the results if you're... If you're using the server designer-based scheduling, that data will be something you can access through MongoDB. If your uses are all scheduling locally, I might have to actually follow up with you on that question because I'm not sure if there's a way to compile the results and be able to view all of that together if that's what you're asking. If not, we can talk and make sure I answer it.

Crowd question:
Can you use the desktop scheduler to publish to gallery?

Julie Mendelson:
Do you have an example of how you would want that to work?

Crowd question:
We don't have server currently. We were thinking of using the public gallery possibly to publish a few workflows, but we didn't know if that was possible.

Julie Mendelson:
I believe you could publish to the public gallery, but it would be separate from your scheduling. You won't be able to make use of the gallery-based scheduling in that case. You can still schedule them to run on the desktops, but publishing would be a separate element to that.

Crowd question:
A follow-on on her question. We use the scheduler on the designer not the server. The thing we take advantage of is the command line capability. We don't really use your scheduler because we need complex conditionals like last business day minus two things like that predecessors. We bought the scheduler but in that way we fire most everything from a command line. When you do that, the command line in instance will capture the output view that you would see if you are manually running it, which has all your air messages and warning messages and such. We just redirect that to a log file so we can scan those logs, see if it succeeded, make decisions based on that and so on. If we move to the server right now, your server functionality isn't sophisticated as we've built. We probably would still be using server for other things but using our own scheduler. Is that still going to work? I would imagine it would because it's just PowerShell on the server.

Julie Mendelson:
Are you asking if you can still use the command line scheduling the way you are now?

Crowd question:
Yes.

Julie Mendelson:
That should still work for you.

Crowd question:
If I understood her question if she fires the workflows that she has scheduled from a command line batch that they create dynamically, then you'll capture the output window, which I think is what she was looking for.

Julie Mendelson:
That could be a good way to look at it. We can certainly talk about specifically what you guys are doing.

Thank you all so much. Feel free to come find me if you have other questions.

^Top

Experience the
Power of Alteryx
For Yourself.

Get Started