Understanding the Lifecycle of a Workflow
Contents
In my last post I built a simple sequential workflow. I introduced all the details needed to use Visual Studio 2008, a few WF activities, and the workflow runtime to build and run a simple expense report approval workflow. In this post I want to step back from the mechanics of building workflows and look at the workflow runtime and the lifecycle of workflows when they are managed by the workflow runtime. In other words, I want to investigate how the workflow runtime manages a workflow instance from the time it is first created to the time that it is complete and is no longer needed. Understanding what goes on inside the workflow runtime is important when creating durable and highly available workflow enabled applications.
You can download the code for this post at the link below.
The WorkflowRuntime class shown in Figure 1 is primarily responsible for managing the lifecycle of workflow instances. The key to understanding how this class manages workflow lifecycles is to understand all of its events and when each one is raised. The WorkflowRuntime class contains 14 events in total. Some of these events are designed to inform the host process about events occurring within the workflow runtime. However, most of these events represent lifecycle changes occurring within each workflow instance being managed.
Figure 1 – Class Diagram of the WorkflowRuntime class
The first three events shown in Figure 1 are known as workflow runtime events. They inform the host about occurrences within the runtime itself. These events are Started, Stopped, and ServicesExceptionNothandled. Figure 2 provides a brief description of these workflow runtime events.
|
Name
|
Description
|
|
|
If any workflow service encounters an exception that it cannot entirely handle then it should call the WorkflowRuntimeService.RaiseServicesExceptionNotHandledEvent function which will raise this event and send the exception information to the host.
|
|
|
Indicates that the workflow runtime engine has started. The workflow runtime raises the started event after it has validated core service configuration, started all the services that have been added to it and set the IsStarted flag to true.
|
|
|
Indicates that the workflow runtime engine has stopped. The workflow runtime raises the stopped event after it has stopped all the services that have been added to it and set the IsStarted flag to false.
|
Figure 2 – Workflow Runtime Events
What is a WF Runtime Service?
A Workflow Foundation Service is a piece of custom code that can be added to the workflow runtime giving it additional capabilities as it manages the execution of workflows. Examples of WF services that add capabilities are Persistence Services and Tracking Services. WF comes with a Persistence Service and a Tracking service which both use SQL Server as their underlying data store.
WF Services can also be used to override default behaviors of the WF runtime. An example of a WF service that changes default behavior is the Manual Scheduler service which allows the WF runtime to execute workflows on the current thread as opposed to the thread pool.
By default these services are not added to the workflow runtime - they must be explicitly added by the developer.
It is important to note that the events shown in Figure 2 are not triggered by a running instance of a workflow. These events are designed to notify the host about the state of the runtime itself. The Started and Stopped events fire when the StartRuntime function and the StopRuntime function of the WorkflowRuntime class are called respectively. A workflow service can raise the ServicesExceptionNotHandled event when it encounters an exception that it cannot handle during its execution.
The other 11 events are known as workflow instance events. Figure 3 provides a brief description of all the workflow instance events. These events will be raised when a workflow instance changes state or when other important events occur while a workflow is in the Running state. It is interesting to note that a running WorkflowRuntime object is itself a state machine that keeps track of running workflows by tracking their status (or state). Figure 4 is a state diagram that shows the various states that a workflow instance may pass through during its lifetime. A workflow instance can be in one of five states: Created, Running, Suspended, Completed, and Terminated. It is a common mistake to look at the class diagram of Figure 2 and come to the conclusion that there is a state for each workflow instance event. This is incorrect. There are only 5 states that make up the lifetime of a workflow. 5 of the workflow instance events will be raised when a workflow enters a new state. Specifically, WorkflowCreated, WorkflowStarted (for the Running state), WorkflowSuspended, WorkflowTerminated, and WorkflowCompleted will be raised when an instance enters the Created, Running, Suspended, Terminated, or Completed states respectively. The remaining events inform the host about important changes that occur while the workflow is in the Running state.
|
Name
|
Description
|
|
WorkflowAborted
|
Aborting a workflow is only valid when a persistence service is used. When a workflow is aborted the WorkflowRuntime engine throws away the current in memory instance. Application code can abort a workflow by calling WorkflowInstance.Abort. WorkflowInstance.Resume can then be used to restart the workflow from the last persistence point. This should be done only under extreme situations where all work done since the last persistence point needs to be discarded.
|
|
WorkflowCompleted
|
The workflow runtime engine raises the WorkflowCompleted event after the workflow completes but before it is removed from memory. This event can be used to send output parameters back to the host via the WorkflowCompletedEventArgs class.
|
|
WorkflowCreated
|
The WorkflowCreated event is raised by the workflow runtime engine after a workflow instance has been completely instantiated but before the instance is started via a call to the workflow runtime’s Start function.
|
|
WorkflowIdled
|
A workflow that is not executing any of its activities because it is waiting for an external event, an external message, or a delay activity is said to be idle. The WorkflowIdled event is raised to inform the host of this condition.
|
|
WorkflowLoaded
|
If a persistence service has been added to the workflow runtime engine then the WorkflowLoaded event is raised after the persistence service has restored the workflow instance to memory but before the workflow runtime engine begins to execute any activities.
|
|
WorkflowPersisted
|
If a persistence service has been added to the workflow runtime engine then the WorkflowPeristed event is raised after the state of the workflow instance has been saved by the persistence service.
|
|
WorkflowResumed
|
A workflow instance that has been previously suspended may be resumed and begin executing again at the point it was suspended. The WorkflowResumed event occurs after the workflow instance has been scheduled to begin executing but before any activities begin to execute.
|
|
WorkflowStarted
|
The WorkflowStarted event indicates the entry of the workflow instance into the Running state. The WorkflowStarted event is raised when the workflow instance’s root activity is scheduled for execution.
|
|
WorkflowSuspended
|
The WorkflowSuspend event occurs when a workflow instance is suspended and enters the Suspended state. A workflow can be suspended by the host via a call to the WorkflowRuntime.Suspend function, by a Suspend activity explicitly placed within the workflow definition, or implicitly by the workflow runtime itself. A suspended workflow can be resumed at the point it was suspended.
|
|
WorkflowTerminated
|
The WorkflowTerminated event occurs when a workflow instance enters the Terminated state. A terminated workflow is cleared from memory. If a persistence service is used then the persistence service is notified that the instance has been terminated. The SqlWorkflowPersistenceService will delete all state information for terminated workflows. A workflow can be terminated by the host via a call to the WorkflowInstance.Terminate function, by a Terminate activity explicitly placed within the workflow definition, or by the workflow runtime engine when an unhandled exception occurs. The terminate event is raised after a workflow is terminated but before the instance is removed from memory.
|
|
WorkflowUnloaded
|
If a persistence service has been added to the workflow runtime engine then the WorkflowUnloaded event is raised after the persistence service has successfully saved a workflow instance’s state but before the instance is removed from memory.
|
Figure 3 – Workflow Instance Events
Figure 4 – Workflow Runtime States and Events
The host process can be made aware of the occurrence of these events by registering delegates with the workflow runtime. Figure 5 shows a revised version of the Main function from the ConsoleHost project created in my last post. In this code a delegate has been registered for each of the workflow runtime events and for each of the workflow instance events. Figure 6 is the implementation of the delegate for the workflow idled event. Notice that for workflow instance events the delegate gets passed the workflow instance ID of the workflow instance that raised the event. In the code download for this post all the other events have been setup in a similar fashion.
static void Main(string[] args)
{
// Create a new Workflow Runtime
WorkflowRuntime wr = new WorkflowRuntime();
// Workflow Runtime events
wr.ServicesExceptionNotHandled += OnServicesExceptionNotHandled;
wr.Started += OnStarted;
wr.Stopped += OnStopped;
// Workflow Instance Events
wr.WorkflowAborted += OnWorkflowAborted;
wr.WorkflowCreated += OnWorkflowCreated;
wr.WorkflowCompleted += OnWorkflowCompletion;
wr.WorkflowIdled += OnWorkflowIdled;
wr.WorkflowLoaded += OnWorkflowLoaded;
wr.WorkflowPersisted += OnWorkflowPersisted;
wr.WorkflowResumed += OnWorkflowResumed;
wr.WorkflowStarted += OnWorkflowStarted;
wr.WorkflowSuspended += OnWorkflowSuspended;
wr.WorkflowTerminated += OnWorkflowTerminated;
wr.WorkflowUnloaded += OnWorkflowUnloaded;
// create and setup the workflow parameter
ExpenseReport expense = new ExpenseReport();
expense.Amount = 2500;
expense.Employee = "Keith Pijanowski";
expense.Title = "Platform Strategy Advisor";
// create the dictionary object to hold the parameter
Dictionary<string, object> parameters = new Dictionary<string, object>();
// this is a key/value pair
parameters.Add("Expense", expense);
// pass the type of the workflow to be created and any parameters
WorkflowInstance instance =wr.CreateWorkflow(
typeof(SequentialExpenseReportApproval),
parameters);
instance.Start();
Console.ReadLine();
}
Figure 5 – Runtime Events and Instance events
static void OnWorkflowIdled(object sender, WorkflowEventArgs e)
{
Console.WriteLine("Workflow Idled. Instance ID: " + e.WorkflowInstance.InstanceId.ToString());
}
Figure 6 – Sample Delegate
When the sequential workflow from my previous post is instrumented as shown in Figure 5 and Figure 6 then the output shown in Figure 7 is produced. Notice that we are notified when the workflow instance is created, started and completed. Also notice that this workflow does not need to idle in order to wait for external events, external messages or a delay activity. Consequently there is no need to unload and persist this workflow.
Figure 7 – Output from the Expense Report workflow
In this section I want to simulate a long running workflow and observe the workflow instance events as the workflow executes. To simulate a long running workflow I will add a Delay activity to the expense report approval workflow which I created in my last post. Using the Delay activity is easy. Merely drop it onto the sequential workflow designer in the desired location. Figure 8 shows the expense report workflow after a Delay activity has been added to it. I want the delay activity to always execute so I did not place it inside one of the IfElseBranch activities. The Delay activity can be configured using the properties dialog as shown in Figure 9. Here I have named the activity delayTest and I have configured it to delay execution of the workflow for 1 second.
Figure 8 - Adding a Delay Activity
Figure 9 - Properties for the Delay Activity
When the modified workflow is run the output shown in Figure 10 is produced. Notice that the Idled event is raised indicating the workflow is no longer running but waiting for the Delay activity’s timeout duration to expire. We would have observed the same behavior if instead of a delay activity we had placed a web service Receive activity or a HandleExternalEvent activity. With respect to lifecycle behavior all these activities result in the same thing. That is the workflow going idle and waiting. I used a Delay activity here for simplicity.
Investigating a timeline that shows the lifecycle of a workflow from creation to completion will clarify the relationship of a workflow’s state and the workflow instance events that are raised by the workflow runtime.
Figure 10 - Output messages showing an Idled event
Figure 11 shows a timeline of a long running workflow from the time it is created until the time that it completes. This timeline is of a workflow that is never aborted, suspended, terminated or persisted. Once the workflow reaches a point where an external event is needed it will raise the Idled event indicating that it is no longer executing. By default the workflow runtime will keep all workflows that are waiting for external events, external messages, or delay activities in memory. Not only does this waste system resources but it also leaves the workflow in a vulnerable situation. If the system or the process in which the workflow runtime is located crashes then the data that the workflow instance contains and the work that it has done are lost.
|
The workflow instance has been created via a call to CreateWorkflow(); but, the workflow has not been started by calling StartWorkflow().
|
The workflow instance has just entered the Running state.
|
This part of the timeline represents a loop. For a long running workflow one Idled event will be raised by the runtime every time the workflow pauses to wait for an event, message, or a Delay activity.
Note: This diagram shows the events in the order that they are raised to the host. The Idled event always occurs before the Persisted and Unloaded events.
|
The Completed event is raised just prior to the workflow instance being removed from memory. This gives the workflow a chance to pass output parameters back to the host if needed.
|
Figure 11 – Timeline for an in-memory lifecycle
Clearly what is needed to solve the problem described in the last section is a way to persist a workflow once it has gone idle. Once a workflow is persisted the workflow can be unloaded from memory.
Fortunately, the Workflow Foundation provides a mechanism that allows workflow instances to be persisted and unloaded when they are not executing. Persisted workflows can be loaded when they need to execute again. This is done by adding a persistence service to the workflow runtime. A persistence service is responsible for saving, and retrieving workflow instance state from some form of non-volatile storage. The runtime can still listen for messages on behalf of the persisted workflow. When a message arrives for a workflow that has been persisted the workflow runtime can then load the workflow into memory and begin executing activities at the point in which the workflow went idle.
Figure 12 is a revised timeline of the long running workflow shown in Figure 11. This timeline shows the workflow being persisted, unloaded and loaded. It also shows some additional workflow instance events that the workflow runtime will raise in order to inform the host when a workflow has been persisted, unloaded and loaded. What is compelling about the timeline shown in Figure 12 is that only workflows with executing activities are in memory. All other workflows are persisted and unloaded.
|
The workflow instance has been created via a call to CreateWorkflow(); but, the workflow has not been started by calling StartWorkflow().
|
The workflow instance has been started via a call to StartWorkflow(). At this point the workflow is in the Running state.
|
This part of the timeline represents a loop. For a long running workflow one Idled event will be raised by the runtime every time the workflow pauses to wait for an event, message, or a Delay activity.
Note: This diagram shows the events in the order that they are raised to the host. The Idled event always occurs before the Persisted and Unloaded events.
|
The persistence service is called to remove the workflow instance from the persistence store. Then the Completed event is raised before the workflow instance is removed from memory. This gives the workflow a chance to pass output parameters back to the host if needed.
|
Figure 12 – Timeline for a workflow lifecycle when a persistence service is used
The workflow runtime determines when a persistence service is called. Workflows are persisted at locations known as persistence points. Below is a list of persistence points that will cause the workflow runtime to call a persistence service if one has been added to the workflow runtime.
· When a workflow goes idle.
· Before a workflow instance completes. (This removes any instance data from the underlying persistence store.)
· Before a workflow instance terminates. (This removes any instance data from the underlying persistence store.)
· On the completion of activities that are marked with the PersistOnCloseAttribute attribute.
· When a developer calls WorkflowInstance.Unload or WorkflowInstance.TryUnload.
The first three persistence points provide default durability for all workflows executing within a workflow runtime that has been equipped with a persistence service. The last two persistence points provide added flexibility for developers that would like to add additional persistence points to their workflows.
All of the out of the box WF activities that utilize transactions use the PersistOnCloseAttribute attribute. Additionally, all custom activities that use transactions should also use this attribute. Finally, any other custom activity may use this attribute if the design of the activity dictates that the state of the workflow instance should be persisted once the activity is done executing.
Workflow Foundation provides the classes and the database scripts that allow Sql Server 2000, Sql Server 2005, Sql Express, and MSDE to be used as a persistence store. However it is important to note that Workflow Foundation also provides the WorkflowPersistence base class and the IPendingWork interface which can be used by developers to create custom persistence services that use any storage mechanism. Therefore it is possible to use Oracle, DB2 or MySQL for persistence.
In this post I introduced the workflow runtime and its default “in-memory” lifecycle. I also presented a lifecycle that saves and unloads idle workflows. This is the lifecycle used when a persistence service is added to the workflow runtime. Persistence Services allow the workflow runtime to use system resources more efficiently when the workflows it is managing are long running. Persistence services also provide a more durable environment for workflows to run.
In my next post I will show how to setup a Sql Server database that can be used by the Sql Server Persistence service. I will also show how to add the Sql Persistence service to the workflow runtime. Finally I will show the various configuration options of this service.