High speed applications – Parallelism in .NET part 4 – TPL, exceptions & cancellation

Hey there!

Nice to see that you’ve endured this far and managed to get through part 1, 2 and 3.

I was trying to figure out a joke with “Await no more”, but since I generally recommend the usage of await, I don’t want to mislead anyone.

Throughout this series I’ve tried to mix background, overview and details when it comes to parallelism in .NET. I will try to cover the things I’ve come across as interesting and useful, but I will in no way cover everything there is to know about parallelism. Please remember to check out the methods used in the code examples here on MSDN to find out more about their overloads, parameters and options. For example, Task.Factory.StartNew has 16 overloads.

I should also mention that the code written in these examples are not necessarily the cleanest and in other ways most optimized code possible. It demonstrates one or a couple of specific features. I’m sure I’ll write a few posts about clean code in the future.

I was going to write about locking in this post too, but the post would be too big, and I want to get this post published now.

Covered in this post

  • How does async/await work
    We got an introduction in the last post and how much easier it makes our life, but what does it do and how does it do it?
  • Task Parallel Library (TPL)  and Task objects
    What can we do with the Task object?
  • TPL and Exception handling
    How do we handle exceptions in our parallel code (not that our code ever have any errors
  • TPL and Cancellation
    How to cancel in TPL/TAP and how to make your code cancellable

How does async/await work?

Async

The async keyword create a state machine of the method marked with the keyword, where every use of await generates a state of it’s own. A state machine is great if we have a workflow that we need to follow but the steps can not be executed synchronously without interruption. If we would write a state machine of our own, it could look something like this:

When we write asynchronous code with TPL, we can instantly see the benefits of having a state machine – our process flow comes back to a fixed “hub” between calls. Writing a state machine with StartNew / ContinueWith is a possibility but as Microsoft and developers around the world used TPL in .NET 4.0, it did not take much time to realize that this makes asynchronous development hard and makes it take a lot of time. This is why Microsoft invented the compiler features async/await together with .NET framework 4.5 (where the framework is extended with methods to support a compiler generated state machine for asynchronous code).

Await

Each await generates a state in the state machine. All objects that implement the GetAwaiter() method can be awaited. The System.Threading.Tasks.Task type implements GetAwaiter() and returns a TaskAwaiter.
You can create your own awaiters. Look at this example console application where we’ve made and awaitable FileSystemWatcher:

Remember – this is just an example of a custom Awaiter. It will need to be rewritten to buffer events that are triggered while the continuation code is executed and some more but it serves the purpose for demonstration.

The FileSystemWatcherAsync class implements GetAwaiter() which returns a FileSystemWatcherAwaiter that implements:

  • INotifyCompletion.OnCompleted – called to register the continuation to execute when the awaiter is done.
  • IsCompleted which is set to true when the awaiter is done.
  • GetResult() which returns void or the result type, in this case an instance of the FileSystemEvent or null if the watcher is shut down.

You can find more information about FileSystemWatcher on MSDN.

The Task.Yield() is an awaiter that executes the continuation on the previous synchronization context (more about synchronization context in a later post). Basically, this means that the rest of the method runs in a continuation and control is returned to the caller.

ConcurrentBag<T> is a thread safe collection which we will cover in the next post.

Stephen Toub has written a great blog post about custom awaiters called “Await anything” (See reference 1), from where this GetAwaiter() example was inspired.

Executing the program

filesystemwatcherasync

“8-False”, “10-True” and “11-True” are “Executing thread id”-“if it’s a thread pool thread”.

What does the generated state machine code look like? I’m glad you asked. Let’s have a look. I suggest you DO NOT SPEND TOO MUCH TIME GOING THROUGH THIS CODE. At this point, just have a quick look and make sure you can see the state machine created by the compiler. This is just the code for the MakeSomeFileChanges method:

It’s pretty sweet that we do not have to write this code ourselves. The code is decompiled with DotPeek.

It’s worth to mention also that there is some overhead to using async/await in comparison to using Task.Factory.StartNew and Task.ContinueWith but in most cases it’s neglectable. If you are writing an application where you need all the CPU and memory you can, you should consider not using async / await or consider using the thread pool directly (ThreadPool.QueueUserWorkItem).

Task Parallel Library and Task objects

Almost all Task-based Asynchronous Pattern (TAP) and TPL methods return an object of type System.Threading.Tasks.Task. Task.Run, Task.ContinueWith, System.Net.Http.HttpClient.GetAsync, SqlConnection.OpenAsync, FileStream.ReadAsync are just a few examples. They all return a Task object. Task.ContinueWith also gives the continuation delegate a reference to the antecedent task. As you remember from part 3, if the method do not need to return a value, they are typeof(Task), and if they do they are typeof(Task<TType>) where TType is the return type.

Task.Status, Task.IsCompleted, Task.IsCanceled, Task.IsFaulted

If we want to know if a task is completed, we can use the IsCompleted property. Remember how the ContinueWith method gives the completed task as parameter.

IsFaulted will be set if the task or some of it’s continuations has thrown an unhandled exception. See exception handling later in this post.

IsCanceled will be set if the task is canceled. See Cancelation later in this post.

The Status property getter is of type System.Threading.Tasks.TaskStatus. We can use it to determine the status of the task. Example:

For the record, this is only an example to see some different statuses. You will probably never write a Task that consume several seconds of CPU without actually achieving anything but spinning. Running the example displays something like this, depending on the speed of your processor.

Task statuses

The first status is WaitingForActivation, since the task returned from GetStringAsync is the continuation and it is not activated until the Task.Delay task is completed. While the task is executing, status is Running, and finally since the task completes without any errors or a cancellation, the task ends with status RanToCompletion.

Returning already completed tasks with Task.FromResult

The Task-based Asynchronous Pattern states that the caller of a method that returns a Task object should never have to start the task. What if we need to build a cache for this existing asynchronous web api call:

We will need a way to return the result without wasting the resources of actually scheduling a task to return data already available. This is where Task.FromResult<T> comes into play. I’ve used ConcurrentDictionary<TKey, TValue> to store the cache values. ConcurrentDictionary<TKey, TValue> is a thread safe dictionary. More about ConcurrentDictionary<T> in the next post..

The reason we cache the Task<DateTime?> instead of the DateTime is to avoid having to allocate a new task every time something is retrieved from cache. Allocating a new completed task each time we retrieve a date from the cache is slower than allocating it once. I mentioned it in the first post that allocation and disposition of objects are events that can trigger garbage collection (more about garbage collection another time). A garbage collection takes time to complete, partially depending on how much memory the garbage collector has to collect. We do not want to allocate and dispose memory for no good reason at all. We could also store the task used to retrieve the data in the first place but that would not make a good example :).

Returning already completed tasks with Task.Completed

Sometimes we need to return a completed task without returning a value. Let’s pretend we have log framework that is based on an interface called ILog. The interface has one method – LogAsync. If the log implementation can complete synchronously, we return Task.Completed. 

Postponing execution with Task.Delay

When simulating asynchronous operations Task.Delay can be very useful but there are other situations when Task.Delay can be very useful. These are some examples. If you have any of your own, please leave a comment.

  • If we for instance want to call a cache service and if we do not get a reply within 20ms, we want to start calling a database service and whichever task completes first will return the result. For an example of this, see the cancellation section in this post.
  • Retrying for example if we want to write a file to disk but the file is locked by another process, we can delay for a second and try again a couple of times before reporting an error.
  • If we have an API where our clients are very “chatty”, making multiple updates, we can delay for a little while before writing data to the database.

Task Parallel Library and Exception handling

If you’ve ever played around with TPL before async/await and things didn’t go as planed, an exception was thrown. Did you notice that all of a sudden you received an AggregateException instead of the exception you expected? This is because a chain of tasks and continuations can have have several exceptions thrown at the same time.

I think it’s always a good idea to avoid throwing exceptions except for when an unrecoverable error has occurred. In the cancellation part of this post, we cover that the cancellation handling in TPL uses exceptions. This is an exception to avoiding exceptions :).

Exceptions can be caught in several ways.

  1. The usual way, that is not propagated to the thread starting the task.

What you might have noticed is that we rethrow the exception, which of course is something we should avoid when the exception is handled, but the program continues to run. If this was the foreground thread, the application would terminate after the exception was rethrown.

Exception1

2. If we wait for the task, the exception will be thrown in the context of the thread calling Wait.

The rethrown exception is thrown again in task.Wait() and caught in the try/catch.

Exception2

The AggregateException has an InnerExceptions property that contains all exceptions thrown.

3. We can have a continuation take care of the exception

We can create separate continuations for if the task succeeds, if it fails (with an exception) or if it’s canceled. In this example, the continuation is only called if the antecedent task fails. Since we are waiting for the continuation which succeeds, the exception is not thrown when waiting in the main method. The continuation will be completed even if the task succeeds and the continuation is not executed.

Exception3

4. Try/catch in an async method

Again, async/await makes our lives easier.

Exception4

Task Parallel Library and Cancellation handling

To be able to cancel a task you start, you must create a CancellationTokenSource and pass the Token property’s value (a CancellationToken) to the methods you want to be able to cancel. Only the keeper of the CancellationTokenSource is able to signal a cancellation. Most of the methods in TPL have overloads that take CancellationToken parameters.

First we create a CancellationTokenSource and pass the CancellationTokenSource.Token (of type CancellationToken) to the methods we want to be able to cancel. If the method we want to call takes a CancellationToken parameter but we do not want to be able to cancel it, we can pass CancellationToken.Cancel as a parameter.

Pass that token to the method / methods we want to be able to cancel. In the example, we pass it to CancellableMethod and Wait.

As an implementer of a cancel-able method, we should pass the CancellationToken to all methods we call that take a CancellationToken as a parameter.

The metod CancellationToken.ThrowIfCancellationRequested will throw an OperationCanceledException if the token has been canceled. Make sure your code handles the cancellation appropriately and release resources and clean up any started/unfinished work.

There are two ways for our code to subscribe to cancellations.

  1. Try/catch(OperationCanceledException)
  2. CancellationToken.Register()

CancellationToken.Register registers a callback/delegate to be executed if the token is canceled.

The CancellationToken passed to Task.Run will only cancel the scheduling of the task if the task has not been started. As soon as the task has been started, the code within the task delegate will have to check the CancellationToken.

You can not cancel a Task that do not want to be canceled. Cancellations are cooperative.

Now, a final example for this post.

Cancellations and Task.Delay to get cached data from a cache server and a web service.

In short, the program start calling the (simulated) cache service. If we have not received a response within 20 ms, we will call the database to get the value from there. If we get a response with data from the cache service before we get a response from the database service, the database service call is canceled. If the database responds before the cache service, the cache call is canceled. We also test what happens when we cancel everything 10 ms after it’s initiated.

Running it looks like this:

CancellationAndDelay 2

We have used Task.WhenAny to return the first of several tasks to complete. And also TaskCompletionSource<T> to have a virtual task finish when we want it to finish.

Next post!

In the next post we will cover concurrency, locking, interlocking and more TPL.

Please leave a comment if you want to say hi, like this, hate this or have a question.

Take care!
Erik

References

  1. “Await anything” by Stephen Toub

2 thoughts on “High speed applications – Parallelism in .NET part 4 – TPL, exceptions & cancellation

  1. stej

    “In the next post we will cover concurrency, locking, interlocking and more TPL.” Still waiting. I hope you didn’t give up 😉

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *