Concurrent asynchronous processing of all the files in a directory with support for cooperative pausing, in C#

The other day Stephen Toub of MSFT published a blog on cooperative pausing async methods. It’s an awesome post, and a much better way of pausing in async methods than what I was doing (which was a loop where I checked a flag and called Task.Delay). He created a type called a PauseToken, which works much the same way as a CancellationToken. I have thus updated all the code that supports pausing async methods in my application to use this.

I now present some code that uses this, as well as running an async task on all the files in the directory. There are several places in the application that do this, so I’ve just randomly picked one of them.

Again, the source, for this and a whole bunch of other stuff, is here:

I have a custom SaveImageDialog, that allows image conversion as follows: (This is not a real save dialog, it’s my own creation; it’s not production code, and not really important to the main topic of this post. I just want to illustrate how to get there in the application.)

Normally, the dialog, which I got to by clicking the Save As button on my image viewer, looks like this:


Clicking the file type dropdown changes what it will do as follows:


That is, it filtered the thumbnail view by PNG files. (There aren’t any in the directory.) It then hid the controls on the dialog that are specific to JPG format, and made the button that will convert all images in the directory visible, as well as the checkbox to enable deleting the source image(s) after conversion.

The code that runs if you convert all the files in the directory is this:

  1. private async void ApplyToDirectoryButtonAsync_Click(object sender, EventArgs e)
  2. {
  3.     DialogResult = DialogResult.None;
  4.     Hide();
  5.     ImageViewer.Instance.Hide();
  7.     var degreeOfParallelism = Environment.ProcessorCount * 2;
  8.     var extension = string.IsNullOrEmpty(newExtension) ? ImageType.Extension() : newExtension;
  10.     // Process all image files, that are not already the destination image type, in the directory
  11.     var files = (await FindUtility.FindImageFilesAsync(IO.CurrentDirectory, null)).Where(f =>
  12.         !Path.GetExtension(f).EqualsIgnoreCultureAndCase(extension)).ToArray();
  14.     var dialog = new ProgressDialog { Count = files.Length, CanPause = true };
  15.     dialog.Text = string.Format(“Converting {0} images”, dialog.Count);
  16.     dialog.InitialProgressDescription = “Converting images”;
  17.     dialog.Show(Browser.Instance);
  19.     cancellationTokenSource = new CancellationTokenSource();
  20.     try
  21.     {
  22.         await files.ForEachAsync(degreeOfParallelism, async f =>
  23.         {
  24.             if (dialog.Paused)
  25.                 await dialog.PauseTokenSource.Token.WaitWhilePausedAsync();
  27.             cancellationTokenSource.Token.ThrowIfCancellationRequested();
  29.             var source = new ImageManager(f).ToBitmap();
  30.             try
  31.             {
  32.                 var newFilename = Path.ChangeExtension(f, extension);
  33.                 var isIcon = IO.IsIcon(newFilename);
  35.                 if (isIcon)
  36.                 {
  37.                     var singleImage = new Image[] { source };
  38.                     await singleImage.SaveIconFileAsync(newFilename);
  39.                 }
  40.                 else if (IO.IsJpeg(newFilename))
  41.                     await (source as Image).SaveJpegAsync(newFilename, JpegQuality);
  42.                 else
  43.                     await (source as Image).GetFormatAndSaveAsync(newFilename);
  45.                 if (deleteOriginal)
  46.                     IO.Delete(f);
  48.                 /* Preload this image’s thumbnail into the cache. Thus all thumbnails
  49.                  * should already be available after the conversion is complete.  */
  50.                 await ThumbnailCache.LoadAndCacheThumbnailAsync(newFilename, cancellationTokenSource.Token);
  52.                 await InvokeOnUIthread(() =>
  53.                 {
  54.                     if (dialog.Cancel)
  55.                     {
  56.                         cancellationTokenSource.Cancel();
  57.                     }
  58.                     else
  59.                         dialog.UpdateProgress(newFilename);
  60.                 });
  61.             }
  62.             catch (FileNotFoundException) { }
  63.             catch (IOException) { }
  64.             catch (UnauthorizedAccessException) { }
  65.             finally
  66.             {
  67.                 source.Dispose();
  68.             }
  69.         });
  70.     }
  71.     catch (OperationCanceledException) { dialog.Close(); }
  73.     await Browser.Instance.RefreshDirectoryAsync(true);
  74. }

As usual, it calls several extension methods defined elsewhere in the solution, as well as common methods that I call frequently in Romy.Core, which is a class library of my commonly used classes and extension methods. My progress dialog exposes the PauseTokenSource type of the blog post mentioned at the start of this post, to enable waiting asynchronously on it, and the Pause button simply sets the token to paused or unpaused. It also uses a standard CancellationToken for cancellation.

ForeachAsync is another extension method from an older blog post by Stephen Toub (I use a lot of his sample code). The code to support saving to icon format is something I forgot to remove; it’s not meant to be there. ImageManager is is my own bastard creation, to enable managing GDI+ references, such that they can be passed around to async Tasks without worrying about the references going out of scope, leading to the fabulous “An unexpected error has occurred in GDI+”, or the even more useful “Object is in use elsewhere”, and the various methods to save images asynchronously are extension methods defined in Romy.Core.

The progress dialog’s design is very simple, so I won’t paste any of its code here. It has a Count property, initialized upfront, and each time its public UpdateProgress method is called with a file name, it displays the file name and updates its progress, which will be 100% when it reaches Count.

It also perhaps worth noting that the code called by the dialog’s Cancel button first unpauses the dialog if it is paused, so that it is possible to cancel while paused. I found this was an oversight when I first wrote code like this. It may seem obvious that this is required now, but when writing the actual code that supports both cancellation as well as pause, it is very easy to miss that it needs to support immediate cancellation from the paused state.

All of this happens in a ThreadPool thread, which of course creates a slight problem: You can’t update the user interface from another thread. Hence all of my forms inherit the InvokeOnUiThread method, which simply does the call using a captured UI-thread-synchronized synchronization context. Thus the code to update the progress dialog calls this method, which effectively switches to the UI-thread.


About Jerome

I am a senior C# developer in Johannesburg, South Africa. I am also a recovering addict, who spent nearly eight years using methamphetamine. I write on my recovery blog about my lessons learned and sometimes give advice to others who have made similar mistakes, often from my viewpoint as an atheist, and I also write some C# programming articles on my programming blog.
This entry was posted in Programming and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s