Split a List<T> into smaller lists of size N.

Well, my stupid header (created years ago when I was still a tweaker) promises an occasional descent into madness, so here you go… use this code at your own risk.

Every so often I find that I have a generic list of something or other, and it makes sense to split the list into smaller lists and then process those “batches” in parallel. This was never a generic method or an extension method, but was a piece of code I copied and pasted all over the show. I made it an extension method now right before checking how other people have done it.

Anyway, if you don’t want to risk running code that I wrote while high on crystal meth a few years ago, you might want to take an answer from this StackOverflow page.

But if you’re feeling adventurous, here’s my code:

using System.Collections.Generic;
using System.Linq;

namespace Demo
{
    public static class Example
    {
        public static List<List<T>> SplitIntoBatchesOfSize<T>(this List<T> list, int count)
        {
            var result = new List<List<T>>();

            if (list.Count > count)
            {
                var listQuery = list.GroupBy(num => list.IndexOf(num) / count);

                for (int i = 0; i < listQuery.Count(); i++)
                {
                    var batch = listQuery.ElementAt(i).ToList();
                    result.Add(batch);
                }
            }
            else result.Add(list);

            return result;
        }
    }
}

Come to think of it, such methods are unnecessary. GroupBy is good enough. You can just use that and then iterate the batches, calling ToList<> or ToArray<> if necessary. For example: (from a custom Task scheduler I shared a long time ago)…

if (taskList.Count > 0)
{
    var batches = taskList.GroupBy(
        task => taskList.IndexOf(task) / maxDegreeOfParallelism);

    foreach (var batch in batches)
        ParallelInvoke(t => base.TryExecuteTask(t), batch.ToArray());
}
Advertisements

About Jerome

I am a senior C# developer in Johannesburg, South Africa. I am also a recovering addict, who spent nearly eight years using methamphetamine. I write on my recovery blog about my lessons learned and sometimes give advice to others who have made similar mistakes, often from my viewpoint as an atheist, and I also write some C# programming articles on my programming blog.
This entry was posted in General, Work and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s