Updating a table using a join in SQL Server (UPDATE FROM)

Yes, this is most probably trivial to everybody except me. For whatever reason, I simply don’t remember this syntax. Every time I have to do this, I Google it and usually end up on this solution. You can go there if you like – I’ve added nothing of value other than putting it in terms of the structure of the tables I’m working with (although I’ve changed field and table names). This way, if I read this post myself a few times, maybe I’ll remember this shit.

Let’s say you have an Widget table and a Customer table, where Widget has a CustomerID foreign key. Both have a foreign key to a Contact table, which was recently added. So I changed the code that generates (inserts) widgets to always use the configured contact on the customer, if there is one as it is a nullable foreign key. (i.e. FK added using WITH NOCHECK ADD CONSTRAINT.) That’s great, but now I need to update the existing widgets to have the contact ID as well.

(The business rule I have is simply that widgets must have the default customer contact when created. If you change the contact on a customer, it does not get applied retroactively on widgets. However, after adding these new fields, I do need to fix the existing widgets. I could leave them null but it makes sense to update all of them to use the correct customer contact as set up today.)

The solution is to use the UPDATE FROM syntax, which I so happen to always forget. I don’t know why that is. Maybe it’s old age approaching; maybe it is the price I pay for my years of drug abuse. (I’m four years clean at the time of writing this, by the way. Meth is a helluva drug.) I don’t know, but it seems that my brain cells dedicated to remembering SQL are unreliable. It’s easy though, and for the tables I have described, the update query would look like so:

UPDATE w
SET 
    w.ContactID = c.ContactID
FROM 
    Widget w
    JOIN Customer c ON c.ID = w.CustomerID
WHERE 
    w.ContactID IS NULL AND c.ContactID IS NOT NULL
Advertisements
Posted in Database, SQL | Tagged , , | Leave a comment

Split a List<T> into smaller lists of size N.

Well, my stupid header (created years ago when I was still a tweaker) promises an occasional descent into madness, so here you go… use this code at your own risk.

Every so often I find that I have a generic list of something or other, and it makes sense to split the list into smaller lists and then process those “batches” in parallel. This was never a generic method or an extension method, but was a piece of code I copied and pasted all over the show. I made it an extension method now right before checking how other people have done it.

Anyway, if you don’t want to risk running code that I wrote while high on crystal meth a few years ago, you might want to take an answer from this StackOverflow page.

But if you’re feeling adventurous, here’s my code:

using System.Collections.Generic;
using System.Linq;

namespace Demo
{
    public static class Example
    {
        public static List<List<T>> SplitIntoBatchesOfSize<T>(this List<T> list, int count)
        {
            var result = new List<List<T>>();

            if (list.Count > count)
            {
                var listQuery = list.GroupBy(num => list.IndexOf(num) / count);

                for (int i = 0; i < listQuery.Count(); i++)
                {
                    var batch = listQuery.ElementAt(i).ToList();
                    result.Add(batch);
                }
            }
            else result.Add(list);

            return result;
        }
    }
}

Come to think of it, such methods are unnecessary. GroupBy is good enough. You can just use that and then iterate the batches, calling ToList<> or ToArray<> if necessary. For example: (from a custom Task scheduler I shared a long time ago)…

if (taskList.Count > 0)
{
    var batches = taskList.GroupBy(
        task => taskList.IndexOf(task) / maxDegreeOfParallelism);

    foreach (var batch in batches)
        ParallelInvoke(t => base.TryExecuteTask(t), batch.ToArray());
}
Posted in General, Work | Tagged , | Leave a comment

Emails I hate receiving

I often wonder if I am the only one who hates receiving emails like this… Names have been omitted.

Subject: Problem with [product brand name].

Hi asshole

I trust this email finds you well.

Please assist with the below. OR…
Kindly find the attached.

Kind regards
Twat

[Nested thread of somewhere around seventy emails with correspondence between sender and various others. Somewhere down there is the context they’re thinking of (or maybe not), and somewhere else there may be clarification on what the brand name, that refers to multiple software projects in our system, might refer to. Or maybe not at all.]

Let’s analyse their email, shall we?

  1. Subject does not contain relevant context. The brand name could refer to any one of several products within the software that we have developed. The client’s client might thus be one of several different kinds of users, since site users and integrating users are different, even within the same product. The subject literally tells me nothing. They could just as well have left it blank.
  2. “I trust this email finds you well.” What does that even mean? It’s not just verbose, since at least verbosity still tries to convey something pertinent, albeit in too many words. This literally adds nothing but annoying false civility.
  3. “Please assist with the below.” Really? So not only does the subject tell me nothing, but the message content doesn’t convey anything either, other than telling me I need to scroll down and try to make sense of their previous correspondence backwards. Even if I skip to the bottom, all I find is something where the context is assumed because it’s from a client they have a relationship with who uses one product (and they know what it is but neglect to mention it in the email), who wrote some generic message about the system not working, or referred to an error that they have had before.
  4. “Kindly find the attached.” Ooh, it’s a spread sheet. Gee, thanks. It’s very nice!
  5. “Kind regards”. What’s the difference between “kind regards” and “regards”? Again, “kind” is redundant. “Regards” would suffice perfectly.

Thank goodness I am normally a CC recipient rather than the primary one. But still, it wasn’t always that way. It’s difficult to figure out how to reply to those kinds of emails… When I used to receive them directly, I’d always reply with a generic response asking for context, but they didn’t learn anything, and later I’d get another useless email from the same person.

Of course, you know that in half an hour or so, you’ll get another email with…

Subject: RE: Problem with [product brand name].

Hi asshole,

Any feedback on the below?

Kind regards
Twat

Posted in General, Work | Leave a comment

How to initialize an enum from a string or int value in c#

I didn’t know you could do this, but I tried and it seems fine so it’s worth sharing.

I’m working in a large code base, where my input might come from a web form, or an imported file, or even be passed in to a WCF service. I didn’t write this originally, but I need to bolster the validation of one of the input fields, which involves doing different validations depending on the type of the thing passed in. By “type”, I refer not to the .NET type but the kind of thing it is. For example, if it were an account it could be a cheque or savings account; if it were an ID it could be a local or international passport or a local ID document or birth certificate number, and so on. We already have the type, but it wasn’t being used.

My problem is, the type is sometimes an integer (hard-coded by my predecessor – for example hard-coded in the markup of a combobox control and it is the ID of the thing type) and other times a string. While I could change all the code everywhere to look up the ID in the database, that would involve a lot of work, such as changing the code that hard-codes values to bind to the database properly… and so on.

What I need then, is to be able to initialize an enum in all of those places, using either:

  1. A string literal.
  2. An integer that represents the value of the enum.

It turns out to be easy. For an integer, I can cast directly to the enum type (in other words, an explicit conversion), and for a string, I use the Enum.Parse method.

Here is a simplified code example that demonstrates how easy it is. This code is only intended to demonstrate how easy the initialization of the enum from a correct value or string is. Note that in my real code, I am not blindly casting a user input field into an enum. I’m using the enum that represents a type of item, which I know to be valid, to validate a second field.

using System;

namespace Demo
{
    enum Thing : int
    {
        Nothing = 0,
        Gizmo = 1,
        Doodad = 2,
        Thingamajig = 3,
        Whatchamacallit = 4
    }

    public static class Example
    {
        public static bool ValidateStringThing(string thing)
        {
            Thing value = (Thing)Enum.Parse(typeof(Thing), thing);
            return ValidateThing(value);
        }

        public static bool ValidateIntThing(int thing)
        {
            Thing value = (Thing)thing;
            return ValidateThing(value);
        }

        private static bool ValidateThing(Thing thing)
        {
            switch (thing)
            {
                case Thing.Nothing:
                    return false;
                case Thing.Gizmo:
                case Thing.Doodad:
                case Thing.Thingamajig:
                case Thing.Whatchamacallit:
                    return true;
                default:
                    return false;
            }
        }
    }
}
Posted in Programming | Tagged , | Leave a comment

New methods not showing up in WCF Test Client? Are you using IIS Express and did you move your code to a new directory without deleting the old one?

I gave the answer to my problem of today in the title…

I work on multiple projects. But we also recently changed our source control provider from SVN to Git. If I haven’t touched a code-base for months, I don’t bother to clone the new repository until I need to work on it. That’s what happened today.

I cloned the repository for this particular solution, and started working on it. I added new code to a few projects, including one new WCF method. Tested everything else, then when I wanted to test the WCF service changes, two things went wrong:

  1. Pressing F5 launched my browser rather than the WCF Test Client.
  2. After I figured out how to fix the first issue, the newly added method was missing.

To fix the first issue, all I had to do was right-click the source file containing my service contract and choose Set As Start Page.

Then off to Google and StackOverflow, but I found nothing that solved my problem.

In my case, what happened was IIS Express was still pointing to the old contract, in the source folder that I used before switching to Git. I confirmed it by trying to move that directory and saw the usual Windows error message telling me that directory was in use.

To fix the issue, I had to manually edit the applicationhost.config file found in:
C:\Users\[My username]\Documents\IISExpress\config

Fixing it was just a matter of searching for my project name, and right away I saw the paths pointing to the wrong directory.

I hope this helps somebody who has a similar problem…

Posted in Programming | Tagged | Leave a comment

Sample code to send email via gmail in c# (updated)

I wrote a post recently with some code to send email via gmail in c#. The other day, I needed to update it to allow for multiple attachments. Rather than update the original post, I’ve written this new one… (The rest is copied and pasted, altered for the new code.)


For some time, I’ve had a block of code that I copied and pasted all over the place, to send emails via gmail. It didn’t add an attachment, but I needed to do so today. So rather than copy and paste it yet again, I’ve created a simple static class that can be reused.

Note that:

  1. I hard-code my gmail address (the username) and my password as constants. (Changed to invalid values here.) You’ll need to change them to something valid.
  2. When debugging, I don’t use gmail. I use an application called Papercut, and there is a similar one called smtp4dev I think… to mock a local SMTP server. It then “receives” the email.
  3. I don’t do anything if there is an exception when trying to send the email. That’s because the applications using this code typically send emails from their exception handlers when there are errors. You might want to do something different.
  4. I’m not going to explain how to enable SMTP on a gmail account. I trust that anybody using this can figure that out for themselves.

This is really simple code, and I consider it junior developer level. I’m just putting it here, to make it easier to reuse. Please don’t ask me any questions about it. It does the basic job I need it to do and that’s all.

I’ll paste an example that uses it below…

using System.Collections.Generic;
using System.Net.Mail;

namespace Example
{
    public static class EmailHandler
    {
        private const string UserName = "name@gmail.com";
        private const string Password = "YourPasswordHere";

        public static void SendEmail(IEnumerable<string> toAddresses, IEnumerable<string> copyAddresses, IEnumerable<string> bccAddresses, 
            string subject, string from, string body, IEnumerable<Attachment> attachments = null, bool isHtml = true)
        {
            using (MailMessage mailMessage = new MailMessage())
            {
                foreach (var to in toAddresses)
                    mailMessage.To.Add(to);

                foreach (var copy in copyAddresses)
                    mailMessage.CC.Add(copy);

                foreach (var copy in bccAddresses)
                    mailMessage.Bcc.Add(copy);

                mailMessage.Subject = subject;

                mailMessage.From = new MailAddress(from);
                mailMessage.Body = body;
                mailMessage.IsBodyHtml = isHtml;

                if (attachments != null)
                {
                    foreach (var attachment in attachments)
                        mailMessage.Attachments.Add(attachment);
                }

                try
                {
#if DEBUG
                    using (SmtpClient smtp = new SmtpClient("127.0.0.1"))
                    {
                        smtp.EnableSsl = false;
                        smtp.Port = 25;
                        smtp.Send(mailMessage);
                    }
#else
                            using (SmtpClient smtp = new SmtpClient("smtp.gmail.com", 587)
                            {
                                Credentials = new System.Net.NetworkCredential(UserName, Password),
                                EnableSsl = true
                            })
                            {
                                smtp.Send(mailMessage);
                            }
#endif
                }
                catch { }
            }
        }
    }
}

In my example that uses the above, I’m calling it from ASP.Net. The attachment I pass to it is a PDF rendered from an SSRS report, using the ReportViewer control. You shouldn’t really need this code, except for the part that initializes the attachment, because it may not be obvious how to do that. The attachment can be initialized via a file name or stream, and needs a MIME type. I use the constructor that takes a stream, a file name, and the content type name. Bizarrely there is also one that takes a content type itself and a stream, without a file name. That one is kind of shitty.

The update… My example code still uses an SSRS report that gets rendered to PDF, but the site also allows the user to upload files. I’m using DevExpress controls, and as each file is uploaded to the site, I put the contents in the session. The code to clear those variables, which is not shown, is called after sending or cancelling sending the email… Just so you can see what’s in the session, the handler for each uploaded file looks like this:

protected void ASPxUploadControl1_FileUploadComplete(object sender, DevExpress.Web.ASPxUploadControl.FileUploadCompleteEventArgs e)
{
    e.CallbackData = e.UploadedFile.FileName;

    Dictionary<string, byte[]> attachments = Session["Attachments"] == null ? new Dictionary<string, byte[]>() : (Dictionary<string, byte[]>)Session["Attachments"];

    attachments[e.UploadedFile.FileName] = e.UploadedFile.FileBytes;

    Session["Attachments"] = attachments;
}

Here is the updated code that sends the email. Note that the handling of attachments is a little messy. For each attachment, I use a Stream to initialize the attachment object, but all those streams need to remain in scope while the email is being sent. Then they are all closed afterwards. So I couldn’t use a using statement, as I normally would…

So this example looks more complicated than it actually is. Code to use the wrapper to send a simple email would be trivial. I trust that most developers, even junior ones, should be able to use my wrapper without reading the example code.

protected void cbSendEmail_Callback(object source, DevExpress.Web.ASPxCallback.CallbackEventArgs e)
{
    string[] toEmails = ((string)flEmail.GetNestedControlValueByFieldName("ToEmail")).Replace(";", ",").Split(',');

    if (toEmails.Length > 0)
    {
        string subject = (string)flEmail.GetNestedControlValueByFieldName("Subject");

        string body = htmlMessage.Html;

        int id = (int)Session["InvoiceID"];
        ReportParameter p = new ReportParameter("ClientInvoiceID", id.ToString());

        reportViewer.ServerReport.SetParameters(new ReportParameter[] { p });
        byte[] data = reportViewer.ServerReport.Render("pdf");

        string invoiceNumber = (string)queryHandler.OpenQuery("SELECT InvoiceNumber FROM ClientInvoice WHERE ID = @ID", new Dictionary<string, object> { { "@ID", id } }).Rows[0]["InvoiceNumber"];

        Dictionary<string, byte[]> attachments = Session["Attachments"] == null ? new Dictionary<string, byte[]>() : (Dictionary<string, byte[]>)Session["Attachments"];

        using (MemoryStream stream = new MemoryStream(data))
        {
            System.Net.Mail.Attachment attachment = new System.Net.Mail.Attachment(stream, string.Format("Invoice {0}.pdf", invoiceNumber), System.Net.Mime.MediaTypeNames.Application.Pdf);

            var emailAttachments = new List<System.Net.Mail.Attachment>();

            emailAttachments.Add(attachment);

            List<Stream> streams = new List<Stream>();

            try
            {
                foreach (var kvp in attachments)
                {
                    streams.Add(new MemoryStream(kvp.Value));
                    emailAttachments.Add(new System.Net.Mail.Attachment(streams[streams.Count - 1], kvp.Key));
                }

                EmailHandler.SendEmail(toEmails, Enumerable.Empty<string>(), new string[] { "redacted@someplace.co.za" }, subject, "redacted@someplace.co.za", body, emailAttachments);
            }
            finally
            {
                foreach (var memStream in streams)
                    stream.Close();
            }
        }
    }
}
Posted in Programming | Tagged , | Leave a comment

Sample code to send email via gmail in c#

For some time, I’ve had a block of code that I copied and pasted all over the place, to send emails via gmail. It didn’t add an attachment, but I needed to do so today. So rather than copy and paste it yet again, I’ve created a simple static class that can be reused.

Note that:

  1. I hard-code my gmail address (the username) and my password as constants. (Changed to invalid values here.) You’ll need to change them to something valid.
  2. When debugging, I don’t use gmail. I use an application called Papercut, and there is a similar one called smtp4dev I think… to mock a local SMTP server. It then “receives” the email.
  3. I don’t do anything if there is an exception when trying to send the email. That’s because the applications using this code typically send emails from their exception handlers when there are errors. You might want to do something different.
  4. I’m not going to explain how to enable SMTP on a gmail account. I trust that anybody using this can figure that out for themselves.

This is really simple code, and I consider it junior developer level. I’m just putting it here, to make it easier to reuse. Please don’t ask me any questions about it. It does the basic job I need it to do and that’s all.

I’ll paste an example that uses it below…

using System.Collections.Generic;
using System.Net.Mail;

namespace Example
{
    public static class EmailHandler
    {
        private const string UserName = "name@gmail.com";
        private const string Password = "password123%";

        public static void SendEmail(IEnumerable<string> toAddresses, IEnumerable<string> copyAddresses, string subject, string from, string body, System.Net.Mail.Attachment attachment = null, bool isHtml = true)
        {
            using (MailMessage mailMessage = new MailMessage())
            {
                foreach (var to in toAddresses)
                    mailMessage.To.Add(to);

                foreach (var copy in copyAddresses)
                    mailMessage.CC.Add(copy);

                mailMessage.Subject = subject;

                mailMessage.From = new MailAddress(from);
                mailMessage.Body = body;
                mailMessage.IsBodyHtml = isHtml;

                if (attachment != null)
                    mailMessage.Attachments.Add(attachment);

                try
                {
#if DEBUG
                    using (SmtpClient smtp = new SmtpClient("127.0.0.1"))
                    {
                        smtp.EnableSsl = false;
                        smtp.Port = 25;
                        smtp.Send(mailMessage);
                    }
#else
                            using (SmtpClient smtp = new SmtpClient("smtp.gmail.com", 587)
                            {
                                Credentials = new System.Net.NetworkCredential(UserName, Password),
                                EnableSsl = true
                            })
                            {
                                smtp.Send(mailMessage);
                            }
#endif
                }
                catch { }
            }
        }
    }
}

In my example that uses the above, I’m calling it from ASP.Net. The attachment I pass to it is a PDF rendered from an SSRS report, using the ReportViewer control. You shouldn’t really need this code, except for the part that initializes the attachment, because it may not be obvious how to do that. The attachment can be initialized via a file name or stream, and needs a MIME type. I use the constructor that takes a stream, a file name, and the content type name. Bizarrely there is also one that takes a content type itself and a stream, without a file name. That one is kind of shitty.

        protected void cbSendEmail_Callback(object source, DevExpress.Web.ASPxCallback.CallbackEventArgs e)
        {
            string[] toEmails = ((string)flEmail.GetNestedControlValueByFieldName("ToEmail")).Replace(";", ",").Split(',');

            if (toEmails.Length > 0)
            {
                string subject = (string)flEmail.GetNestedControlValueByFieldName("Subject");

                string body = htmlMessage.Html;

                int id = (int)Session["InvoiceID"];
                ReportParameter p = new ReportParameter("InvoiceID", id.ToString());

                reportViewer.ServerReport.SetParameters(new ReportParameter[] { p });
                byte[] data = reportViewer.ServerReport.Render("pdf");

                using (MemoryStream stream = new MemoryStream(data))
                {
                    System.Net.Mail.Attachment attachment = new System.Net.Mail.Attachment(stream, "Invoice.pdf", System.Net.Mime.MediaTypeNames.Application.Pdf);
                    EmailHandler.SendEmail(toEmails, Enumerable.Empty<string>(), subject, "redacted@redacted.co.za", body, attachment);
                }
            }
        }
Posted in Programming | Tagged , | Leave a comment

Beware of calling Contains in the Where predicate on IQueryable<T> in LINQ to SQL if your collection has more than 2100 items

Recently I ran into a problem I’d never seen before – the 2100 parameter limit in the SQL that’s generated when you call Contains in the IQueryable<T>.Where predicate in LINQ to SQL. To make matters worse, this happened when filtering data in the Selecting event of a DevExpress LinqServerModeDataSource… which suppresses the error, resulting in an empty result set. (I only saw the error after stepping over the call, and then inspecting the result set in the debugger.)

You don’t hit the problem when your collection contains exactly 2100 items… it happens with less items than that. The 2100 number refers to a limit that’s hit in the SQL code that’s generated. So even as many as 2000 items in your collection will probably trigger the issue. I haven’t experimented or dug deep enough to figure out exactly what the limit is. But it doesn’t matter. What matters is that calling Contains when you don’t know how many items are in your collection, is asking for trouble.

The solution to this is simple – it was finding the problem that took so long. If I have a collection that contains thousands of items, then I split it up into smaller collections, filtering the data each time and adding the result sets to a List<T>, which I then convert back to an IQueryable<T> at the end.

The code below is an example selecting event for the data source. I’ve put that rather than only the relevant code, so that it’s a better example of how one can filter using such an event. It’s using a strongly typed Linq to SQL class for a view. I then have an ASPxGridView control that’s bound to the LinqServerModeDataSource.

You can ignore the first bit… which gets the lines from a memo in the drop down edit window of a drop down edit control, and puts any valid ones into a List<long>. The idea here is that users can filter the data by pasting in a column of account numbers they copied from Excel. Just in case they have bad data, I then ignore anything that isn’t numeric. (Of course this introduced the problem. Allowing users to paste in an unlimited number of rows was probably a bad idea, although in theory it solved a problem they had. But until this point, we never needed to filter a list of many thousands of items using Contains.)

Also, my way of splitting the list into smaller lists is to use IEnumerable<T>GroupBy with index/count – you could achieve the same thing with less code, using Skip and Take in a loop. I like writing it this way because I find the resulting code easier to read.

Note also that the unfortunate part of this is it will put the whole data set in memory. For that reason, I don’t use this approach if there are less than 1000 items in the list, and in my real code (this one is simplified and has the view name changed), this is the last step of filtering the data.

Aside: The closest I came to finding a solution online was this question on StackOverflow, which doesn’t have an accepted answer, but ironically the top one that I tried moved the call to Contains such that it actually threw the Exception that had been suppressed all the time in my code. Hopefully what comes across in this post is not so much exactly how I implemented a solution for this problem, but that the solution is to split the call to Contains between smaller collections, add all the results to a List<T>, and convert back to the IQueryable<T> at the end. If you understand the principle, you don’t even need to use my code…

Edit: The important code to take note of is really from the block comment downwards. You should be able to see all of it when dragging the scrollbar around at the bottom of this code block. That block comment got its indenting a little messed up when I unindented the code and copied it here… (And since this code was copy-pasted to a temporary class that I deleted, it’s too much trouble to fix on here. Sorry.)

protected void dsFutureDatedTransactions_Selecting(object sender, DevExpress.Data.Linq.LinqServerModeDataSourceSelectEventArgs e)
{
    FutureDatedTransactionsDataContext Context = new FutureDatedTransactionsDataContext();
    IQueryable<FutureDatedTransactions> Data = Context.FutureDatedTransactionss;

    // Account numbers
    ASPxMemo memo = (ASPxMemo)ddeAccountNumbers.FindControl("memoAccountNumbers");

    if (!string.IsNullOrEmpty(memo.Text))
    {
        List<long> accountNumbers = new List<long>();

        string[] numbers = memo.Text.Split(new char[] { '\n' }, StringSplitOptions.RemoveEmptyEntries);

        foreach (string accountText in numbers)
        {
            long accountNumber = 0;

            if (long.TryParse(accountText, out accountNumber))
            {
                accountNumbers.Add(accountNumber);
            }
        }

        if (accountNumbers.Count > 0)
        {
            /* If there are too many elements in the list, the call to Contains will suppress an Exception from the
                * underlying SQL which cannot contain more than 2100 parameters. (Known issue in LinqToSQL.) Then we get 
                * an EMPTY grid with no error. To work around this, using 1000 to be safe, if there more than 1000 items, 
                * break it into batches, and call Contains on each batch of 1000, add the results to a List<T>, then 
                * convert back to IQueryable<T> at the end. */
            if (accountNumbers.Count <= 1000)
            {
                Data = Data.Where(a => accountNumbers.Contains(a.AccountNumber));
            }
            else
            {
                var accountQuery = accountNumbers.GroupBy(num => accountNumbers.IndexOf(num) / 1000);

                List<FutureDatedTransactions> dataList = new List<FutureDatedTransactions>();

                for (int i = 0; i < accountQuery.Count(); i++)
                {
                    var batch = accountQuery.ElementAt(i).ToList();

                    var subSet = Data.Where(a => batch.Contains(a.AccountNumber)).ToList();
                    dataList.AddRange(subSet);
                }

                Data = dataList.AsQueryable<FutureDatedTransactions>();
            }
        }
    }

    e.KeyExpression = "ID";
    e.QueryableSource = Data;
}
Posted in Programming | Tagged , | Leave a comment

How to load a cursor from embedded resources in C#

Last night, being the last night my son was staying with his cousins, I was bored and spent some time looking at some old code. I don’t know if anyone else still uses Windows Forms applications, but in case you do, I’m sharing this…

Via some code that I found and then modified, my image viewer, when viewing an image that’s larger than the viewable area, allows you to use the mouse to drag around and pan to different parts of the image. When doing this, the program changes the mouse cursor, to one of two cursor images, one for a hand that’s up and one for a hand that’s down.

But as it so often happened to me when building that application, dot Net has no way of loading a cursor from the application’s embedded resources. It can load normal images, but not CUR files. That shit just isn’t implemented. So I had to do so myself, and it took a while to figure out because there are several different, confusing API’s that work with cursors, and the MSDN documentation is confusing.

Here’s a screenshot what it looks like when the hand down cursor is displayed. (Image viewer code not included here. This is just about the code to load the cursor.)

viewer

And here’s the code that loads the cursor, which is saved as a file resource, from the applications resources… (The plugin I have used for ages, that shows syntax highlighting in the code, doesn’t seem to work properly anymore. Also the code is wrapped on the blog, but you can copy it to a file.)

using System;
using System.Runtime.InteropServices;
using System.Security;
using System.Windows.Forms;

namespace Romy.Core
{
    public static class CursorResourceLoader
    {
        #region Methods

        public static Cursor LoadEmbeddedCursor(byte[] cursorResource, int imageIndex = 0)
        {
            var resourceHandle = GCHandle.Alloc(cursorResource, GCHandleType.Pinned);
            var iconImage = IntPtr.Zero;
            var cursorHandle = IntPtr.Zero;

            try
            {
                var header = (IconHeader)Marshal.PtrToStructure(resourceHandle.AddrOfPinnedObject(), typeof(IconHeader));

                if (imageIndex >= header.count)
                    throw new ArgumentOutOfRangeException("imageIndex");

                var iconInfoPtr = resourceHandle.AddrOfPinnedObject() + Marshal.SizeOf(typeof(IconHeader)) + imageIndex * Marshal.SizeOf(typeof(IconInfo));
                var info = (IconInfo)Marshal.PtrToStructure(iconInfoPtr, typeof(IconInfo));

                iconImage = Marshal.AllocHGlobal(info.size + 4);
                Marshal.WriteInt16(iconImage + 0, info.hotspot_x);
                Marshal.WriteInt16(iconImage + 2, info.hotspot_y);
                Marshal.Copy(cursorResource, info.offset, iconImage + 4, info.size);

                cursorHandle = NativeMethods.CreateIconFromResource(iconImage, info.size + 4, false, 0x30000);
                return new Cursor(cursorHandle);
            }
            finally
            {
                if (cursorHandle != IntPtr.Zero)
                    NativeMethods.DestroyIcon(cursorHandle);

                if (iconImage != IntPtr.Zero)
                    Marshal.FreeHGlobal(iconImage);

                if (resourceHandle.IsAllocated)
                    resourceHandle.Free();
            }
        }

        #endregion Methods

        #region Native Methods

        [SuppressUnmanagedCodeSecurity]
        static class NativeMethods
        {
            [DllImportAttribute("user32.dll", CharSet = CharSet.Unicode)]
            [return: MarshalAs(UnmanagedType.Bool)]
            public static extern bool DestroyIcon(IntPtr hIcon);

            [DllImport("user32.dll", SetLastError = true)]
            public static extern IntPtr CreateIconFromResource(IntPtr pbIconBits, int dwResSize, bool fIcon, int dwVer);
        }

        #endregion Native Methods

        #region Native Structures

        [StructLayout(LayoutKind.Explicit, Pack = 1)]
        struct IconHeader
        {
            [FieldOffset(0)]
            public short reserved;

            [FieldOffset(2)]
            public short type;

            [FieldOffset(4)]
            public short count;
        }

        /// <summary>Union structure for icons and cursors.</summary>
        /// <remarks>For icons, field offset 4 is used for planes and field offset 6 for 
        /// bits-per-pixel, while for cursors field offset 4 is used for the x coordinate 
        /// of the hotspot, and field offset 6 is used for the y coordinate.</remarks>
        [StructLayout(LayoutKind.Explicit, Pack = 1)]
        struct IconInfo
        {
            [FieldOffset(0)]
            public byte width;

            [FieldOffset(1)]
            public byte height;

            [FieldOffset(2)]
            public byte colors;

            [FieldOffset(3)]
            public byte reserved;

            [FieldOffset(4)]
            public short planes;

            [FieldOffset(6)]
            public short bpp;

            [FieldOffset(4)]
            public short hotspot_x;

            [FieldOffset(6)]
            public short hotspot_y;

            [FieldOffset(8)]
            public int size;

            [FieldOffset(12)]
            public int offset;
        }

        #endregion Native Structures
    }
}

The code that uses it, in another class, is just this:

/// <summary>Loads the two hand cursors from project resources.</summary>
private static void InitializePanBoxCursors()
{
    if (handUpCursor == null && handDownCursor == null)
    {
        handUpCursor = CursorResourceLoader.LoadEmbeddedCursor(Properties.Resources.Hand_up);
        handDownCursor = CursorResourceLoader.LoadEmbeddedCursor(Properties.Resources.Hand_down);
    }
}

Note that those cursors were added to the project resources as file resources, with the names Hand_down and Hand_up. They are then accessible by name in code that Visual Studio generates. Those resources give you each cursor as a byte array, and my code wraps the Windows API CreateIconFromResource function.

Edit: Now I see there is a “Community Addition” at the bottom of that MSDN article linked immediately above, that tells you not to destroy the loaded icon using DestroyIcon. Ignore that annoying bullshit comment. Of course you must release the memory of the icon image. My code creates a new cursor from the loaded resource handle, and then releases the handle properly, and also releases the other memory it allocates. (A pinned handle of the unmanaged resource, unmanaged memory for the icon image which is used to create the cursor resource, and the Windows handle of the cursor itself.) One should always release such handles and any memory allocated explicitly. This is common sense. Honestly, it was better before MSDN allowed idiot programmers to make random “Community Addition” bullshit comments on their documentation. The comment complains about leaks when allocating thousands of cursors, while contradicting itself by telling you not to free the memory you allocate. Yes, that is a sure way to leak memory…

Posted in Programming | Tagged , , | Leave a comment

A pet hate and anti-pattern: A method returning a bool that can never return false, but can throw an Exception.

Recently I came across some code that, while syntactically correct, does something that always gets to me.

Let’s illustrate it with an example:

using System;

namespace Example
{
    public class AntiPatterns
    {
        public bool NeverReturnsFalse()
        {
            try
            {
                // Do something complicated
                // that can fail and throw.
            }
            catch (Exception ex)
            {
                // Code that logs the Exception.

                throw;
            }

            return true;
        }
    }
}

Full disclosure: I didn’t study programming or computer science, but slipped into this industry by accident. I’m a self taught programmer. However, one must use one’s common sense…

Methods that return a bool should return either true or false. (I can’t believe I had to write that.) They represent actions attempted which may succeed or fail. The above method can never return false. Its return value is hard-coded as true, so it will only ever return true unless an error occurs, in which case it will throw an Exception.

Why return a bool at all here? It makes no sense because the operation attempted, whatever it may be, can’t possibly return false (although it can fail). That means whoever consumes the method is probably going to write code, either in an else block or maybe a conditional statement, that checks the return value of this method. That is, whoever consumes this method is going to waste their time writing code that can never run, or check a condition that doesn’t need checking, because unless they have access to this code – which they won’t if it is a library – they must assume that a method returning a bool can either succeed or fail.

A less serious issue is that it can throw an Exception, but this is only less serious because everybody should always catch Exceptions anyway. However, what they need to do in their Exception handler is now ambiguous. Should I treat an Exception like false? Is it a recoverable error? Who knows? If I’m calling it from a thread that’s not critical to my application, I’m going to have to abort whatever that thread does gracefully and just report that it failed (and hope that it might work next time?). If I’m calling it from the one and only thread of a desktop application, I have no choice but to crash the application because I really have no idea what this error means and it is thus unsafe to continue. And that’s not cool.

Just don’t do that.

What common sense dictates you do instead:

  1. If you’re going to return a bool, it must be possible for it to be true or false. Otherwise don’t return a bool. Return void.
  2. If you’re writing a library that a third party may use, but won’t have your source code, don’t just rethrow any random Exception. Throw your own custom Exception if you must, but document that it can do so and what it means, possibly with the random one in the InnerException property.

Of course, if you do throw random Exceptions from a library, it’s probably not going to be used by anyone for long.

Posted in Programming | Tagged | Leave a comment

Beware of WCF POST service methods that receive JSON that contain DateTime values

I ran into a weird problem today… I have a third party that returns some data to our endpoint in JSON format.

I created my endpoint, posted JSON to it, and all seemed well. But when the third party posted it, the service returned Http 400, Bad Request. Turns out the third party was seeing this:

The server encountered an error processing the request. The exception message is ‘There was an error deserializing the object of type ReturnService.ResponseStructure. DateTime content ‘2016-10-26 13:02:38’ does not start with ‘\/Date(‘ and end with ‘)\/’ as required for JSON.’. See server logs for more details.

So far, I had been working with JSON that I serialized and deserialized myself using JavaScriptSerializer. However, when creating a service method to do the deserializing for me, the WCF service uses something else based on a JSON spec older than ISO8601, and expects the dates to be in a very different format. Since this behaviour happens before it even hits my code, it’s rather annoying.

The hacky solution I used was to treat the dates as strings, ironically something I always advise everybody not to do. Then use DateTime.TryParseExact to get the actual dates.

That is, my service method and the class for the JSON is now declared like this:

using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Web;

namespace ExampleService
{
    [ServiceContract]
    public interface IExampleReturnService
    {
        [OperationContract]
        [WebInvoke(UriTemplate = "/PostJSONExample",
            Method = "POST",
            RequestFormat = WebMessageFormat.Json)]
        void PostJSONExample(ResponseStructure request);
    }

    #region JSON structure

    [DataContract]
    public class record
    {
        [DataMember]
        public string text { get; set; }

        [DataMember]
        public string timestamp { get; set; }

        [DataMember]
        public int pairedMessageId { get; set; }

        [DataMember]
        public string receiver { get; set; }
    }

    [DataContract]
    public class data
    {
        [DataMember]
        public record[] record { get; set; }
    }

    [DataContract]
    public class ResponseStructure
    {
        [DataMember]
        public data data { get; set; }
    }

    #endregion
}

And the implementation, with only the code relevant to parsing the date, is this:

    public class ExampleReturnService : IExampleReturnService
    {
        public void PostJSONExample(ResponseStructure request)
        {
            foreach (record r in request.data.record)
            {
                DateTime responseDate;
                if (DateTime.TryParseExact(
                    r.timestamp, 
                    "yyyy-MM-dd HH:mm:ss", 
                    CultureInfo.InvariantCulture, 
                    DateTimeStyles.AdjustToUniversal, out responseDate))
                {
                    DateTime date = responseDate.ToLocalTime();
                    // Do something with date...
                }
            }
        }
    }
Posted in Programming | Tagged , , , | Leave a comment

T-SQL to get the counts of records within age ranges

This post is written just in case future me has to do this again…

I was emailed a spread sheet containing thousands of birth dates by my boss, and asked to return a segregation based on the number of them less than 20 years old, between 20 and 30, and so on… (The dates were, of course, strings in a weird format, but handling that is outside of the scope of this post.)

What I did was, import the dates into a table in SQL Server, which for want of better names, I called DOB, with a single field in it called DateOfBirth. I found an almost working solution on that god-forsaken portal of yesteryear, experts-exchange, of all places.

This query is based on the solution there, except it works properly.

select result.Age, count(*) Count
from 
(
select
  case when age < 20 then 'less than 20'
           when age between 20 and 30 then '20-30'
           when age between 31 and 40 then '31-40'
           when age between 41 and 50 then '41-50'
           when age between 51 and 300 then '51+'
/* and so on */
  end Age
from (SELECT DATEDIFF(year,DateOfBirth, getdate()) Age 
    from DOB where DateOfBirth is not null) age
) result
group by result.Age
order by Age

Posted in Database | Tagged | Leave a comment

How to convert a string to ASCII in c#

This is one of those cases where smart-ass developers will tell you that you should not be converting strings to ASCII, because reasons

However, if you’re sending a file to a host-to-host banking system, or sending text to an external web service that then sends SMS messages, you might find yourself in a situation where your request will be rejected. Now while it might be frowned upon to blindly try converting Unicode to ASCII, I prefer, when submitting millions (or maybe thousands) of debit orders to a bank, that the whole fucking batch doesn’t get rejected because of some stupid character that the bank doesn’t understand.

Anyway, I can’t remember where I found this code, but the key is to use String.Normalize, to normalize a string using full compatibility decomposition – which generally results in a conversion that looks about as correct as you’re going to get by manually fixing a badly input string… and then throw away all characters that are outside of the ASCII range.

This works:

using System.Linq;
using System.Text;

namespace Demo
{
    class Example
    {
        public static string LatinToAscii(string s)
        {
            var builder = new StringBuilder();
            builder.Append(s.Normalize(NormalizationForm.FormKD)
                                            .Where(x => x < 128)
                                            .ToArray());
            return builder.ToString();
        }
    }
}
Posted in General, Programming | Tagged | Leave a comment

How to save a simple tabular Excel file from C# using EPPlus

An application I wrote was exporting a couple of different sets of objects as CSV files, but the people using it wanted Excel files. Although Microsoft designed C# and everybody involved assumed that such formats are native to dot Net, they are not.

I used a class library called EPPlus, which seems to be the most popular way of doing this. Unfortunately the examples on their website are not useful, showing things like how to set some arbitrary cell on an existing spread sheet to an image or bar chart. The library also supports functions and all sorts of shit, but really, the only thing I care about is exporting a damn table because the users don’t know how to import CSV into their proprietary system.

All I want is to take a list of objects, that I was exporting to CSV before, and write it to a new XLSX file. I found a solution on StackOverflow here, but here is a more complete implementation. (Assuming that you have an instance of a class called Response. This should be general enough for you to modify it according to whatever your objects and fields are. I’ve simplified this so that it doesn’t look anything like the actual objects I work with.)

The EPPlus Worksheet class has a handy method to load a worksheet from a table, and if you load it in the very first cell of a spread sheet, it gives you what you want. (Why didn’t they document that? There is nothing standard or intuitive about it at all. Like, I’d expect a newly instantiated work sheet to have no cells, and have an empty collection by default, and would never have guessed that I can just load a table into a magically accessed cell A1.) So all I needed to do was create a temporary DataTable instance to hold the data, and then it was easy… I don’t already have the objects in a table, because I’m a couple of levels of abstraction away from that. So this code does everything it needs to do, including creating a temporary in-memory table, a new file to contain the data, and then writes the data to the file.

public void SaveToExcel(List<Response> responses)
{
    string responsePath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location) + "\\Responses";

    DataTable table = new DataTable();
    table.Columns.Add("Response", typeof(string));
    table.Columns.Add("ResponseDate", typeof(DateTime));
    table.Columns.Add("CellNumber", typeof(string));

    foreach (var response in responses)
        table.Rows.Add(response.Response, response.ResponseDate, response.CellNumber);

    string filename = DateTime.Now.ToString("yyyy-MM-dd HHmmssfff") + ".xlsx";
    filename = Path.Combine(responsePath, filename);
    Directory.CreateDirectory(responsePath);

    using (var stream = new FileStream(filename, FileMode.Create, FileAccess.Write, FileShare.None, 0x2000, false))
    {
        using (ExcelPackage pck = new ExcelPackage(stream))
        {
            ExcelWorksheet ws = pck.Workbook.Worksheets.Add("Responses");
            ws.Cells["A1"].LoadFromDataTable(table, true);
            pck.Save();
        }
    }
}
Posted in Export XLSX file | Tagged | Leave a comment

A SQL helper class that allows executing a query or running a stored procedure, with or without dynamic parameters

A while back I wrote about my extension method to copy a non-generic collection to an array. A commenter pointed out that it isn’t really necessary. Oh well… here is some code that uses it… (If you use this code, you will need the extension method from the linked post, or else just do what it does instead of calling ToArray<DataColumn> in the code included.)

I use this all over the show, where I want to do a simple SQL query or run a stored procedure and get the results in a table. I like it because it makes it easy to do queries with any number of parameters, by using a collection initializer for a Dictionary type.

An example of usage is this:

        private void Example()
        {
            int id = 666;
            var queryHanlder = new QueryHandler(this.connectionString);

            var data = queryHanlder.OpenQuery(
                "SELECT Devil FROM Details WHERE ID = @ID", 
                new Dictionary<string, object> { {"@ID", id} });

            // Do something with data...
        }

And here’s the class… It contains the method called in the example above, overloaded to be called with or without parameters, and similar overloaded methods to call a stored procedure with or without parameters. I wrote this code years ago, but use it all the time, and it has crept into every project at work.

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Linq;

namespace Demo
{
    public class QueryHandler
    {
        private string connectionString;

        public QueryHandler(string connectionString)
        {
            this.connectionString = connectionString;
        }

        public DataTable OpenQuery(string query)
        {
            return OpenQuery(query, new Dictionary<string, object>());
        }

        public DataTable OpenQuery(string query, Dictionary<string, object> parameters, CommandType commandType = CommandType.Text)
        {
            using (var connection = new SqlConnection(this.connectionString))
            {
                using (var adapter = new SqlDataAdapter(query, connection))
                {
                    connection.Open();
                    try
                    {
                        var table = new DataTable();
                        adapter.SelectCommand.CommandType = commandType;

                        foreach (var kvp in parameters)
                            adapter.SelectCommand.Parameters.AddWithValue(kvp.Key, kvp.Value);

                        using (var reader = adapter.SelectCommand.ExecuteReader())
                        {
                            using (var schemaTable = reader.GetSchemaTable())
                            {
                                foreach (DataRow row in schemaTable.Rows)
                                    table.Columns.Add((string)row["ColumnName"], Type.GetType(row["DataType"].ToString()));
                            }

                            var columns = table.Columns.ToArray<DataColumn>();

                            while (reader.Read())
                                table.Rows.Add(columns.Select(c => reader[c.Ordinal]).ToArray());
                        }
                        return table;
                    }
                    finally { connection.Close(); }
                }
            }
        }

        public DataTable OpenStoredProcedure(string procName)
        {
            return OpenQuery(procName, new Dictionary<string, object>(), CommandType.StoredProcedure);
        }

        public DataTable OpenStoredProcedure(string procName, Dictionary<string, object> parameters)
        {
            return OpenQuery(procName, parameters, CommandType.StoredProcedure);
        }
    }
}
Posted in Database, SQL | Tagged , | 1 Comment

Who else modified QBASIC’s NIBBLES to run on Windows 95?

I just read a blog post by Raymond Chen. It’s a  historical post about getting an old MS-DOS game to run on Windows 95…

But what really got my attention was one of the comments, where the commenter mentioned a problem with the old QBASIC game, NIBBLES. He mentioned the issue it had when trying to run it on Windows 95, and the obvious fix that anyone could make since it came with source code.

I did that too! I feel like such an old geek now. I wonder who else did that back then?

Posted in General | Tagged | Leave a comment

I hate SQL. One way of doing a subquery in a JOIN statement

I really do.

I’m writing this mostly for myself because I never remember this shit. I wrote this query  now but I know that I won’t remember how to do this in future, so here is the public note to self for next time. (This is slightly changed from the entity names at work so that I can’t get into trouble.)

I want a simple query that returns a couple of fields, the cash on hand, and the sum of the balance of orders in progress for a particular store, given these conditions…

  1. We have generic things called Stores. Each store has a balance.
  2. We also have generic things called purchase orders, multiple of them per store, where each refers to the store by a foreign key.
  3. The purchase orders have an amount, but not a balance column.
  4. There are multiple stock received entries against every purchase order.
  5. The purchase order balance is thus the purchase order amount, minus the sum of all the stock received entries against that purchase order.

So I really need two select statements… One from Store and one from PurchaseOrders. The problem is the purchase order balance is calculated by joining with the stock received table, and aggregating, so I can’t do the whole thing in one simple query, or so I thought.

It turns out that you can do a subquery in a join statement, then simply alias it, and this worked for me… All I needed to do was add the store ID field to the subquery so that I could treat it just like a table and join against it. I’ve done something like this before, and forgotten how… hence this stupid post. Did I mention I hate SQL?

SELECT Store.Balance CashOnHand, ISNULL(SUM(t.Balance), 0) OrderInProgress
FROM Store 
LEFT JOIN (
    SELECT po.Amount - ISNULL(SUM(s.amount), 0) Balance, po.StoreID
    FROM            
        PurchaseOrder po 
        LEFT JOIN StockReceived s ON s.PurchaseOrderID = po.id 
    WHERE 
        po.StoreID = @StoreID
    GROUP BY 
        po.ID, po.Amount, po.StoreID ) t ON t.StoreID = @StoreID
WHERE Store.ID = @StoreID
GROUP BY Store.Balance
Posted in Database, SQL | Tagged | 1 Comment

When cleanup fails and tells me that I need to run cleanup

Huh? Normally I wouldn’t write a post like this because I don’t have a solution. But this one is just too funny in its own stupid way.

Yesterday I needed to commit a simple change to a project, and we use TortoiseSVN. Before that, I had to do an update because my colleague had made an unrelated change to the same page. This was an ASP.Net project. Then there were some unrelated conflicts. For whatever unknown reason, it always screws up the publishing profile. I lose my password for the remote machine. So I resolved the conflict. I published the site. Then I want to commit the simple change… just two files. And lo and behold, I get the error displayed in the screenshot. But I get that error for anything I try to do in TortoiseSVN, including cleanup.

When I find a solution, I’ll probably add it. This is not too big a deal right now, because it’s only a tiny change and not the main project I’m working on. One solution would be to delete the whole directory… make a new repository and download all the code from scratch. Not the end of the world because I can back up my tiny change and then overwrite the files. So this is not a train-stopper… It’s just annoying. (And more than a little annoying because I have some configuration files in the directory tree that I never commit. So I’d have to back up quite a few files, but that solution will probably work.)

Cleanup

Posted in Programming, Work | Tagged | Leave a comment

Convert a non-generic collection to an array – my most useful extension method ever…

Funny how things work out… I wrote one of my most simple extension methods years ago, one that converts a non-generic collection to an array (because arrays implement IEnumerable<T> on which I can call Linq extension methods).

Lately I find I’ve been using this all the time. This:

using System.Collections;

namespace Example
{
    public static class Extensions
    {
        /// <summary>Converts a non-generic collection to an array.</summary>
        public static T[] ToArray<T>(this ICollection collection)
        {
            var items = new T[collection.Count];
            collection.CopyTo(items, 0);

            return items;
        }
    }
}

An example where this becomes useful is this… Say you have an in-memory dataset in the form of a table, called “table” for want of a better name, containing some rows, and one of the columns is an amount field. If you wanted the sum of the amount, you could just do this:

            var rows = table.Rows.ToArray<DataRow>();
            var total = rows.Sum(r => (double)r["Amount"]);
Posted in Programming | Tagged , , | 2 Comments

How to get the row index in the client side code of a DataItemTemplate in an ASPxGridView column

This was pissing me off for far too long today. I also have a DataItemTemplate with an ASPxHyperlink control in one of my columns of an ASPxGridView. When the user clicks the hyperlink, I need to call a JavaScript function that uses the index of that row in the grid… (Then, and this is outside the scope of this post but for information and context, I do a callback to the server side, look up some more specific information in the database, pass some data back to the client, and when the callback is complete, show a confirmation popup with a that shit data that I returned from the server. The user can then click an OK button on the popup which does another callback and that does some other stuff.) For some unknown reason, the complicated-sounding part was easy, but I got stuck on step one… How to get the index of the row in the grid where the link was clicked?

Consider that when the link is clicked, it nether sets the grid row selected nor focused. So the usual suspects (the devexpress controls’ functions to get visible row index or focused row index) are not helpful here… They return null.

So the column in the markup of the grid looks like this… (Nothing special here.)

<dx:GridViewDataTextColumn FieldName="Status" ShowInCustomizationForm="True" VisibleIndex="7">
    <DataItemTemplate>
        <dx:ASPxHyperLink ID="hlAuthorise" runat="server" Text='<%# Eval("Status") %>' OnInit="hlAuthorise_Init">
        </dx:ASPxHyperLink>
    </DataItemTemplate>
</dx:GridViewDataTextColumn>

The JavaScript function that needs to be called when the link gets clicked is this… (Actually it’s not this, but you don’t need to see it. All that matters is it needs to be passed the index.)

        function AuthoriseClick(s, e, index) {
            alert(index);
        }

And the way to do it turns out to be easy. Just not so easy to find online. The trick is to set the index on the server side for each row, via the Init event of the hyperlink. (So I lied when I said there was nothing special above. If you scrolled to the right, you’d have noticed the Init event handler in the markup.) The C# code on the server-side is this:

        protected void hlAuthorise_Init(object sender, EventArgs e)
        {
            /* This is the only way I've found to allow the client side code in a 
             * DataItemTemplate to get the index of the row in its container grid. */
            ASPxHyperLink link = sender as ASPxHyperLink;
            GridViewDataItemTemplateContainer container = ((ASPxHyperLink)sender).NamingContainer as GridViewDataItemTemplateContainer;
            link.ClientSideEvents.Click = String.Format("function (s, e) {{ AuthoriseClick(s, e, {0}); }}", container.VisibleIndex);
        }
Posted in Programming | Tagged , , , | 2 Comments

How to insert multiple literals explicitly into a table, only if they don’t already exist

It might seem silly, but recently I needed to insert some values into a new table I added to a system, but only if those values didn’t already exist, and I did not know how. (Yes, it’s a new table and is thus empty. But this technique should work when adding other values into it in future.) I know how to do that when inserting a selection from another table, but in this case I wanted to insert literals, and all my Googling came up empty…

So even though this is really quite an easy task, I figured writing it here might help somebody from going to the few minutes of trouble I had to. (And more importantly, forgetful me can refer to this post when I forget how I did it.)

To illustrate what I mean, let’s start with a simple table, which will hold the names of some super heroes. (I can’t give the details of the real table I created if I value my employment.) Because I’m lazy, I create tables in the designer in SQL Server Management Studio, and then script them afterwards. The creation script for such a table could be as follows:

CREATE TABLE [dbo].[JusticeLeague](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Code] [varchar](10) NOT NULL,
    [Description] [varchar](100) NOT NULL,
    [DateCreated] [datetime] NOT NULL,
 CONSTRAINT [PK_JusticeLeague] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

I usually insert multiple literals like so:

DECLARE @Today datetime = getdate();

INSERT INTO JusticeLeague (Code, Description, DateCreated) values
('SUP', 'Superman', @Today),
('BAT', 'Batman', @Today),
('WON', 'Wonder Woman', @Today),
('FLASH', 'The Flash', @Today),
('AQUM', 'Aquaman', @Today)

But that doesn’t prevent duplicates. What I want my script to do is:

  • Insert the rows only if they aren’t already in the table.
  • If run more than once, it should only insert the first time. When run again, it should do nothing.
  • If new rows are added to the script, when rerun it should only insert those new rows.

Since my searches came up empty (actually I got nothing but irrelevant results), I came up with a way… All this does is:

  1. Select the values as if they are a table, by selecting all rows’ explicit values and doing a union.
  2. Then add a filter using the EXISTS keyword.

It works quite well. So my solution was this:

DECLARE @Today datetime = getdate();

INSERT INTO JusticeLeague (Code, Description, DateCreated)
select t.* from (
select 'SUP' Code, 'Superman' Description, @Today DateCreated union all
select 'BAT', 'Batman', @Today union all
select 'WON', 'Wonder Woman', @Today union all
select 'FLASH', 'The Flash', @Today union all
select 'AQUM', 'Aquaman', @Today) t
where not exists(select 1 from JusticeLeague i where i.Code = t.Code)

This script can be rerun, and when adding new values, which will definitely happen in the real table that I based this contrived example on, the script can safely be rerun and only the new rows will be inserted.

For example, I can safely change the script as below, and only one new hero, Green Lantern, will be added:

DECLARE @Today datetime = getdate();

INSERT INTO JusticeLeague (Code, Description, DateCreated)
select t.* from (
select 'SUP' Code, 'Superman' Description, @Today DateCreated union all
select 'BAT', 'Batman', @Today union all
select 'WON', 'Wonder Woman', @Today union all
select 'FLASH', 'The Flash', @Today union all
select 'GRE', 'Green Lantern', @Today union all
select 'AQUM', 'Aquaman', @Today) t
where not exists(select 1 from JusticeLeague i where i.Code = t.Code)

Note that I’m using UNION ALL. This would actually cause duplicates if the script contained duplicate values, but performs better than UNION, which scans for duplicates. This is fine for me, because when building the script, I take care not to include duplicate literals.

Posted in SQL | Tagged , | Leave a comment

How to add to the Windows recent documents in C#

I realize that I haven’t written anything here for ages… so I looked through my old code for something simple but useful, and found this.

If ever you need to add a file to the Windows recent document list, this does the trick. I’m using this in a Windows Forms application that can open and view a number of file types, such as videos, audio and text files. Adding them to the recent documents allows me to open the files using the registered file handlers (I don’t register my application to handle the files on a system level) to compare my viewers to others. The code is simple and self-explanatory, so here it is without any explanation:

using System.Runtime.InteropServices;
using System.Security;

namespace Sample
{
    public static class Example
    {
        public static void AddToRecentDocuments(string path)
        {
            NativeMethods.SHAddToRecentDocs(ShellAddToRecentDocsFlags.Path, path);
        }
    }

    internal enum ShellAddToRecentDocsFlags
    {
        Pidl = 0x001,
        Path = 0x002,
    }

    [SuppressUnmanagedCodeSecurity]
    internal static class NativeMethods
    {
        // Note: The function call fails if we use CharSet.Unicode or CharSet.Auto or if path is not Ansi.
        [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Globalization", "CA2101:SpecifyMarshalingForPInvokeStringArguments", MessageId = "1"), DllImport("shell32.dll", CharSet = CharSet.Ansi)]
        public static extern void SHAddToRecentDocs(ShellAddToRecentDocsFlags flag, [MarshalAs(UnmanagedType.LPStr)] string path);
    }
}
Posted in Programming | Tagged , | Leave a comment

Spammers need better targeting algorithms

I don’t check this blog’s stats much; my other blog is the one I focus more on these days. Since I just published something here, I decided to clean up the spam.

Why would you auto-spam a programming blog, in South Africa, with something about “cheap eggs UK”? For that matter, why would you target any blogs with spam about fucking buying eggs? If you want to get some cheap-ass eggs, do you scan random WordPress blogs to find them, for fuck’s sake?

Posted in General | Tagged | 2 Comments

How to ensure that a number is always round up to the nearest integer

Yes, this is silly. I never used Math.Ceiling before; so I never even knew what it was for.

In my use case, I’m sending SMS messages via a third party WCF service. It works well, but in my own WCF service, I forgot to limit the length of the message text to 160 characters. That turned out to be a good thing, because they send it anyway, and charge us for the extra messages, which is fine. So all I needed to do was add a parts field to our database table, so that we know how many messages to bill our clients for.

So the requirement is that any multiple of 160 characters gets a part. Anything below 160 characters is one message. Anything greater than 160 characters and less than 320 characters is two messages, and so on…

So, for example, 168 characters is 168 / 160 = 1.05. Math.Round will round down to one. I want it to be two, because any part of a message is a whole message, obviously.

Somebody asked the question on StackOverflow. Actually I found it via this duplicate. Note that I am ignoring the most efficient answer, the “correct” answer if you will… I like the simplest answer, which in my code comes out to:

command.Parameters.AddWithValue("@Parts", (int)Math.Ceiling(message.Length/160D));

So in my example that would be (pseudocode) Math.Ceiling(168/160), which equals 2.

Posted in Programming | Tagged , | Leave a comment

Mailto links with subject and body

This is most probably not news to anybody besides myself, but regardless, I didn’t know that you can create email links including both the subject and body…

I was writing something on my other blog and needed to include some snark with a link to my email address. I found the solution on StackOverflow here.

I didn’t specify the body, but the idea is, you can just include the body and subject in the link as query strings. Really, I had no idea. For example, just with the subject:

<a href="mailto:viveirosjerome@gmail.com?subject=My Banking Details">Send me your banking details</a>

And to specify the body too:

<a href="mailto:viveirosjerome@gmail.com?subject=My Banking Details&body=I didn't know you could do this">Send me your banking details</a>

Here follows the bit of my other blog-post that used it:


Cure your ailments with magic socks

I love reading about science based medicine, and I love the Science based Medicine Blog and it’s debunking of nonscience nonsense. To think that somewhere in the civilized world, “experts” in a naturopathic university advocate the healing of all your ailments with magic socks (also known as cold wet socks taken out of the fucking freezer). The mind boggles! Read the thorough debunking by an actual medical doctor here. (My brief version: Naturopaths come up with some solid sounding sophisticated pseudoscientific nonsense… some hand-waving and the child having to maintain homoeostasis while wearing freezing socks all night kick starts the immune system. But according to the author, naturopaths’ feet fixation does not end with magic socks.)

By the way, if you believe in magic socks, I have just the thing for you. Send me your banking details (please attach permission for me to debit your account) and physical address, and I’ll send you the secret to infinite riches, along with some magic beans.

Posted in HTML | Tagged | Leave a comment

How to convert Unicode text to plain ASCII

Let’s say you have a string that was input by somebody out of your control… Maybe it got to you via a call to your web service from a remote client, indirectly via somebody else’s software. And you need to ensure that the string only contains ASCII characters but still want it to be human-readable. How? Most examples you’ll find online when asking how to do this, do not do what I want…

And they’re right. Sure, accented characters and unaccented characters are not the same. But I know of one time when I don’t care:

  1. When sending mobile text (SMS) messages via a third party, and that third party fails to send the messages if Unicode characters outside the range 0 to 127 are included.

So while converting accented characters to unaccented characters might not be strictly correct, it is as correct as I need in my use case. The answer is the String.Normalize method, which I’d never heard of.

The sample code that follows returns the following output: (where the first string is the input, and the second is the normalized ASCII string)

image

And here’s the code. I’d give credit to the LatinToAscii method, which I found on StackOverflow, but I lost the link. (Although this works for me, it wasn’t even the accepted answer.)

using System;
using System.Linq;
using System.Text;

namespace AsciiTest
{
    class Program
    {
        static void Main(string[] args)
        {
            string inputString = "Räksmörgås";
            string asAscii = LatinToAscii(inputString);

            Console.WriteLine(inputString);
            Console.WriteLine(asAscii);

            Console.ReadKey();
        }

        private static string LatinToAscii(string inString)
        {
            var builder = new StringBuilder();
            builder.Append(inString.Normalize(NormalizationForm.FormKD)
                                            .Where(x => x < 128)
                                            .ToArray());
            return builder.ToString();
        }
    }
}

It should be obvious what the code does, so I’m not going to explain. See this explanation for the NormalizationForm value passed to the method, for a thorough understanding.

Posted in Programming | Tagged , | Leave a comment

A minor pet peeve: “Send us the XML!”

I still haven’t gotten around to writing my pet peeves post, but in the meantime, here’s a really short one… A single, work-related pet peeve.

Send us the XML so that we can try to find the problem!

This has come up in two different contexts at work recently:

  1. We use a web service, so we are the customers. I won’t say what for because that’s not something I can disclose. So let’s just say that we use this service to send some requests, and sometimes, the response from the external party is useless. They return unprocessed results for everything we sent them.
  2. In the other case, we have a customer using our service. Once again, never mind what exactly it does. We send a result to their “listening” service, and then we get back a HTTP 200 (OK). And they claim they never received it.

In both cases, they ask us: “Please send us the XML”. In case one, they ask for the XML that was generated in code and sent to them. In case two, they ask for the XML that they sent us.

What I really want to tell them is, “Fuck off. There is no XML anymore.” I mean, the C# code either interprets the XML and then works with objects without saving the XML anywhere, or in the other direction the XML was generated automatically from objects defined in code and sent over the wire. This is a production server and the XML is temporary data used only for submission. You shouldn’t be asking me for the fucking XML! (You especially shouldn’t be asking for the XML that you sent to us.)

Posted in Uncategorized | Tagged | Leave a comment

How to encode or decode a base64 string

This is an absolutely trivial programming task, but one that annoyed me enough to have to write some code to do…

I use Filezilla for FTP uploads/downloads at work, but never remember my passwords for FTP sites. Nor do I store them anywhere. What I’ve always done is, if I need to give someone a FTP password, just use Filezilla’s export functionality to export the list of sites. I did that for my colleague the other day, and to my surprise saw that the current version of the application writes every password in the exported file as a base64 encoded string. Why? Other than to be annoying, I can not imagine a reason.

So I wrote a stupid little application to do this:

image

It uses a couple of simple extension methods:

using System;
using System.Text;

namespace Base64StringHandler
{
    public static class StringExtensions
    {
        public static string Base64Encode(this string input)
        {
            byte[] data = Encoding.UTF8.GetBytes(input);
            return Convert.ToBase64String(data);
        }

        public static string Base64Decode(this string input)
        {
            byte[] data = Convert.FromBase64String(input);
            return Encoding.UTF8.GetString(data);
        }
    }
}

The code for the form is this:

using System;
using System.Windows.Forms;

namespace Base64StringHandler
{
    public partial class Base64StringHandlerForm : Form
    {
        public Base64StringHandlerForm()
        {
            InitializeComponent();
        }

        private void btnEncode_Click(object sender, EventArgs e)
        {
            if (!string.IsNullOrEmpty(tbPlaintext.Text))
                tbEncoded.Text = tbPlaintext.Text.Base64Encode();
            else
                MessageBox.Show("Input a string to encode.", "No input", MessageBoxButtons.OK, MessageBoxIcon.Error);

        }

        private void btnDecode_Click(object sender, EventArgs e)
        {
            if (!string.IsNullOrEmpty(tbEncoded.Text))
                tbPlaintext.Text = tbEncoded.Text.Base64Decode();
            else
                MessageBox.Show("Input a string to decode.", "No input", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
    }
}

If you dislike extension methods and think a Windows Forms application is overkill for this (It is!) then just use this console application instead. (Code to encode commented out. I just run it from Visual Studio anyway. If I cared to write something to be a reusable executable, I would’ve coded for expected command line arguments or something.)

using System;
using System.Text;

namespace DecodeBase64String
{
    class Program
    {
        static void Main(string[] args)
        {

            //Console.WriteLine("Enter string to base64-enocde:");
            //string enoceded = EncodeBase64(Console.ReadLine());
            //Console.WriteLine(enoceded);

            Console.WriteLine("Enter base64-encoded string to decode:");
            string decodedString = DecodeBase64(Console.ReadLine());
            Console.WriteLine(decodedString);

            Console.ReadKey();
        }

        private static string DecodeBase64(string base64EncodedValue)
        {
            byte[] data = Convert.FromBase64String(base64EncodedValue);
            return Encoding.UTF8.GetString(data);
        }

        private static string EncodeBase64(string plainText)
        {
            byte[] data = System.Text.Encoding.UTF8.GetBytes(plainText);
            return System.Convert.ToBase64String(data);
        }
    }
}

Note that none of this code contains any error handling at all. That’s because its purpose is only to illustrate the simple concept of how to encode or decode a base64 string. Error handling is left as an exercise to anyone who uses this code…

Posted in Programming | Tagged | Leave a comment

The curious case of the generic complimentary comment. I hate SPAM!

I published this on my other blog yesterday. But it is related to programming in a way, so now it’s published here too.


Recently I received this comment to one of my posts:

Hello! I’ve been reading your site for a long time now and finally got the bravery to go ahead and give you a shout out from Huffman Tx! Just wanted to say keep up the great work!

So I thought to myself, “That’s odd“. Reading this new blog for a long time? Sure, the old one redirects here, but judging by the number of page views (on average at the old blog, around 70 per day, but up to a max of about 400, while the most ever here was 15 on one day), most of my old readers haven’t found this one, and my Google indexing is pretty much gone. (Which makes me sad, by the way.) Then I followed the link, and it led to a membership page of a site that contained another link advertising a website for painters. A painters’ website in Toronto, Canada! So I’m getting a shout out from Texas advertising a website in Canada? Those crazy Canaditexans…

So sure enough, a quick search for similar comments elsewhere turned up a blog with this:

Hey there! I’ve been following your blog for some time now and finally got the bravery to go ahead and give you a shout out from Humble Texas! Just wanted to say keep up the great job!

And this:

Hello! I’ve been reading your blog for a long time now and finally got the bravery to go ahead and give you a shout out from Lubbock Texas! Just wanted to tell you keep up the excellent job!

And this:

Hi! I’ve been reading your website for a while now and finally got the courage to go ahead and give you a shout out from Huffman Texas! Just wanted to say keep up the fantastic work!

As a general rule, generic compliments in comments don’t come from readers actually reading the content and commenting, they are generated and bulk-submitted SPAM. They usually contain a single link to an ad as the commenter’s “website”. Since it’s only one link and SPAM filters like Akismet normally look for two or more, these are often not detected by SPAM filters, but any human reader/moderator/blogger can spot them. In this case, the comments were built programmatically from a template, by a despicable program (like this one) that posts comments in bulk to blogs. There are many other examples. I’m leaving the comment here for educational purposes, after removing the spammer’s URL and mailto link of course.

Two years ago Scott Hanselman wrote about such a template. This is clearly generated from that template, which the spammers are still using. Here is the relevant bit that generated all of those comments:

{Hola|Hey there|Hi|Hello|Greetings}! I’ve been {following|reading} your {site|web site|website|weblog|blog} for {a long time|a while|some time} now and finally got the {bravery|courage} to go ahead and give you a shout out from {New Caney|Kingwood|Huffman|Porter|Houston|Dallas|Austin|Lubbock|Humble|Atascocita} {Tx|Texas}! Just wanted to {tell you|mention|say} keep up the {fantastic|excellent|great|good} {job|work}!

The bottom line is this: Beware of generic compliments in comments. They are not what they seem. (I shouldn’t even have followed the link. Merely clicking may have generated revenue for somebody, via their affiliate program that pays according to the number of clicks. Also, such links may take you to porn or malware sites.) Fortunately the people who generate this kind of SPAM are idiots; were they not it might not be so easy to detect.

Then again, any generic comment that’s not related in any way whatsoever to your written content should be a huge red flag. A challenge for a good programmer would thus be to generate a comment that somehow relates to the article being commented on. (To be clear, I mean a purely intellectual challenge. Anybody who publishes or uses such a program deserves to be shot in the stomach, doused with petrol, set on fire and thrown off a cliff.) At the very least, it could quote a random paragraph of your content and then proceed to “comment” in a generic way on that content. (This would be fairly easy to do and would look more like a real comment, but might not be detected so easily fool bloggers desperate for comments.) Of course that would require actually reading your site or blog’s content, at least programatically, and would defeat the purpose and likely the profit of bulk SPAM comment publishing. (Assuming that there is a profit. Do people actually follow those SPAM comment links by clicking on the commenter’s name? Are there bloggers so desperate for comments that they appreciate generic compliments?)

Here’s the full SPAM template:

{
{I have|I've} been {surfing|browsing} online more than {three|3|2|4} hours today, yet I never found any interesting article like yours. {It's|It
is} pretty worth enough for me. {In my opinion|Personally|In my view}, if all {webmasters|site owners|website owners|web owners} and bloggers made good content as
you did, the {internet|net|web} will be {much more|a lot more}
useful than ever before.|
I {couldn't|could not} {resist|refrain from} commenting. {Very well|Perfectly|Well|Exceptionally well} written!|
{I will|I'll} {right away|immediately} {take hold of|grab|clutch|grasp|seize|snatch}
your {rss|rss feed} as I {can not|can't} {in finding|find|to find} your {email|e-mail} subscription {link|hyperlink} or {newsletter|e-newsletter} service. Do {you have|you've} any?
{Please|Kindly} {allow|permit|let} me {realize|recognize|understand|recognise|know} {so that|in order that} I {may just|may|could} subscribe.
Thanks.|
{It is|It's} {appropriate|perfect|the best} time to make some plans for the future and {it is|it's} time to be happy.
{I have|I've} read this post and if I could I {want to|wish to|desire to} suggest you {few|some} interesting things or {advice|suggestions|tips}. {Perhaps|Maybe} you {could|can} write next articles referring to this article. I {want to|wish to|desire to} read {more|even more} things about it!|
{It is|It's} {appropriate|perfect|the best} time to make {a few|some} plans for {the future|the
longer term|the long run} and {it is|it's} time to be happy. {I have|I've} {read|learn} this {post|submit|publish|put up} and
if I {may just|may|could} I {want to|wish to|desire to} {suggest|recommend|counsel} you {few|some} {interesting|fascinating|attention-grabbing} {things|issues} or {advice|suggestions|tips}.
{Perhaps|Maybe} you {could|can} write {next|subsequent} articles {relating to|referring to|regarding} this article.
I {want to|wish to|desire to} {read|learn} {more|even
more} {things|issues} {approximately|about} it!|
{I have|I've} been {surfing|browsing} {online|on-line} {more than|greater than} {three|3} hours {these days|nowadays|today|lately|as of late}, {yet|but} I {never|by no means} {found|discovered} any {interesting|fascinating|attention-grabbing} article like yours. {It's|It is} {lovely|pretty|beautiful} {worth|value|price} {enough|sufficient} for
me. {In my opinion|Personally|In my view},
if all {webmasters|site owners|website owners|web owners} and bloggers made {just right|good|excellent} {content|content material} as {you did|you probably did}, the {internet|net|web} {will be|shall be|might
be|will probably be|can be|will likely be} {much
more|a lot more} {useful|helpful} than ever before.|
Ahaa, its {nice|pleasant|good|fastidious} {discussion|conversation|dialogue} {regarding|concerning|about|on the topic of} this {article|post|piece of writing|paragraph} {here|at this place} at this
{blog|weblog|webpage|website|web site}, I have read all
that, so {now|at this time} me also commenting {here|at this
place}.|
I am sure this {article|post|piece of writing|paragraph}
has touched all the internet {users|people|viewers|visitors}, its really really {nice|pleasant|good|fastidious} {article|post|piece of writing|paragraph} on
building up new {blog|weblog|webpage|website|web site}.
|
Wow, this {article|post|piece of writing|paragraph} is {nice|pleasant|good|fastidious}, my
{sister|younger sister} is analyzing {such|these|these kinds of} things, {so|thus|therefore} I am going to {tell|inform|let
know|convey} her.|
{Saved as a favorite|bookmarked!!}, {I really like|I
like|I love} {your blog|your site|your web site|your website}!
|
Way cool! Some {very|extremely} valid points! I
appreciate you {writing this|penning this} {article|post|write-up} {and the|and also the|plus the}
rest of the {site is|website is} {also very|extremely|very|also really|really} good.
|
Hi, {I do believe|I do think} {this is an excellent|this
is a great} {blog|website|web site|site}. I stumbledupon it 😉 {I will|I am going
to|I'm going to|I may} {come back|return|revisit} {once again|yet again} {since I|since i have} {bookmarked|book marked|book-marked|saved as a favorite} it. Money and freedom {is the best|is the greatest} way to change, may you be rich and continue to {help|guide} {other people|others}.|
Woah! I'm really {loving|enjoying|digging} the template/theme of this {site|website|blog}.
It's simple, yet effective. A lot of times it's {very
hard|very difficult|challenging|tough|difficult|hard} to
get that "perfect balance" between {superb usability|user friendliness|usability} and {visual appearance|visual appeal|appearance}.
I must say {that you've|you have|you've} done a {awesome|amazing|very good|superb|fantastic|excellent|great} job with this.
{In addition|Additionally|Also}, the blog loads {very|extremely|super} {fast|quick} for me on {Safari|Internet explorer|Chrome|Opera|Firefox}.
{Superb|Exceptional|Outstanding|Excellent} Blog!
|
These are {really|actually|in fact|truly|genuinely} {great|enormous|impressive|wonderful|fantastic} ideas
in {regarding|concerning|about|on the topic of} blogging.
You have touched some {nice|pleasant|good|fastidious} {points|factors|things}
here. Any way keep up wrinting.|
{I love|I really like|I enjoy|I like|Everyone loves} what you guys
{are|are usually|tend to be} up too. {This sort
of|This type of|Such|This kind of} clever work and {exposure|coverage|reporting}!
 
Keep up the {superb|terrific|very good|great|good|awesome|fantastic|excellent|amazing|wonderful} works guys I've {incorporated||added|included} you guys to {|my|our||my personal|my own} blogroll.|
{Howdy|Hi there|Hey there|Hi|Hello|Hey}! Someone in my {Myspace|Facebook} group shared this {site|website} with us so I came to {give it a look|look it over|take a look|check it out}. I'm definitely {enjoying|loving} the information.
I'm {book-marking|bookmarking} and will be tweeting this to my followers! {Terrific|Wonderful|Great|Fantastic|Outstanding|Exceptional|Superb|Excellent} blog and {wonderful|terrific|brilliant|amazing|great|excellent|fantastic|outstanding|superb} {style and design|design and style|design}.|
{I love|I really like|I enjoy|I like|Everyone loves} what you guys {are|are usually|tend to be} up too. {This sort of|This type of|Such|This kind of} clever work and {exposure|coverage|reporting}! Keep up the {superb|terrific|very good|great|good|awesome|fantastic|excellent|amazing|wonderful} works guys I've {incorporated|added|included} you guys to
{|my|our|my personal|my own} blogroll.|
{Howdy|Hi there|Hey there|Hi|Hello|Hey} would you mind {stating|sharing} which blog platform you're {working with|using}? I'm {looking|planning|going} to start my own blog {in the near future|soon} but I'm having a {tough|difficult|hard} time {making a decision|selecting|choosing|deciding} between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your {design and style|design|layout} seems different then most blogs and I'm looking for something {completely
unique|unique}.                  P.S {My apologies|Apologies|Sorry} for {getting|being} off-topic but
I had to ask!|
{Howdy|Hi there|Hi|Hey there|Hello|Hey} would you mind letting
me know which {webhost|hosting company|web host} you're {utilizing|working with|using}? I've
loaded your blog in 3 {completely different|different} {internet browsers|web browsers|browsers} and I must say this blog loads a lot {quicker|faster} then most.
 
Can you {suggest|recommend} a good {internet
hosting|web hosting|hosting} provider at a {honest|reasonable|fair}
price? {Thanks a lot|Kudos|Cheers|Thank you|Many thanks|Thanks}, I
appreciate it!|
{I love|I really like|I like|Everyone loves} it {when people|when individuals|when folks|whenever people} {come together|get
together} and share {opinions|thoughts|views|ideas}. Great {blog|website|site}, {keep it up|continue the good work|stick with
it}!|
Thank you for the {auspicious|good} writeup. It in fact was a amusement account it.
Look advanced to {far|more} added agreeable from you! {By the way|However}, how {can|could} we communicate?
|
{Howdy|Hi there|Hey there|Hello|Hey} just wanted to give you a quick heads up.
 
The {text|words} in your {content|post|article} seem to be running off
the screen in {Ie|Internet explorer|Chrome|Firefox|Safari|Opera}.
I'm not sure if this is a {format|formatting} issue or something to do with {web browser|internet browser|browser} compatibility but I {thought|figured} I'd
post to let you know. The {style and design|design
and style|layout|design} look great though! Hope you get the {problem|issue} {solved|resolved|fixed} soon.
 
{Kudos|Cheers|Many thanks|Thanks}|
This is a topic {that is|that's|which is} {close to|near to} my heart... {Cheers|Many thanks|Best wishes|Take care|Thank you}! {Where|Exactly where} are your contact details though?|
It's very {easy|simple|trouble-free|straightforward|effortless}
to find out any {topic|matter} on {net|web} as compared to {books|textbooks}, as I found this {article|post|piece of writing|paragraph} at this {website|web site|site|web
page}.|
Does your {site|website|blog} have a contact page?
I'm having {a tough time|problems|trouble} locating it but, I'd
like to {send|shoot} you an {e-mail|email}. I've got some {creative ideas|recommendations|suggestions|ideas} for your blog you might be interested in hearing. Either way, great {site|website|blog} and I look forward to seeing it {develop|improve|expand|grow} over time.|
{Hola|Hey there|Hi|Hello|Greetings}! I've been {following|reading} your {site|web site|website|weblog|blog} for {a long time|a while|some time} now and finally got the {bravery|courage} to go ahead and give you
a shout out from  {New Caney|Kingwood|Huffman|Porter|Houston|Dallas|Austin|Lubbock|Humble|Atascocita} {Tx|Texas}!
Just wanted to {tell you|mention|say} keep up the {fantastic|excellent|great|good}
{job|work}!|
Greetings from {Idaho|Carolina|Ohio|Colorado|Florida|Los angeles|California}!
 
I'm {bored to tears|bored to death|bored} at work so I decided to {check out|browse} your {site|website|blog} on my iphone during lunch break. I {enjoy|really like|love} the {knowledge|info|information} you {present|provide} here and can't wait to
take a look when I get home. I'm {shocked|amazed|surprised} at how {quick|fast} your blog loaded on my {mobile|cell phone|phone} .. I'm not even
using WIFI, just 3G .. {Anyhow|Anyways}, {awesome|amazing|very good|superb|good|wonderful|fantastic|excellent|great} {site|blog}!
|
Its {like you|such as you} {read|learn} my {mind|thoughts}!
You {seem|appear} {to understand|to know|to grasp} {so much|a lot} {approximately|about} this,
{like you|such as you} wrote the {book|e-book|guide|ebook|e book} in it or something.
{I think|I feel|I believe} {that you|that you simply|that you just} {could|can} do with
{some|a few} {%|p.c.|percent} to {force|pressure|drive|power} the message {house|home} {a bit|a little bit}, {however|but} {other than|instead of} that, {this is|that is} {great|wonderful|fantastic|magnificent|excellent} blog. {A great|An excellent|A fantastic} read. {I'll|I will} {definitely|certainly} be back.|
I visited {multiple|many|several|various} {websites|sites|web sites|web pages|blogs} {but|except|however} the audio {quality|feature} for audio songs {current|present|existing} at this {website|web site|site|web page} is {really|actually|in fact|truly|genuinely} {marvelous|wonderful|excellent|fabulous|superb}.|
{Howdy|Hi there|Hi|Hello}, i read your blog {occasionally|from time to time} and i own a similar one and i was just {wondering|curious} if you get a lot of spam {comments|responses|feedback|remarks}? If so how do you {prevent|reduce|stop|protect against} it, any plugin or anything you can {advise|suggest|recommend}? I get so much lately it's driving me {mad|insane|crazy} so any {assistance|help|support} is very much appreciated.|
Greetings! {Very helpful|Very useful} advice {within this|in this particular} {article|post}! {It is the|It's the} little changes {that make|which will make|that produce|that will make} {the biggest|the largest|the greatest|the most important|the most significant} changes. {Thanks a lot|Thanks|Many thanks} for sharing!|
{I really|I truly|I seriously|I absolutely} love {your blog|your site|your website}.. {Very nice|Excellent|Pleasant|Great} colors & theme. Did you {create|develop|make|build} {this website|this site|this web site|this amazing site} yourself? Please reply back as I'm {looking to|trying to|planning to|wanting to|hoping to|attempting to} create {my own|my very own|my own personal} {blog|website|site} and {would like to|want to|would love to} {know|learn|find out} where you got this from or {what the|exactly what the|just what the} theme {is called|is named}. {Thanks|Many thanks|Thank you|Cheers|Appreciate it|Kudos}!|
{Hi there|Hello there|Howdy}! This {post|article|blog post} {couldn't|could not} be written {any better|much better}! {Reading through|Looking at|Going through|Looking through} this {post|article} reminds me of my previous roommate! He {always|constantly|continually} kept {talking about|preaching about} this. {I will|I'll|I am going to|I most certainly will} {forward|send} {this article|this information|this post} to him. {Pretty sure|Fairly certain} {he will|he'll|he's going to} {have a good|have a very good|have a great} read. {Thank you for|Thanks for|Many thanks for|I appreciate you for} sharing!|
{Wow|Whoa|Incredible|Amazing}! This blog looks {exactly|just} like my old one! It's on a {completely|entirely|totally} different {topic|subject} but it has pretty much the same {layout|page layout} and design. {Excellent|Wonderful|Great|Outstanding|Superb} choice of colors!|
{There is|There's} {definately|certainly} {a lot to|a great deal to} {know about|learn about|find out about} this {subject|topic|issue}. {I like|I love|I really like} {all the|all of the} points {you made|you've made|you have made}.|
{You made|You've made|You have made} some {decent|good|really good} points there. I {looked|checked} {on the internet|on the web|on the net} {for more info|for more information|to find out more|to learn more|for additional information} about the issue and found {most individuals|most people} will go along with your views on {this website|this site|this web site}.|
{Hi|Hello|Hi there|What's up}, I {log on to|check|read} your {new stuff|blogs|blog} {regularly|like every week|daily|on a regular basis}. Your {story-telling|writing|humoristic} style is {awesome|witty}, keep {doing what you're doing|up the good work|it up}!|
I {simply|just} {could not|couldn't} {leave|depart|go away} your {site|web site|website} {prior to|before} suggesting that I {really|extremely|actually} {enjoyed|loved} {the standard|the usual} {information|info} {a person|an individual} {supply|provide} {for your|on your|in your|to your} {visitors|guests}? Is {going to|gonna} be {back|again} {frequently|regularly|incessantly|steadily|ceaselessly|often|continuously} {in order to|to} {check up on|check out|inspect|investigate cross-check} new posts|
{I wanted|I needed|I want to|I need to} to thank you for this {great|excellent|fantastic|wonderful|good|very good} read!! I {definitely|certainly|absolutely} {enjoyed|loved} every {little bit of|bit of} it. {I have|I've got|I have got} you {bookmarked|book marked|book-marked|saved as a favorite} {to check out|to look at} new {stuff you|things you} post�\
Posted in Programming | Tagged | Leave a comment

Experimental new C++ operator joke

This is fucking hilarious… Raymond Chen introduced some “new C++ tadpole operators” on his blog. It’s a joke, one that I didn’t get immediately, but what makes it even funnier is that many commenters also didn’t get it. They’re upset about the way it makes the code less readable and “solves a problem that doesn’t need solving” and similar sentiments. Mr Chen, you have an awesomely wicked sense of humour.

Posted in Programming | Tagged , | Leave a comment

Do NOT cache SSL certificates on the client side – How to post XML asynchronously and read the response using HttpClient in c#

I had one hell of a day yesterday. The evening before, I’d renewed the certificate on a WCF service of ours, because the old one was about to expire. After renewing it, I tested the service to confirm that I could still access it, now that it uses the new certificate. I could reach it from my notebook and my home computer; I could post requests to it, and it responded appropriately. All was well, so I went offline and proceeded to play Torchlight II, which is what I do in my spare time these days.

But things didn’t go so well for our major client… the one who sends thousands of requests every day, which come from their portal via their clients. They got this error:

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel

I wasted many hours on this yesterday, and was totally stumped. I confirmed that the service was accessible from many different computers, even their development machines, but not from their server. I’m not going to tell you what they did wrong exactly, because I have no access to their server, but it seems that somehow they cached our certificate on their server. How? I have no fucking idea…

To put it into perspective, here’s some c# code to post XML to an imaginary service which is very much like ours. It’s just a quick hack that I threw together in 5 minutes, and it has no error handling because this is just an example, but it works. Note that to use SSL, all it needs to do is specify https in the Uri. It shouldn’t be doing anything with the certificate.

What I did here was:

  1. In .Net framework version 4.5+, start a new Windows Forms application.
  2. Add a reference to the System.Net.Http assembly.
  3. Drop a button on the form, and name it btnTest.
  4. Add to the uses list of the form’s code-behind: using System.Net.Http;
  5. Double-click the button and add the code below.
        private async void btnTest_Click(object sender, EventArgs e)
        {
            string xmlContent = "Let's pretend this is valid XML, OK?";

            using (var client = new HttpClient())
            {
                var httpContent = new StringContent(xmlContent, Encoding.UTF8, "application/xml");
                var testUri = new Uri("https://www.YourServerName.com/AService.svc/DemoAction");
                var httpResponseMessage = await client.PostAsync(testUri, httpContent);

                if (httpResponseMessage.StatusCode == HttpStatusCode.OK)
                {
                    var messageContents = await httpResponseMessage.Content.ReadAsStringAsync();
                    MessageBox.Show(messageContents);
                }
            }
        }
Posted in Programming | Tagged , , , , | Leave a comment

Link to: Emulating double dispatch with c#

“Link to” in the title indicates that I am writing about something that somebody else wrote, somebody a lot smarter than myself.

Eric Lippert wrote a great post on emulating double dispatch in c#. Actually this is part 3 of a series, and I recommend reading the whole thing.

I’ve mentioned it before, but must reiterate: there’s something about his writing that makes me feel so damn inadequate every time. I haven’t gotten my head around that post yet, which is not unusual because I frequently need to read his posts more than once to understand them. But I do understand just enough to want to understand more… Enjoy!

Posted in Programming | Tagged | Leave a comment

My recovery blog is no longer accessible to the public, and never will be again

My recovery blog is no longer going to be publically accessible. I will eventually delete it, but need to retrieve the data from it first. I will be starting another blog, with a somewhat different focus: On recovery, atheism and skepticism. The tagline will be something like “The adventures of a skeptical ex addict in a credulous world”.

There are several posts on that blog that I will reuse on the new one; this is why I can’t delete it yet. Actually, I will probably, after creating the new blog, purchase a redirect upgrade to it, so all requests to the old blog will go to the new one, but the old posts will not be accessible.

Posted in Uncategorized | Tagged | Leave a comment

I hate you, Windows Update!

I have a deadline today, and as of last night, this computer froze midway when rebooting after a Windows Update. Now my SQL Server doesn’t start any more. This error is in the event log:

Cannot recover the master database. SQL Server is unable to run. Restore master from a full backup, repair it, or rebuild it. For more information about how to rebuild the master database, see SQL Server Books Online.

I tried doing a system restore, which made no difference; then tried rebuilding the master database, which also didn’t work. This is fucking brilliant… teaches me to run SQL Server 2008 R2 on Windows 8.1. So now I am downloading SQL Server 2012. Meanwhile I can’t make any progress on my work.

I hate you, Windows Update!

Posted in Work | Tagged | Leave a comment

Failed to establish a backside connection

We get this error occasionally when using a bank’s web service for AVS-R. (Real-time bank account validation.) Maybe I’m weird but I laugh every time I see that error. I read it as “Cannot find anus.”

Posted in Programming, Work | Tagged | Leave a comment

Two ways to change the default backup directory for SQL Server

This has driven me nuts for years.

I always change the default data directory in SQL Server Management Studio. That’s easy to do and the feature to do so is easily discoverable. Just right-click the server instance in the Object Explorer, and go to Properties. Then change it as per the screen shot below.

(This image looks crap because it is scaled. The dialog doesn’t allow resizing any smaller, so I had to scale it or have the image overlay the sidebar on the right of this blog. Clicking the image will open it with the correct size in a new tab.)

image

I like all my databases to be in C:\Data. No digging through directories. But it gives me two really annoying problems:

  1. The default location for backups is somewhere else, and I’d really like them to be here too, in the same directory as the data. So I have to browse to the correct directory whenever I make a backup.
  2. When I restore a backup, I want to overwrite the database. But when I select a backup to restore and go to the Options tab, one of the two files (the .LDF file) always defaults to the wrong file name – without a suffix of “_1”, even though I have the right data and log directories preselected and the log file already exists. So I have to manually type in the right file name or browse for it every time I restore a backup. This is painful.

I hoped that changing the default backup directory would fix this odd behaviour. (It does!) But how do you change this setting?

Method One

This setting to do so isn’t discoverable. I couldn’t even find it, and neither could the person who wrote this tip. Their solution, which is the one I used, is to change the registry setting, using regedit.exe, for the value found at:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\MSSQLServer

Here’s what my system’s registry looks like:

SNAGHTML6d886e19

Note that the key will be slightly different depending on the version of SQL server that you have installed. I’m running SQL Server Express 2008 R2, 64-bit, which corresponds with the sub-string MSSQL10_50.SQLEXPRESS. The rest of the key stays the same so it shouldn’t be difficult to find what key applies to your particular version.

Method Two

But it turns out there is a user interface to change this. Right-click on the server instance in Object Explorer and then clicks Facets. Then you can change the BackupDirectory value in the user interface, as indicated in the screen shot below. I don’t know why they didn’t put this setting together with the data and log directories, because they really need to be together to avoid the odd behaviour of the log file name being wrong when restoring backups.

image

Posted in Database, Work | Tagged , | Leave a comment

Zipping a directory and including relative directory paths is non-intuitive with DotNetZip

I recently had to write some code to auto-update an application via an FTP server. It uses home-grown code since ClickOnce is too limiting, and is written in a Windows Forms .NET 3.5 Visual Studio 2010 project. It also had to back up existing files before updating, and I was forced to use DotNetZip, which the project already references. (In .NET 4.5 and up, I’d much rather use System.IO.Compression.ZipArchive since I don’t require a password-protected archive. I’ve even written some decent extension methods to read and write zip files asynchronously that way.)

For some reason, when you zip a directory with DotNetZip, by default rather than it using a sensible relative path for each file added, which it could infer from the paths of the files added, when you call the ZipFile.AddDirectory method, it uses the entire hierarchical path structure from the root of the drive. This Stack Overflow answer didn’t solve my problem because it throws away the path info for each file, putting them all in the root of the zip file. I want the recursive path in the zip file entries, just not the whole damn path. This seems to me like a simple requirement, and one I would expect to be supported out of the box. Yet it’s not only unsupported, but also none of the examples on their wiki behave this way.

Here is my solution, which zips everything in the given directory recursively, other than zip files. Rather than adding the directory with a single method-call, it adds the individual files in a loop, and for each file, specifies a directory in the zip file relative to the specified root directory, by simply replacing the root directory part of each path string with an empty string.

Edit: Wrapped the code more so it’s easier to read. It makes this look like more code though… actually it’s very little.

using System.IO;
using System.Linq;
using Ionic.Zip;
using Ionic.Zlib;

namespace Demo
{
    static class Example
    {
        public static void BackupDirectory(string directory)
        {
            using (ZipFile zip = new ZipFile
            {
                CompressionLevel = CompressionLevel.BestCompression
            })
            {
                var files = Directory.GetFiles(directory, "*",
                    SearchOption.AllDirectories).
                    Where(f => Path.GetExtension(f).
                        ToLowerInvariant() != ".zip").ToArray();

                foreach (var f in files)
                {
                    zip.AddFile(f,
                        Path.GetDirectoryName(f).
                        Replace(directory, string.Empty));
                }

                zip.Save(Path.ChangeExtension(directory, ".zip"));
            }
        }
    }
}
Posted in Programming | Tagged , | Leave a comment

How to implement IDisposable properly

Every so often I write a class with disposable members, so it needs to implement IDisposable. I always copy and paste from one of my other classes, so I figured it would be useful to share the pattern of IDisposable alone in a simple class that does nothing else.

This example has a single disposable member, a BackgroundWorker, only because that’s what my latest class happened to use. Actually I wouldn’t recommend using this component anymore. The Task-based Asynchronous Pattern (TAP) is much easier to use once you get used to it, but this was a .Net 3.5 project, and a BackgroundWorker was the easiest way to do some funky progress stuff easily in its ProgressChangedEventHandler event handler…

Also there are other reasons BackgroundWorker is a bad example… Namely that it doesn’t need to be disposed and also, how do you know when to dispose it? (My “real” class is used at system start-up. It creates the worker only when the method is called; then gets disposed immediately after. The problem obviously is, the calling code can’t use it in a using statement because it will free the worker before the method can use it. I raised an event when its work was complete. Not shown here.) Anyway, this is just an example. OK? Pretend the BackgroundWorker is some other managed disposable type, as it doesn’t really matter as far as the pattern is concerned.

I’m not going to explain the code. Anybody who can read should be able to understand it.

using System;
using System.ComponentModel;

namespace Example
{
    public class Disposable : IDisposable
    {
        private BackgroundWorker worker;
        private bool disposed;

        ~Disposable()
        {
            Dispose(false);
        }

        public void DoSomething()
        {
            worker = new BackgroundWorker();

            // Examples of using BackgroundWorker not shown.
            // You could set up your event handlers here.
        }

        protected virtual void Dispose(bool disposing)
        {
            if (!disposed)
            {
                if (disposing)
                {
                    // Release managed resources.
                    if (worker != null)
                    {
                        worker.Dispose();
                        worker = null;
                    }
                }

                // Free unmanaged resources
                // Nothing unmanaged to free in this example.

                disposed = true;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }
}
Posted in Programming | Tagged , | Leave a comment

SA Mobile Drug Testing… Located somewhere in the tropical Atlantic ocean off the Gulf of Guinea?

This is what happens when you let clueless developers build a website…

I was searching for local drug testing options when I found this… Surprising that their top two results for drug testing in Johannesburg are in Cape Town and Durban respectively, but to make matters worse, apparently their Durban mobile clinic can only be reached by ship, and is to be found (floating?) in the tropical Atlantic ocean, off the Gulf of Guinea. (Yes, geography was never my strong suit – I had to Google “Gulf of Guinea”.)

image

(Click the preview above to open the full size screen shot in a new tab.)

Posted in Programming | Tagged | Leave a comment

So funny: I never thought of Bobby Tables this way

I just found this in a comment to Raymond Chen’s latest blog post. I hadn’t seen it before and it’s too funny not to share.

Posted in Programming | Tagged | Leave a comment

A pattern for implementing the == operator in C#

I’ve been neglecting this blog for a while because I have very little time to write here…

Every now and then I need to implement the == operator on one of my classes; typically because I want to compare two instances of a custom type that can’t be compared automatically. So I always copy and paste more or less the same code. I figured I may as well share it. This is, of course, an oversimplified example. It’s just intended to show the pattern.

In the code that follows, the pattern is quite simple:

  1. Implement a public Equals method that takes a parameter of the same type as your class. In that method, compare whatever it is you consider a reasonable representation of equality between instances. (In this trivial example, I just compare properties. You might want to do something more, but try not to do anything overly complex. Nobody expects the code to perform processor-intensive calculations while comparing two objects.)
  2. Override object.Equals and internally call your strongly typed method if the object passed is of the same type; otherwise return false (because you don’t compare apples and oranges).
  3. Override the == and != operators, which need only call your strongly typed Equals method.
  4. Since you implemented == and !=, you should also override object.GethashCode.

Here’s the code:

namespace EqualsExample
{
    public class Example
    {
        public int Number { get; set; }
        public string Text1 { get; set; }
        public string Text2 { get; set; }

        public static bool operator !=(Example example1, Example example2)
        {
            return !example1.Equals(example2);
        }

        public static bool operator ==(Example example1, Example example2)
        {
            return example1.Equals(example2);
        }

        public override bool Equals(object obj)
        {
            var other = obj as Example;

            if (other != null)
                return this.Equals(other);

            return false;
        }

        public bool Equals(Example other)
        {
            return this.Text1 == other.Text1 && this.Text2 == other.Text2 && this.Number == other.Number;
        }

        public override int GetHashCode()
        {
            return this.Text1.GetHashCode() ^ this.Text2.GetHashCode() ^ this.Number;
        }
    }
}

One thing that’s worth pointing out is the first line of the overridden object.Equals method, which uses the as operator to cast to the appropriate type. That will return null if the object is not the correct type. If you were writing a value type (and Example was a struct rather than a class), you could not use the as operator, and would have to do this:

var other = (Example)obj;

Then again, maybe you don’t want the code to throw an InvalidCastException… Then use the is operator. I don’t because I don’t want to cast twice, and I tend not to pass the wrong types for comparison in my own code.

Posted in Programming | Tagged , , | Leave a comment

Stupid Firefox is pissing me off

Every time I navigate to any page that uses flash (on either my home or work computer thanks to Firefox sync) I get this:

Annoying01

So I went to look at my plugins. I get this:

Annoying02

OK, so click the Update Now link…

Annoying03

Right… version 11 to 11.7 have been disabled. I am using the latest (at this point in time) version, and last time I checked, 15 > 11.x. For those who do not understand the > (greater than operator), last time I checked, fifteen was greater than eleven. Maybe Firefox math is different?

Seriously? Idiots.

I don’t even like flash, and would prefer to use HTML5 everywhere, as I can do on YouTube, but some sites that I visit still use flash.

Posted in Uncategorized | Tagged , | Leave a comment

A nice way of handling timesheets

I mentioned a while back that I have started using Toggl for my time sheets…

What I did this month, which was the first month I sent the Toggl-based report to my boss, was export a detailed report, then remove some columns not needed in Excel, and gave my boss that. And he wasn’t happy, because this time it included too much detail. (I work on multiple projects, so I tend to reuse work items several times per day as I switch between projects. This creates detail entries with hundreds of lines and many duplicate tasks per day. What I need to do is aggregate them into work items per day, with a duration for each.)

So I have a procedure that I’ll be using going forward for my time sheets. Since it might be useful to others, and it doesn’t give away anything confidential about my employment there, I thought I’d share it. (Then anyone else who uses Toggl can use this technique.)

  1. First export a detailed report from Toggl, as CSV.
  2. Open it with Excel and save it as an Excel file.
  3. In a new SQL database, run the import data wizard. (Right-click your database, then from the menu choose Tasks –> Import Data).
  4. Choose Excel as the data source and browse for your file. Something like this:Import-1
  5. Click Next a couple of times, and then choose to Copy data from one or more tables or views. Something like this:Import-2
  6. On the next page, click Edit Mappings. This page:Import-3
  7. This is important. The fields that have a time only get imported as type DATETIME, which we do not want. I only really select one of them, but I change all of them to the TIME type anyway. That is, change the value of the Type column to TIME for the fields Start time, End time and Duration. Like so:Import-4
  8. That will create a table in your database with all the time entries in it.
  9. I normally export everything, so I can just run a query as I need to get the details that are pertinent to my timesheet, for a date range. I rename this imported generated table to TimeEntries so that the stored procedure (below) can then be used to get the results that I need.

Here’s my stored proc. (My database is named Timesheets.)

USE Timesheets

GO

IF EXISTS (SELECT * FROM sys.objects 
           WHERE type = 'P' AND name = 'spGetTimeSheet')
DROP PROCEDURE spGetTimeSheet

GO

CREATE PROCEDURE spGetTimeSheet(
    @StartDate DATETIME, @EndDate DATETIME) AS
SELECT 
    ISNULL(Project, '') [Project], 
    Description, 
    CAST([Start date] AS DATE) [Date],
    CAST(
        DATEADD(ms, SUM(DATEDIFF(ms, '00:00:00', duration)), 
        '00:00:00') AS TIME) [Duration]
FROM 
    TimeEntries
WHERE 
    CAST([Start date] AS DATE) >= @StartDate AND 
    CAST([Start date] AS DATE) < @EndDate
GROUP BY 
    CAST([Start date] as date), project, Description
ORDER BY 
    Project, CAST([Start date] AS DATE), Description
    
GO

Execute it like so:

exec spGetTimeSheet '27 September 2014', '27 October 2014'

Then it’s simply a matter of copying the query results into Excel, and adding a total to the Duration column. The only tricky bit there is to format the cell that contains the total as  [h]:mm:ss; otherwise it will not display more than 24 hours.

Of course this is just an example; you might need to select different columns from the imported table, but this is what works for me. The difficult part (in the stored procedure) was figuring out how to sum the Duration column, because SUM(time column) is invalid SQL, but StackOverflow came to my rescue on that one…

Posted in Programming, Work | Tagged , , | Leave a comment

An asynchronous implementation of File.WriteAllLines

This is an example that (optionally) uses an extension method I wrote about back in April 2013. See the original post here. Last time around the post was about the caveats involved when working with streams. This time, I’m going to assume you know all about that, and not start by describing what you shouldn’t do…

The easy way

First, here’s the short answer. This is all you need for an asynchronous implementation of File.WriteAllLines() without using my extension method.

using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Example
{
    public static class Constants
    {
        public const int BufferSize = 0x2000;
    }

    public static class FileAsync
    {
        /// <summary>Asynchronously creates a new file, writes a collection
        /// of strings to the file, and then closes the file.</summary>
        /// <param name="path">The file to write to.</param>
        /// <param name="contents">The lines to write to the file.</param>
        /// <returns>A Task that represents completion of the method.</returns>
        public static async Task WriteAllLinesAsync(string path, IEnumerable<string> contents)
        {
            await WriteAllLinesAsync(path, contents, Encoding.UTF8);
        }

        /// <summary>Asynchronously creates a new file by using the specified encoding,
        /// writes a collection of strings to the file, and then closes the file.</summary>
        /// <param name="path">The file to write to.</param>
        /// <param name="contents">The lines to write to the file.</param>
        /// <param name="encoding">The character encoding to use.</param>
        /// <returns>A Task that represents completion of the method.</returns>
        public static async Task WriteAllLinesAsync(string path, IEnumerable<string> contents, Encoding encoding)
        {
            using (var memoryStream = new MemoryStream(contents.SelectMany(s => encoding.GetBytes(s.EndsWith("\r\n") ? s : s + "\r\n")).ToArray()))
            {
                using (var stream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None, Constants.BufferSize, true))
                {
                    await memoryStream.CopyToAsync(stream, Constants.BufferSize);
                }
            }
        }
    }
}

As you can see, all that’s needed is to call the built-in Stream.CopyToAsync() method.

The only tricky part was to figure out how to convert an IEnumerable<string> such as a string array, into a byte array, in order to pass that to a MemoryStream constructor. Then simply copy the memory stream to a new file stream asynchronously.

The slightly harder way

(If you read the old post, you don’t need to read this. But if you’re lazy like me, here’s the extension method I wrote about last time.)

That code should be suitable most of the time. But for whatever reason, in my personal code that copied large streams often, not text files but image streams, especially when the images could be huge Photoshop files or Bitmaps, the built-in CopyToAsync() method just wasn’t reliable for me. (Or maybe I told myself so because writing a stream-to-stream copying method was more interesting.) So I implemented my own version, and I now always use that one because it has proved to be reliable. So I’ll now describe how to write your own Stream.CopyToAsync alternative method.

Also, this code can be used for many other async helper methods besides writing files asynchronously. An async WriteAllLines implementation is just one example. Another application of this technique that I use is reading streams asynchronously, and caching image contents, as well as keeping copies of image streams in the undo stack of my rudimentary image editor. Of course you could also go low-level and implement the Stream methods asynchronously yourself for even better control of them, but I am far too lazy for that, and copying streams asynchronously using a memory stream helper object suits all my needs pretty well.

The first method below is just a helper method, to format a size in bytes in a more friendly way, similarly to the way file sizes are displayed in Windows. This is only used for an informational message if the code needs to throw an exception.

public static class StreamExtensions
{
    private static string FormatBytes(long bytes)
    {
        const long KiloByte = 1024L;
        const long MegaByte = KiloByte * KiloByte;
        const long GigaByte = MegaByte * KiloByte;
        const long TeraByte = GigaByte * KiloByte;
        const long PetaByte = TeraByte * KiloByte;
        const long ExaByte = PetaByte * KiloByte;

        var formattedBytes = string.Empty;

        if (bytes < KiloByte)
            formattedBytes = string.Format("{0:F2} bytes", bytes);
        else if (bytes >= KiloByte && bytes < MegaByte)
            formattedBytes = string.Format("{0:F2} KB", Math.Round((double)bytes / KiloByte, 2, MidpointRounding.AwayFromZero));
        else if (bytes >= MegaByte && bytes < GigaByte)
            formattedBytes = string.Format("{0:F2} MB", Math.Round((double)bytes / MegaByte, 2, MidpointRounding.AwayFromZero));
        else if (bytes >= GigaByte && bytes < TeraByte)
            formattedBytes = string.Format("{0:F2} GB", Math.Round((double)bytes / GigaByte, 2, MidpointRounding.AwayFromZero));
        else if (bytes >= TeraByte && bytes < PetaByte)
            formattedBytes = string.Format("{0:F2} TB", Math.Round((double)bytes / TeraByte, 2, MidpointRounding.AwayFromZero));
        else if (bytes >= PetaByte && bytes < ExaByte)
            formattedBytes = string.Format("{0:F2} PB", Math.Round((double)bytes / PetaByte, 2, MidpointRounding.AwayFromZero));
        else if (bytes >= ExaByte)
            formattedBytes = string.Format("{0:F2} EB", Math.Round((double)bytes / ExaByte, 2, MidpointRounding.AwayFromZero));

        return formattedBytes;
    }
    /// <summary>An implementation to copy asynchronously from one stream to another,
    /// similar to <see cref="System.IO.Stream.CopyToAsync(Stream)"/></summary>
    /// <remarks>This was written because the default implementation would sometimes throw an OutOfMemoryException.</remarks>
    public static async Task CopyToStreamAsync(this Stream source, Stream destination, int bufferSize)
    {
        if (source == null)
            throw new ArgumentNullException("source");

        if (destination == null)
            throw new ArgumentNullException("destination");

        if (bufferSize <= 0)
            throw new ArgumentOutOfRangeException("bufferSize", "bufferSize must be greater than zero");

        /* The source stream may not support seeking; e.g. a stream
         * returned by ZipArchiveEntry.Open() or a network stream. */
        var size = bufferSize;
        var canSeek = source.CanSeek;

        if (canSeek)
        {
            try
            {
                size = (int)Math.Min(bufferSize, source.Length);
            }
            catch (NotSupportedException) { canSeek = false; }
        }

        var buffer = new byte[size];
        var remaining = canSeek ? source.Length : 0;

        /* If the stream is seekable, seek through it until all bytes are read.
         * If we read less than the expected number of bytes, it indicates an
         * error, so throw the appropriate exception.
         *
         * If the stream is not seekable, loop until we read 0 bytes. (It's not
         * an error in this case.) */
        while (!canSeek || remaining > 0)
        {
            var read = await source.ReadAsync(buffer, 0, size);

            if (read <= 0)
            {
                if (canSeek)
                    throw new EndOfStreamException(
                        string.Format("End of stream reached, but {0} remained to be read.",
                        FormatBytes(remaining)));
                else
                    break;
            }

            await destination.WriteAsync(buffer, 0, read);
            remaining -= canSeek ? read : 0;
        }
    }
}

The CopyToStreamAsync method is the one that does the work. All it really does is use Stream.ReadAsync and Stream.WriteAsync in a loop, with a bit of code to allow for streams that don’t support seeking. (This code is unchanged from my original post. For a better explanation of what it does and the issues it avoids, see my original post on this subject.)

To use this in the WriteAllLinesAsync method is a one-word change since it has the same signature as CopyToAsync, so I won’t bother to paste it here again.

Posted in Programming | Tagged | Leave a comment

Feeling like an imposter again

Sheesh… I read a few programming blogs, but one of them always ends up making me feel really stupid. Eric Lippert, I hate you! … OK, maybe I don’t, but the guy writes the most beautiful, concise code I have ever seen, and every time I read it, even if it’s the silliest, most simple example, I have to read it again, and again before I understand it. It makes me feel really dumb every time.

I do read quickly, and most of the time, whether I’m reading one of my favourite sceptic’s blogs or about the latest in medicine, I do understand it right away, but something about Mr Lippert’s posts is always just out of my brain’s reach, and that’s ironic because C# development is what I specialize in. I don’t know if this is a case of the imposter syndrome or if I am really just not as intelligent as I should be, but it always intimidates me…

Here’s a link to his latest post that makes me feel dumb. Even his English is better than mine; in terms of both grammar and vocabulary. In a comment to a question today, he stated that the algorithm he presented was “the most pedagogically interesting”. I had to look up “pedagogically”.

So if you too read programming blogs and come away from them feeling more stupid, rather than wiser, you are not alone.

Posted in General, Programming | Tagged | Leave a comment

Thank you, Word auto recovery. Gosh, which one shall I use?

Perplexing… My notebook crashed on Friday, and I just returned to this Word document now. Word conveniently saved two versions of the document for me, including the original draft, apparently written some 370 years before I was born, and 51 years before Jan Van Riebeeck sailed to sunny South Africa.

DocRecovery

Anyway, I did some digging (very little), and the significance of the weird date is explained on Raymond Chen’s blog here. It’s a short post, so here it is quoted verbatim:

Public Service Announcement: This weekend marks the start of Daylight Saving Time in most parts of the United States.

The FILETIME structure records time in the form of 100-nanosecond intervals since January 1, 1601. Why was that date chosen?

The Gregorian calendar operates on a 400-year cycle, and 1601 is the first year of the cycle that was active at the time Windows NT was being designed. In other words, it was chosen to make the math come out nicely.

I actually have the email from Dave Cutler confirming this.

In English, what that means is that the date is an uninitialized Win32 FILETIME structure, so the original date was somehow lost, and the Office developers didn’t test for that variable being undefined. (Hey, at least it didn’t crash. And they did correctly convert the date to a local date time for this country.)

So if you see that same confusing date anywhere, now you know…

Posted in General, Programming | Tagged , | Leave a comment

I finally found a good timesheet system

Not that I ever looked for one before, but never mind.

One thing I have never been able to get right is timesheets. Never. This has caused some problems for me at work… Lately I have to submit my timesheets on the 26th of each month. Each entry on the timesheet has the date, starting and ending time, and a calculated time duration. That is, there are multiple tasks for every day. Since I haven’t been keeping it up to date, at the end of last month, it took me three hours to “remember” my time sheet, using my sent email, subversion commits etc… It was a disaster… I’d be like, “I committed these six things on the fourth. But when did I actually work on them?” Never again.

I found a very good time-keeping web app called Toggl. Now I just add tasks, giving each a name, and click a button for them to record. (They are reusable too. If you click the play button on a task from yesterday, it will create an entry for today.) If I forget to stop it at the end of the day when I go home, I just click Stop in the morning, and then modify the start and end time. It also has a CSV export, which gives me almost exactly what I need to create my Excel timesheet, including all the ad hoc queries and other little tasks that I’d forgotten about. I highly recommend it.

Posted in Work | Tagged | Leave a comment

Memory gates checking failed because the free memory (n bytes) is less than 5% of total memory.

Ever had this horrendous error?

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/45653674
 Exception: System.ServiceModel.ServiceActivationException: The service '/YourServiceName.svc' cannot be activated due to an exception during compilation.  The exception message is: Memory gates checking failed because the free memory (186429440 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (186429440 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.
   at System.ServiceModel.Activation.ServiceMemoryGates.Check(Int32 minFreeMemoryPercentage, Boolean throwOnLowMemory, UInt64& availableMemoryBytes)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CheckMemoryCloseIdleServices(EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   --- End of inner exception stack trace ---
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)
   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath, EventTraceActivity eventTraceActivity)
 Process Name: iisexpress
 Process ID: 992

If you don’t want to scroll, the relevant part is:

Memory gates checking failed because the free memory (n bytes) is less than 5% of total memory.

That is indeed an ugly error to see. Its precursor is that you sit waiting all day for your computer to do anything at all. Switching between windows becomes a 30 minute task. It also takes about 30 minutes to write a single line of code, if you’re lucky. (And 10 minutes before the debugger launches your code.)

I’m getting more RAM tomorrow, but should you also need to run a webservice despite all your common sense telling that there isn’t enough memory to do so, just add this to the system.serviceModel section of your config file:

<serviceHostingEnvironment multipleSiteBindingsEnabled="true" minFreeMemoryPercentageToActivateService="0" />

Again, if you are as lazy to scroll as I am to wrap that to more than one line, the relevant part is:

minFreeMemoryPercentageToActivateService=”0″

Posted in Programming, Work | Tagged , , | 2 Comments

How to force (hack?) kendo-ui grid to display the correct date

We have some grids displaying the data from SQL queries, including dates. But for whatever reason, the time zone is being applied to the date twice. For example, the date is today, at 12PM, and we are using GMT-2, here in South Africa. The data comes back correctly from the server, but displays on the client side in the kendo grid as 2PM instead. So all dates are displayed 2 hours off.

(Come to think of it, the time zone is not being applied at all, yet the date appears to be in the correct format. All my code below does, is to force the time zone to be applied correctly. This should be unnecessary.)

Edit: Aargh! These dates can be very confusing… I always forget whether we are GMT plus 2 or minus 2… It turns out, we are GMT+2. So kendo is indeed applying the time zone a second time. I was correct in the first place. My hack below works, but I still don’t believe it should be necessary.

Here’s my solution, although my colleague doesn’t like it:

            { field: 'd', 
              type: "date", 
              title: 'Date Captured', 
              template: '#= kendo.toString(
                new Date(data.d.getTime() + 
                    data.d.getTimezoneOffset()*60000), 
                "dd/MM/yyyy h:mm:ss tt") #', 
              width: 150},

All it does is, use JavaScript to add the time zone offset to the date, which effectively subtracts 2 hours here, correcting kendo’s weirdo automatic-incorrect-formatting of the date.

Update: An alternative solution

This one really depends on whether you use Microsoft SQL Server, as we do. When we input the data, we use the SQL GETDATE() function…

The alternate solution is simply to use GETUTCDATE() instead. Then allow the kendo grid to add the time offset as it does, which ends up displaying the correct date. i.e. since both my server and client are here in South Africa, the server will now store the date minus two hours, and the client will add on two hours.

I’m still not comfortable with this though… As I see it, using standard date-time functions, the local date should always be correct. The kendo grid is wrong to always add the time zone offset to the date, because in so doing it is assuming that the date is a UTC date, which it usually will not be.

Posted in Programming | Tagged , | 5 Comments

Bootstrap’s modal jQuery dialog doesn’t have to be dismissed when the user clicks elsewhere

We have a Node server running, developed using the express framework and a few other components, all new to me. I’m no Node expert… actually I hate it, and estimate it takes me more than five times longer to develop anything in this mess (using IntelliJ Idea IDE) as I skip from file to file, writing bits and pieces of code here, there and everywhere without a fluid workflow at all. (The other developer here wrote the system. I’m just adding new features and helping fix bugs.)

We have a nasty issue with duplicated requests being sent through to the server. It seems to be related to user workflow… i.e. tab through the controls and then hold down the Enter key (with the submit button focused), and you can get 20 to 30 requests separated by milliseconds. (This is costly as the request in question does a bank account verification, with the bank, which is charged for.)

The obvious fix here, and I don’t know for sure if this is the real issue, is to disable the buttons immediately after they are clicked.

We also use jQuery modal dialogs, of the Bootstrap variety. I’m now firing off a modal dialog that will remain on the screen after the user clicks the submit button, and until the request gets handled by the server. One thing that left me somewhat mystified in all this, is even in the examples online, you can dismiss all the modal dialogs, just by clicking somewhere else. This behaviour is not exactly what I would want, to say the least…

But the solution is right there in that page I linked to…

$('.modal-loading-data').modal({ show : true, keyboard : false, backdrop : 'static' });

In case you are too lazy to scroll to the right… backdrop : ‘static’ is the solution. This prevents the modal dialog from fucking off before you tell it to. Excuse the language, but the default option doesn’t make any sense to me.

Posted in Programming | Tagged , | Leave a comment