If an operation seems extremely common, check if it's done for you

I've been working with an application that uses an SQL Anywhere 11 database. Like hopefully all SQL implementations, SQL Anywhere supports autoincrementing columns. Instead of taking advantage of that, though, this application maintains a table of the next available primary key value for each table. To generate IDs for new rows, it calls upon a stored procedure.

The application has been around for a long time, and I'm sure if the authors were developing it today, they would make a better choice. I've certainly rigged up systems that were weirdly engineered or not necessary. Something I've learned is that very general, very useful functionality is probably already implemented in the platform. If it's implemented in a well developed, widely used piece of software, it is likely a solid solution. That's not always the case, but it's at least worth checking over. Before deciding to reimplement something already done in the platform, you should have a specific reason, like performance or a missing capability.

Test mocks are definitely worth it

I'm currently working on an application that interacts with a system outside the machine it runs on - specifically, it sends and receives text messages through an Internet SMS gateway. That gets rather unwieldy in test environments, since obviously I can't use real people's data, and I only have one or two phones available to me at any given time. Besides, using the SMS gateway costs money.

It used to take me a while to get the SMS gateway connected for tests, and waiting for the texts to go through one direction and back the other also took up several moments, but it was never too much of an inconvenience. Then one day while planning a large round of tests on new features, I decided to finally sit down and write a mock for the gateway. Instead of actually sending the texts out to the Internet, this message provider (a drop-in replacement for the real library) just puts them up in a window. Instead of me needing to text back on a phone, I just type the "from" number and the text into some fields and click a button.

This is fast and exceptionally convenient compared to the old system. I wasn't sure before creating it whether it would be worthwhile, but it unquestionably was. Of course, this presupposes an architecture that allows easy drop-in replacement of the component being mocked, but if you have that infrastructure, strongly consider making a test mock.

The For-If Anti-Pattern

This isn't anything new in terms of coding advice (you can find stuff on the Internet about it everywhere), but I just caught myself using the for-if anti-pattern. The term anti-pattern refers to a coding style (a pattern) that is suboptimal or flawed in some way. For-if is an inefficiency that fails to leverage the given lookup or access mechanism and instead enumerates every single entry in a collection, checks to see whether it has the right ID, and then does something with it. In obvious terms:

For n = 0 to 300
 If n = CInt(userChoice) Then
  Console.WriteLine(messageTable(userChoice))
 End If
Next

It's obvious here; the above code could be:

Console.WriteLine(messageTable(userChoice))

And the message table is accessed directly. I noticed that I had been doing this with the Properties collection of DirectoryEntry:

For Each p as PropertyValueCollection In GetCurDir.Properties
 If p.PropertyName = Arguments(1) Then
  val = p(0)
 End If
Next

It hit me that I could have simply written:

val = GetCurDir.Properties(Arguments(1)).Value(0)

Moral of the story: check to see whether your collection supports direct accessing. You should definitely not be looping through a Dictionary to find a pair with a certain key and extract the value from the KeyValuePair - get the value directly by passing the key to its default property!

How to Actually Get the Active Directory Date Information out of __ComObject in .NET

If you're familiar with the .NET LDAP library (System.DirectoryServices), you may have noticed that some entries in the Properties collection for a given DirectoryEntry are instances of System.__ComObject. This is really inconvenient because there's no way to tell what type of thing such an object really is. Even when you know what it should be, it's kind of a pain to extract the data.

For instance, the pwdLastSet and lastLogon properties of user objects show up as __ComObject despite being date-time things. Specifically, they (and most AD objects that you expect to be date-times) are IADsLargeInteger. To convert them into normal DateTime objects, you need a reference to a COM interop DLL in your project, specifically the "Active DS Type Library".

Let's say you have a __ComObject that is supposed to be a date-time, I'll call it p. Use the following code to get it as a nice .NET DateTime:

                Dim li As ActiveDs.IADsLargeInteger = p.Value
                Return DateTime.FromFileTimeUtc((CLng(li.HighPart) << 32) + li.LowPart)

Collection Performance in .NET

I had to do a mock lab report for Chemistry class (it didn't have to be about chemistry, just use the normal scientific method and report style), so I quickly threw together a paper/experiment analyzing the performance of a few collections in .NET. It's obviously nothing shocking, but could possibly serve as a nice reference for combining the scientific method with computer science.

Download paper (PDF)

Restricted Interfaces, Please

In Abiathar's extension API, there are two or three interfaces that would mess everything up if implemented by something other than Abiathar Core. They need to be interfaces because I don't want the extensions to need a reference to Abiathar's internal details, which may change drastically. It wouldn't really be a problem unless objects of the interface types didn't have to be passed back through the API, which they do. (Level wrappers.)

Also, in F#, Clojure, and (maybe) Haskell, there is an Option type that may or may not be an instance of Some, which contains a real value. Option is also subclassed by None, which is kind of the entire point of being an option type. Now, it's possible to inherit Option with some other class that breaks all the logic. This can't be stopped because Option has to be inheritable by Some and None to have an option.

What I really want is some way to restrict which assemblies can implement an interface or inherit a type. I at first thought adding a line like "InheritableBy etc" to the definition would work to allow only those types to inherit it, but this significantly increases compiler/parser complexity. Also, it would be difficult to make it not break when used in different assemblies (would a reference to the original be needed? that would defeat the purpose).

Super Inline Functions?

I did some more thinking about what would be awesome to have in .NET and remembered something about my new implementation of SupaChat Server. For a lot of the commands, it tries to find a group object for a group ID specified somewhere in the arguments array. Therefore, I have to have code like this in those case blocks:

Dim TargetGroup As Group = Groups.Find(Function(g) g.ID = _
 Args(1))
If TargetGroup Is Nothing Then
 Sender.SendData("*G* Group doesn't exist!")
 Return
End If
If Not Sender.RepUser.InGroup(TargetGroup) Then
 Sender.SendData("*G* You're not there!")
 Return
End If

In addition to being simplified by the breaker subs I talked about last time, this is kind of a pain and is very repetitive. I would like to always define variables for things like the target group, target user, and sending rank and have them only set by one line - a "set-up" call if you will. However, that would require a new kind of inline sub. It's hard to explain, but I was thinking of something like this:

Breakable Sub HandleCommand(Command As String, Args() As String)
 Dim TargetGroup As Group
 ' other variables
 Inline Breaker(1) Sub GetGroup(ByInline TargetGroup As Group, _
  ArgPos As Integer, NeedsPresent As Boolean)
  TargetGroup = Groups.Find(Function(g) g.ID = Args(ArgPos))
  ' etc from code in last section
 End Sub
 Select Case UCase(Command)
  Case "LEAVEGROUP"
   GetGroup(1, True)
   ' other instructions
  ' other commands
 End Select
 ' other stuff
End Sub

That "inline" sub inside the command handler sub is only visible and runnable in the context of that one sub. Variables passed ByInline pass a double-strong pointer, so setting where it points inside the inline sub would change what it points to back in the root sub. Inline subs must appear after all local variables they use. When invoking inline subs, ByInline parameters cannot be specified; they are taken from the current accessible variable set.

This would be an amazing feature. Get on it, Microsoft!

Force Multiple Returns - "Breakers"

I would really, really like a way in .NET to unwind the stack twice in a return-like statement. This would be exceedingly useful in request handling methods that check some conditions and exit the big handler if the check fails. There is currently no way to do this without inserting "if check is false then exit" everywhere. It's even more of a pain in ASP .NET applications that require the request to be aborted if the page will not be fully served.

Sub Page_Load() Handles Me.Load
 If Not IsUserAdmin(Session("UserID")) Then
  Response.Redirect("PermissionDenied.html")
  Exit Sub
 End If
 ' actual page loading stuff
End Sub

Here, it would be excellent if the check function could cancel the Page_Load routine. (It might be necessary to pass the Response variable to an auth function.) I was thinking something like this:

Breaker(1) Sub EnsureUserAdmin(ID As Integer, _
 Handler As HttpResponse)
 If Not IsUserAdmin(ID) Then
  Handler.Redirect("PermissionDenied.html")
  Break 1
 End If
End Sub

Breakable Sub Page_Load() Handles Me.Load
 EnsureUserAdmin(Session("UserID"), Response)
 ' administrative things
End Sub

The parameter to the Breaker keyword specifies how many frames up the stack the method can break. The Breakable keyword flags the method as being able to use breaker methods.

I realize it's kind of redundant for void returners as they could be changed into Booleans, but for functions that can actually have a useful output, it would be a real pain to use a composite/nullable structure. Please, Microsoft?

Watch out for Shadows

While messing around with types for a personal project, I ran into some unusual problems involving the Shadows operator. For background, let me explain that I had two classes, one of which extended the other, each of which containing an identically named class that would hold additional data for it. I wanted one class to override the other in the derived class, thereby swapping out the data type of that extra storage field. Since classes can't be declared Overloads, I settled for Shadows, which seems to have the same effect. I applied it to both the class and the field.

But it doesn't! Members declared Shadows will only be called if the invocation is on an object that is strictly known to be of the derived type! I was actually declaring two fields with an identical name but different data types. When I used the polymorphic approach and simply read from the field, I was only accessing the field declared by the superclass, never the subclass.

Shadows is not the same as Overloads. Make sure you know you're creating two different members when using it. In my case, the issue was solved by changing the field to a property and giving the conflicting members (both class and field) different names. Then, I could just have the subclass report its own field of its own data type.

The CLI/CIL/CLS/CTS Specification

Most .NET programmers aren't aware that it's compatible with other languages that we don't usually consider part of that family, like Mono. This is because its written assemblies and execution environment are standard - set by Microsoft and quite a few other companies. The spec provides details on the Common Language Specification, the Common Intermediary Language, the Common Type System, in addition to a lot of helpful tips for programmers. The thing, at its heart, is comprised of a list of rules for compilers and translators that, if followed, guarantee security and compatibility. The ECMA also packages an XML file than contains information on all necessary base classes.

ECMA page
Direct PDF link
Mirrored BCL XML

F#

I saw the name of F# in my Visual Studio add-ons dialog today and it made me remember how interesting that language is. I can't write anything it it without extensively using reference material, but it appears very powerful and very well integrated with the CLI. A while ago, I thought it was Microsoft's answer to R, but that does not appear to be the case. It is more like a .NET-compliant version of Haskell with more object-oriented features.

Some of its most interesting abilities are its handlings of the functional pattern. Types are handled in a way less painful than in VB and even more flexibly than in C#. However, the nature of variables and assignments to them seems a little bit strange, at least from my VB perspective.

Who should use F#? I'm not sure, to be honest. It does look like a good language to know should you ever stumble upon an amazing possible use of it.

.NET - Writing Your Own .NET Compiler

Reflection in .NET is exceedingly easy. It's also incredibly versatile; in fact, it's possible to create new classes, methods, and other fancy stuff with the *Builder classes. Add those to an Assembly and you've got yourself an entire library in memory. There's one method that makes it all even more amazing: the Save method on AssemblyBuilder. It will take all the dynamically generated code and dump it into a legit .NET library on your hard drive. I think you can even make DLLs by specifying the appropriate options in AppDomain.DefineDynamicAssembly.

So, if you're good at using ILGenerator, you can take in a text file, do whatever parsing and understanding you need, create the appropriate classes and method bodies on the fly, and create a new CLI-compliant executable. This is, essentially, a compiler. Learn more by reading the documentation on the types in System.Reflection!

Creating Extension Methods in VB .NET

An "extension method" is a method of a class or interface that is actually defined in a different module, usually by a different author. They are usually used to "extend" the functionality of existing types without creating new ones.

In C#, creating extension methods is pretty easy. Simply put a "this" before the type on the first argument, which indicates that it is the instance on which the method is running. In VB, it's slightly different but still pretty simple. First, import System.Runtime.CompilerServices at the top of the file containing the module. Then, apply the <Extension> attribute to the Sub or Function. When writing the method body, write as if the first argument is the current object. It obviously has to be the same type as the type to which you'd like to apply the extension.

Here's one I like to add to the string type:

Imports System.Runtime.CompilerServices

Module Extensions
 <Extension> Public Function EqCaseless(This As String, _
  Other As String) As Boolean
  Return String.Equals(This, Other, _
   StringComparison.InvariantCultureIgnoreCase)
 End Sub
End Module

Then, I can compare strings caselessly like this:

If input.EqCaseless("splosions") Then End

.NET - Guide to Server Basics, Part II

Now that you have a listener to catch new connections, it's time to actually communicate with them. I recommend another thread for reading in data. We'll use a queue of messages, but we'll need a new class to keep track of messages.

Public Class QueuedMessage
 Public Received As DateTime
 Public Data As String
 Public Sender As ClientCon
 Public Sub New(R As DateTime, D As String, S as ClientCon)
  Received = R
  Data = D
  Sender = ClientCon
 End Sub
End Class

Now we can make that queue and add data-reading and message-processing loops.

Public ProcessQueue As New List(Of QueuedMessage)
Public Sub ReadData()
 Do
  For Each c In Cons
   Try
    If Not c.Connected Then Exit Try
    Dim s As String = c.Reader.ReadLine()
    Dim msg As New QueuedMessage(Now, s, c)
    If Trim(s) <> "" Then ProcessQueue.Add(msg)
   Catch ex As Exception
   End Try
  Next
 Loop
End Sub
Public Sub ProcessLines()
 Do
  If ProcessQueue.Count > 0 Then
   Dim d As QueuedMessage = ProcessQueue(0)
   'Your handling code here using d
   ProcessQueue.RemoveAt(0)
  End If
 Loop
End Sub

Insert calls to start threads for those loops to the server's initialization sub. For convenience, I recommend having a writing sub in the connection class. Also, soon we'll need a time field to keep track of keep-alives.

Public LastKA As DateTime
Public Sub Send(Data As String)
 Writer.WriteLine(Data)
End Sub

Set the timer and give the client a little head start in the client acceptance code:

c.LastKA = Now.AddSeconds(5)

With that ready, insert a line into the data-reading loop that resets that field when data is received.

c.LastKA = Now

Prepare to enforce the timeout.

Public Sub Discon(Con As ClientCon)
 Con.Client.Close()
 Cons.Remove(Con)
End Sub

Finally, add a check loop sub.

Public Sub CheckKA()
 Do
  For Each c In Cons
   If c.LastKA.AddSeconds(10) < Now Then Discon(c)
  Next
 Loop
End Sub

Just make sure to have the client send updates or useless KA messages once every few seconds. I'm not sure what I'll cover next time.

.NET - Guide to Server Basics, Part I

It's moderately difficult to create a server application for the first time. This guide should help you establish a listener and an array of connections.

First, you're going to need to import System.Net.Sockets and System.IO. System.Threading is useful if you want a multithreaded listener. I recommend creating a class to hold connection objects.

Class ClientCon
 Public Client As TcpClient
 Public Stream As NetworkStream
 Public Reader As StreamReader
 Public Writer As StreamWriter
End Class
Dim Cons As New List(Of ClientCon)

The collection of connections is helpful to check all the clients in one sweep. Declare a listener as a TcpListener. I usually create a server class to manage all the servery stuff.

Class Server
 Public ShutDown As Boolean
 Public Listen As TcpListener
 Public Sub StartListening(Port As Integer)
  Listen = New TcpListener(IPAddress.Any, Port)
  Listen.Start()
  ThreadPool.QueueUserWorkItem( _
   New WaitCallback(AddressOf Listener))
 End Sub
 Public Sub Listener()
  Do
   If Listen.Pending() Then
    Dim c As New ClientCon
    c.Client = Listen.AcceptTcpClient()
    c.Stream = c.Client.GetStream()
    c.Reader = New StreamReader(c.Stream)
    c.Writer = New StreamWriter(c.Stream)
    c.Writer.AutoFlush = True
    Cons.Add(c)
   End If
  Loop
 End Sub
End Class

Creating a new instance of that class and telling it to start listening will create a new thread that listens for client connections, accepts them, and puts them in the connection list.

Next post, we'll get some communication and keep-alives going.

VB .NET - Lambdas

I have recently discovered that lambdas are amazing things. They take a little bit of getting used to, but once mastered, correct use of them will save programmer time and increase program efficiency. A lambda, essentially, is a function or subroutine written in one line that can be passed as an argument, usually in a LINQ method. When a method expects an argument of type Func or Sub (like IEnumerable.ForEach), it's time for a lambda. Start a lambda by writing Function or Sub and an argument in parentheses without type - the object that's going to be put in as the input of the lambda method. Then, simply write the return value. The simplest lambda is the identity lambda: Function(n) n. It takes in object n and returns it. For a Sub, you obviously won't have a return value, so just write what action it performs. Sub(n) MsgBox(n.ToString) accepts object n and writes its string representation in a message box. If you are using more complicated methods, use an AddressOf pointer to indicate that you want to pass the objects through a declared method.

Advanced Programming with Twisted Oak Studios

Twisted Oak Studios is a game development consultation firm that also does all manner of other high-level and complicated programming things that I do not understand. Over the past few weeks, I've read every blog article they've ever published. I've learned a lot, especially about functional programming ideas. I highly recommend that you read all the articles on their blog.

Essay: Classifying Programming Languages

I wrote a classification essay for writing class entitled Classifying Programming Languages.  It's in MLA format; signal phrases are used because the teacher said so.  Download (DOCX), 15KB.

LINQ - Get First Unused ID

I just spent two hours pounding my head against the proverbial wall trying to get some Logofrag stuff to work correctly.  After messing around with all manner of methods, I found an amazing StackOverflow answer that put all the pieces in place.  This is how I finally was able to find the first unused permission ID:

Dim idsQuery = From right In data.ThreadRights Select right.RightID

Return (From n In idsQuery Where (Not idList.Select(Function(x) x).Contains(n + 1)) Order By n Select n + 1).First

I had never before used lambdas, so that's another thing I learned.  They're very easy, just say whether it's a function or sub, give the arguments it takes, and write the return value.

Visual Basic .NET - Binary Operations

In Visual Basic, I sometimes start missing Java's nice bit operators, especially when I really need to pack boolean data into a single number.  Fortunately, VB has this; it's just less intuitive.  The operator for logical functions is indistinguishable from that of bit functions.  So, I can just do this:

For x = 0 To 7
    ChecklistPunctuation.Items(x).Selected = ((t.Punctuation And 2 ^ x) <> 0)
Next

Less fortunately, VB does not have shift operators.  You can however make do with integer division, multiplication, and exponents of two.  To shift a collection of bits N to the left, do this:

shifted = (bitcol \ (2 ^ N))

Maybe it's true that the language wasn't intended to be used in advanced/professional environments, but it certainly has been tortured into providing pretty much any functionality you could want.