Close WCF Hosts in Service OnStop

Today I noticed that my .NET service never stopped correctly - the SCM always threw a warning before placing it in the "stop pending" state. It turns out that even after exiting OnStop, I still had an extra thread hanging around from a WCF ServiceHost I had opened and then let go out of scope before closing.

The listening thread from the ServiceHost kept the service process alive, which prevented a smooth stop of the service. The fix was to keep the ServiceHost in scope (as an instance member) and then close it in OnStop.

Don't Set the Thread Desktop from Thread Pool Threads

A month or so ago, I had written a .NET method that used Task.Factory.StartNew to show a form on an alternate desktop. I used P/Invoke to call SetThreadDesktop on that other thread before showing the form with Application.Run. I then waited for the thread to end by calling Wait on the returned Task. Once that finished, I switched the user's desktop back to the normal one and then closed the temporary desktop.

The problem was that the CloseDesktop call always failed with a Win32 error indicating that the resource was in use. After pondering that for a moment, I realized that my method of creating a new "thread" just borrowed a .NET thread pool thread and run my code on it for a while. As Raymond Chen says, using the thread pool is like renting a house - don't go changing the carpet. I had been changing the desktop of a random thread pool thread, which of course stuck around and continued being on it, causing the error.

The fix was to use New Thread to force the creation of a thread completely under my control.

Remote Desktop Surprise: WTSGetActiveConsoleSessionId Possibly Unreliable

Native Windows developers should be aware that it's possible for zero (only seen from service), one, or multiple users to be logged onto a computer while their application is running. The WTSGetActiveConsoleSessionId API function can be used to get the session ID of the user physically logged in at the console... or can it?

Even if no user has logged in at the console since the machine rebooted, Remote Desktop creates a pseudo-session that belongs to nobody. (It does have a few of the critical processes, like winlogon, running in it.) This wouldn't be much of a problem, except that it is the ID of that session that is returned by the function in question. So, if you create a process on the session returned by this function, you'll create a window that nobody can ever see if the user is logged on with Remote Desktop.

This makes sense, considering that the function's name includes console, but I really wish it would return the "no user logged on" value of -1 rather than the ID of a useless session.

Abiathar Online

The whole Web 2.0 (or is it 3.0 now?) thing is kind of a big deal, and it a few weeks ago it hit me that I know nothing about programming for that world. There are all sorts of amazing things being created in JavaScript and HTML5, and I have zero programming experience in those. With everything going to the Web, I risk becoming a dinosaur by restricting myself to .NET desktop apps.

So, I gave myself a challenge: create a fully client-side Keen Galaxy level editor using HTML5 and JavaScript. This is both a worthy challenge for me (considering my complete lack of  web programming knowledge, but markedly nontrivial experience in general programming) and will produce a useful result. There are not yet any web-based level editors in the Keen community.

I would be OK with a partially server-side solution (probably using ASP.NET), but there is the major issue of finding a good but reasonably priced host. If it's all JavaScript, I can throw it up anywhere or even let people download a ZIP and put it wherever they want. I plan on using Dropbox's Chooser and Saver controls/APIs to load and save levels; HTML5 currently has no nice way of modifying local files.

Since JavaScript is all client-side and source-code-a-button-push-away anyway, I figured I might as well make this a real open-source project and start using GitHub in earnest. I'll still be using Visual Studio for text editing, and probably Google Chrome for testing/debugging.

I haven't made any progress except getting the critical files together (the VS solution, an HTML page, and a stylesheet), so there's not much to show yet. It'll come together slowly, probably with a good deal of Googling. It's an adventure!

FMod - v2.5.3

In a screenshot posted by a community member, I noticed a serious problem in Abiathar's routine for rendering ModKeen-style bitmap graphics. It was treating every pixel as opaque, even when the mask section of the bitmap said it should have been transparent. That led to ugly opaque black overlays, completely covering the background tile whereever the foreground had a non-null tile.

It turns out that the equality operator was not overloaded by System.Drawing.Color in the way I expected. I had to convert the pixel color to an integer with ToArgb before performing the comparison. (That's kind of strange, because I'm sure I tested this. Maybe it varies between framework versions?) The problem is fixed, and ModKeen graphics now display correctly.

While testing that fix, I also noticed a problem with the simple (TOM-style) version of the New Project Wizard. It recorded the absolute path to the GameMaps file rather than the path/filename relative to the auto-detected project folder. That forced the user to fix the path manually if they wanted to change their resources in Project Settings (which is not hard to do - just delete the absolute part of the path - but still unexpected/annoying). It also would have caused issues if a project folder was sync'd across multiple computers. The issue has been fixed, but I can't safely do anything about existing dependency files that were created in the simple NPW with this bug.

Before releasing, I changed the color of the "Active" plane state label to blue, making it match the blue/cyan theme of the rest of the program.

.NET Service Surprise: Release Mode Required

Today while working with .NET Windows services, I found that my service had inexplicably stopped reporting its status to the SCM. Despite returning successfully from the OnStart method, the service was always killed by the SCM, which cited a timeout of 30 seconds without response.

After much Googling, I discovered the answer: .NET services compiled in debug mode do not report to the SCM, even though OnStart is called as usual. The solution was to always compile in Release mode.

Windows 10 Auto-Update Fail

The earliest public build of Windows 10 (9851?) expired on April 15. This shouldn't be a problem, though, because it automatically downloads and installs the newest version of the operating system. Right?

Right?

Nope. If the computer is configured to sleep after only a short period of inactivity or if it is never on at the scheduled upgrade time (I think the default is 2:00 AM), it will never get upgraded. A while after its expiration date, it will start blue-screening with END_OF_NT_EVALUATION_PERIOD, indicating that the trial version of Windows can no longer be used.

On some computers, I've found that the PC Settings place to check for new builds says there are no new builds when there definitely are. This leaves me with no means of upgrading short of manually downloading, expanding, and using a DVD-size ISO.

ADMX Migrator Surprise: Enabling a Setting Must Affect a Key

(Maybe this isn't so much of an ADMX Migrator surprise, since it could happen even if I manually wrote the ADMX file. And I suppose it's not too much of a surprise either, since careful reading of documentation might have prevented my misunderstanding. Eh. I'll write this post anyway.)

I was working on some custom Group Policy settings that have at least one text field each. I noticed that the settings wouldn't stay Enabled after I set them (they would revert to Not Configured when I hit OK), but they would stay Disabled.

Now, ADMX Migrator makes you specify the following information for each setting: registry key, registry value name, "enabled" value, and "disabled" value. Those last two can be one of "None", "Deleted", "String", or "Numeric". Since I was doing all the work in the registry value associated with the text box, I wasn't sure what to supply for the setting's key. I had used the same registry value name for the text box and the setting. I had also used "None" for the Enabled value.

That last choice occasionally produced a warning message in the GPME saying something about an XML error. My first mistake, though, was much more subtle. You see, every Group Policy setting has to either set or destroy a registry value. (Not necessarily order-respective for Enabled and Disabled there.) The fix was to change the setting's value to something like "ConfiguredSpn" so that the program knows to then look for "Spn", the value set by the text box.

Creating Group Policy Settings with ADMX Migrator

I recently found myself wanting to add Group Policy options for one of my applications, so I looked up the ADM and ADMX formats, the files used by the Group Policy Management Editor to load administrative templates. Those formats are both really tricky, and I don't really want to do manual XML/text editing.

After some research, I discovered the confusingly named ADMX Migrator, which was created by FullArmor and is available through Microsoft. It is, for some reason, an MMC snap-in, but the "ADMX Migrator" entry it adds to the Start menu works just fine for launching a new console containing it.

ADMX Migrator can, as the name implies, convert ye olde ADM files to the new ADMX style. More interestingly, it can create new ADMX+ADML files. It has a nice enough GUI for doing that, and a help file (which integrates with MMC help). Those files can then be dropped into the PolicyDefinitions folder to add the settings to the Administrative Templates category.

Definitely a helpful tool. I have and will continue to make use of it.

Download ADMX Migrator (same as first link above).

SPNs: What they are and how to work them

I've been wrangling WCF's security modes for a few days now, and I think I finally understand what this whole SPN thing is about. It took some research and experimentation to figure this out, so it might be helpful to others.

SPN stands for Service Principal Name. It's kind of like a user account in that it's stored in Active Directory and that it is used to verify an identity, but the identity that SPNs verify is that of a service (an application whose traffic is secured with Kerberos). Each SPN is bound to a specific user account, so it can only be used by applications running as that user. The interesting thing about that fact is that machine accounts are users too, so any application running as the System or NetworkService account can use the SPNs set aside for the machine.

SPN names are strings of the form "service/place". "service" is unique to your application, but you can name it anything you want, so long as your client is expecting it. "place" is a disambiguator, in case you want multiple instances of the service running on the network. (It seems only one machine can use an SPN at a time.) "place" can also be blank as long as there's a slash in the SPN name.

To create and manage SPNs for the domain, you'll need to use the "setspn" tool that comes with any domain-joinable version of Windows. Type setspn username to get a list of all the SPNs usable by that user. (You'll need to be running as an elevated domain admin to use most of setspn's features.) If you run that for a machine account, you'll see several SPNs that were automatically generated for the computer, including the often-referenced host/machine.example.com one.

To add a new SPN for a user, do something like this:

setspn -S AwesomeServer/TheOneAndOnly HostingUser

HostingUser will then be able to open a Kerberos-secured WCF endpoint for AwesomeServer. The user can certainly be a machine account, but if your application will always be running as an account that can access the computer's SPNs, you might as well just have it use the auto-generated host/machine one.

Opening the WCF endpoint is easy. Just create a new SpnEndpointIdentity, pass it the SPN name, and pass that identity object to CreateChannel, make sure to enable transport security, and you're secure!

Automatically Expanding Ranges in Excel 2013

Today I found myself wanting to sum up a bunch of values in a column whose length expands as I add more values. One option is to write a range that is really tall (like D4:D200) but there's the possibility of values eventually being entered after the range, and I'm not really a fan of arbitrary limits, even if they're easy to adjust. I think Excel has a type of range definition that selects the entire column, but my worksheet has unrelated data above where I need to start summing.

I did some research, and it turns out that there are things called dynamic named ranges that can change their selection depending on the value of one or more cells. I'm sure you can do all sorts of amazing stuff with them, but all I needed was the ability to sum as long as there are values.

Named ranges can only be managed with Name Manager, but the new Ribbon UI makes it really hard to access. In fact, the only way to get to it is to open Excel Options and add Name Manager (under All Commands) to the Quick Access Toolbar or a new custom group in the Ribbon. Once you have the button, click it to open the Name Manager, then click the New button in the resulting window.

Write an appropriate name (without spaces) in the Name field, then paste the following into the "Refers to" box:

=OFFSET(Sheet1!$D$4,0,0,COUNT(Sheet1!$D:$D),1)

I underlined the parts you'll need to change. Change "Sheet1" to the name of the sheet containing the range. Change "$D$4" to the starting cell of the data column you want to sum. Change the other "$D" instances to the name of the column in which your data resides.

Be aware that editing that string in the Name Manager may be really inconvenient because Excel tries to "helpfully" shove cell names into the text field whenever you use the arrow keys. You may want to paste into and edit in a Notepad window.

Once you have your custom range, you can use it's name in place of the usual rectangular definitions (your standard F5:J22 stuff). I named my range "Times" and replaced the total cell's formula with =SUM(Times).

WCF is Awesome

I had heard of the Windows Communication Foundation (WCF) a while ago while researching inter-process communication in .NET. I recently tried it out, and I discovered that it's really awesome.

WCF is based on services which expose endpoints. A service is a server application, and can be in the form of a normal .NET process, a dedicated Windows service, or an IIS web site. Each service has one or more endpoints, which are like network listeners. WCF supports several protocols, including HTTP, TCP, and (should you really feel the need) MSMQ.

For a client to consume (use) a service, both parties must be able to load a .NET interface type that defines what the service does. An interface designed for WCF must be tagged with ServiceContract, and each function must be tagged with OperationContract. Those attributes can specify other options, such as the required encryption level or whether a connection must remain open. The server application actually implements the service interface, and tells the ServiceHost the implementation type. Once the Open method is called on the ServiceHost, the server is listening for clients.

The client creates a channel with an EndpointAddress configured with the server's address and connection properties (e.g. encryption) and passes it the .NET type of the server interface. Magic happens, and poof you get an object that implements the server interface and that talks to the server over the network when you call its methods, acting like a local object all the while. (Like ye olde .NET Remoting, except it won't fall over as soon as you turn your back.)

Since you probably want to give information to the client even if the client didn't initiate anything, WCF also includes support for duplex bindings. When a ServiceContract is marked as requiring duplex communication, there is also a client interface associated with the service. It is then the client application's job to implement it. Once the duplex channel is opened, poof the server can call methods on what looks like a local object, but actually sends information over the network to the client.

WCF doesn't actually have to be over a network - it works for inter-process communication on the local machine too. I haven't tried this yet, but it looks like named pipe binding is the way to go.

When the channel is authenticated with Windows credentials, impersonation (and if you really persevere, delegation) is a simple matter of calling a method. The client spews its security token over the network to the server, which then checks it with a domain controller using Kerberos and knows for sure that the client's identity is legit.

There are all kinds of properties on the communication context object that let you manage the state of a specific connection and read properties about the client, such as its IP address.

Unfortunately, there are lots of Internet tutorials on WCF that are really bad and are super inconvenient. I'm also not really a fan of the XML (WSDL) configuration style; I prefer that my code do the set-up of network connections. There doesn't seem to be as much information on programmatic control of WCF bindings, channels, and endpoints as there is for WSDL, but it's there if you poke around and experiment. This might be more of a personal preference than anything (maybe nostalgia?), and I'm sure WSDL set-up of WCF works fine for lots of people.

So, WCF is really awesome. I'm never writing my own networking code again.

FMod - More Crash Bugs

Another crash bug was reported in Abiathar's New Project Wizard. (That one component has been the source of many issues over its half a year of existence.) When the "get prompted for basic resources (simple)" option was selected and the first file browse operation was canceled, the wizard would move to the directory selection page. When either the Next or Back button was pressed, Abiathar would crash after attempting to use a null reference.

It turns out I had forgotten the "else" part of an if statement that checked for the successful completion of the file browse. After adding an "Exit Sub" to the Else block, the bug was fixed. If the browse is canceled, the wizard stays on the first page to let the user choose a different option.

While messing around in testing, I discovered that dragging an ADEPS file onto the Abiathar executable to open it resulted in a crash. This bug was introduced in v2.5.1 when I added the zoom slider - when the command line arguments were dealt with, UI elements had not been fully initialized. After moving argument processing to the end of the startup procedure, the bug was fixed.

These fixes were released as v2.5.2.

FMod - v2.5.1

I ran final tests on Abiathar v2.5.1, especially on the New Project Wizard. Everything worked as intended; all the nice subtle intuitive default choices worked as I wanted. No rendering bugs are apparent, and everything seems good with user interaction.

So, I compiled the whole solution in Release configuration, bundled it all up, wrote the update notice, and published all these changes as Abiathar v2.5.1. With all known bugs fixed, I think I'll take a break from Abiathar to focus on other projects.

FMod - More NPW Fixes

I finally got around to testing the edit mode of the New Project Wizard (which can be found at Edit | Project Settings). It turns out that the "fixes" I had made to correct the crash that occurred when the template was changed actually caused the wizard to completely forget all the project's settings when opened in edit mode.

That's a serious problem, but the fix wasn't terribly complicated. I had left behind a fragment from an attempt at a fix for the aforementioned bug that wiped the visited-tabs list after the Level Format tab was visited. I removed that instruction and everything was good.

I also made the mouse-over coordinate label disappear correctly when the dependency file is closed.

FMod - Availability Tweaks

I continued testing the changes made for v2.5.1 and discovered some issues with the modifications to the New Project Wizard.

First, the checkbox that causes the graphics loader to use the game's default EgaDict was showing up usable even if the template selected did not include a default EgaDict. (The BioMenace ones, for instance.) The fix was the addition of one line that sets the Enabled property on the checkbox to whether DefaultSet is non-null.

It also turned out that the MapHead tileinfo detection didn't work unless the Abiathar executable was actually in the same directory as the MapHead. There was no crash (I had a fallback to the default choice of loading tileinfo from a separate file), but it was still a bug. The fix was a Path.Combine between the MapHead name and the depsfile directory.

I still need to make sure all this stuff works when editing an existing project, but other than that, v2.5.1 is just about ready to go out the door.

FMod - Default EgaDict Loading

I noticed that several Galaxy mods ship modified EgaGraph and EgaHead resources, but the EgaDict is neither replaced nor removed via patching. Such mods are very difficult to open directly in Abiathar because it requires you to specify all three EGA files. (The check box that controls whether it requires an EgaDict completely changes the decompression algorithm.) The workaround was to copy the game's default EgaDict into the mod directory, then specify that.

That's really inconvenient, so I added an option to the New Project Wizard that causes the EGA graphics loader to use the game's default EgaDict, even if the other EGA files are being loaded from disk. The "this graphics set requires an EGA dictionary" check box was reworded to "this game uses compressed graphics" so as to minimize misunderstanding of what changing its state will mean.

The revised Graphics Files page

FMod - Zoom Slider

The community member who reported the NPW crash and the Essential Manipulator UI glitch requested in a follow-up post that I add a zoom slider to Abiathar's status bar, like you see in the Office applications. This seemed like a good idea to me, and not too hard to do, so I started doing it.

It took a bit of research to figure out how to jam a TrackBar (slider control) into the status strip, but after I did it just worked like any normal slider. After entangling the zoom keyboard shortcuts with the slider update routine, it works just like I wanted.

There is, however, a problem. When the user uses the mouse to adjust the slider, focus is set to the slider control. Key-down events are then not delivered to the form when the user presses keyboard buttons. (They go to the slider instead.) I tried having the slider forward all keypresses to the form's handler, but it still takes the arrow keys. Fortunately, some Googling turned up the solution, which I will implement tomorrow.

Abiathar Bug Tracker

I am beginning to find that forum posts, especially when spread across multiple threads or even message boards, aren't the greatest means of tracking bugs and feature requests. For a while, I had been looking around for a decent bug tracker that was [1] free and [2] didn't require me to host anything. That's kind of a tall order, because it's rare that you get something good for nothing.

However, I think GitHub has what I'm looking for. Every repository has an "Issues" tab, which functions as a defect tracker. It even lets me configure labels that can be attached to issues, so I can look at just feature requests or just bugs of a certain type, if I want.

Abiathar isn't open-source, and probably won't ever be unless/until I discontinue maintaining it. So, I just made a blankish/placeholder repository named "Abiathar Support" for housing the Issues manager. See it on GitHub.

I added a link to this in Abiathar's Help menu, so there is now an official way to report bugs.

FMod - New Project Wizard Improvements

I fiddled with the New Project Wizard some more today to improve its intuitiveness, especially when it comes to default settings.

The Graphics Source screen now changes its default selection depending on whether or not existing levels are being loaded. This makes sense because it's very likely that somebody creating a project from existing levels is going to want to use existing graphics and someone starting fresh probably wants to start out with the default game graphics. If existing level files have been selected, EGA or bitmap graphics are selected according to whether EGA configuration is present in the template.

The tileinfo source works similarly. If there are existing level files selected, Abiathar automatically checks the size of the map header to determine whether there is tileinfo present in it, and if so selects the in-maphead option.

Tomorrow I need to tackle some UI bugs and add an NPW option to use the default EGA dictionary while loading existing EGA resources.

FMod - NPW Reset Crash Fix

An Abiathar crash bug was reported today: the New Project Wizard caused a crash when moving to the Map Files tab after the dependency file had been reset by moving back to the beginning (rather than reopening the wizard). It turned out that the visited tabs list was not properly cleared when starting from a template after resetting without reopen. After adding a call to VisitedTabs.Clear, the bug has been fixed.

Fixing Driver Problems with Sysprep

If you're at the end of your rope with mysterious driver problems (especially Code 10, "the device could not start"), I believe I've found a solution. It's rather heavyhanded, but it's worked for me.

The System Preparation Tool, also known as Sysprep, has a "generalize" option, which blows away all machine-specific information, which includes drivers. When a machine comes back up from a generalize, it works just like you reinstalled the OS, with device detection and OOBE. It also destroys things like the computer name, certificate, and domain-joinedness.

Sysprep can be found at C:\Windows\System32\Sysprep\Sysprep.exe. Check "generalize", make sure the dropdown says "Enter System Out-Of-Box-Experience (OOBE)", and begin the operation. When the computer comes back up, your device issues may very well be solved.

(An OS reinstall does something very similar, but with this approach you get to keep your files, programs, and local user accounts.)

FMod - v2.5

While doing some final testing, I ran into a crash bug involving the Essential Manipulator. If it hadn't been used yet, it would dereference a null pointer and crash when the Palette Copier notified it of a clipboard change. This was fixed by a quick null check.

I then released all the changes done over the last two weeks as Abiathar v2.5, the "intuitive update."

FMod - Toward Easier Undo

It comes to mind that the way I've been having all the tools collect the undo history is really inconvenient for me and that there's a better way. Currently, every tool has to get the current tile before replacing it and push a big Tuple(Of UShort, UShort, Byte, UShort, UShort) to the ModdedTiles list in the TileUndo instance it's using to represent the action.

I've grown rather tired of coding that, so I think I should do something different. Since tools can only modify the level through the instance of IAbiatharLevelWrapper they're given, I could add methods to that interface that cause it to track modified tiles and automatically spit out the complete IUndoableAction. Tile changes already go through a property on that interface, so it would be a simple matter of adding a few lines to its implementation.

I haven't done this yet, but I probably will if I ever have to make serious additions to the toolbox.

FMod - OOBE

I've read several times that settings in a textual config file are extremely unlikely to actually be changed; default values might as well be constants. Since people seem to have very diverse preferences in level editor UI, it's really important that they know about and can use Abiathar's configuration options.

So, I added something of an OOBE screen that appears the first time Abiathar is run. It gives a brief rundown on how to get started and provides access to the 16 most common settings:

The Abiathar OOBE, showing default settings

A similar dialog is shown when the user chooses the new File | Preferences menu item. I'm afraid it might be too scary for novice users, so I'll see if I can think of anything else.

I also made some other fixes:

• Simultaneous Tileset is now preserved across the opening of a different dependency file
• Tools that change their drag translation mode to click-on-drag-endpoints in response to the first click-on-every step message no longer receive a spurious duplicate click message
• The number of song chunks in newly-linked audio resources is now detected correctly
• The Essential Manipulator now:
• Updates its preview overlay when the clipboard is changed via the Palette Copier
• Skips inactive planes when placing tiles (even if those planes are in the clipboard)
• Sets the clipboard to a 1x1 of the selected tiles if the clipboard is empty
• The Palette Copier no longer leaves behind a selection rectangle if it is the default tool

Windows 10 Server: Missing in Action

Today upon logging into my new Windows Technical Preview Server (a.k.a. Windows 10 Server), I received a notification that the evaluation edition will expire on April 14, after which the machine will reboot every few hours. That's fine, I expected preview builds - especially ones from October 2014 - to expire in the nearish future.

The problem is that there doesn't seem to be a newer build of the Windows 10 Server available. The only place to download the server version is at the TechNet Evaluation Center, and they're still serving the one from October 2014. I can't find any word from Microsoft about the future of the Windows 10 Server or where I can get the latest build of it.

So, unless they suddenly release a new build of it, I'll just have to deal with random reboots after April 14.

FMod - Music Autodetect

Somebody noticed that Abiathar does not preserve extra songs when attaching existing audio resources to a dependency file. That is, if you attach some audio files where you've inserted more songs than the vanilla game can deal with, the extra ones are essentially removed. (FleexCore2 doesn't bother to load them unless its chunk ID falls between ImfStart and ImfEnd.) This can be worked around by changing the ImfStart entry under AudioSettings in the dependency file before attaching audio, but that's trickier than most people want to deal with.

So, since attaching pre-extended audio files is probably not a super rare thing, I decided to make it a little nicer. Abiathar now scans backward from the vanilla ImfStart to see if there are any populated chunks in the no man's land between AdlibEnd and ImfStart. If there are, it auto-adjusts the ImfStart value under AudioFiles to make the patching happen properly.