AbiathRPC - Version check

I might eventually want to add some features to AbiathRPC after its first public release. Therefore, it'll need some sort of versioning system. I added an operation contract to the server's service contract that gets a string describing the server's version. If a client connects to a server with a different version, it warns the user about possible bizarre behavior. Since the client reports its own version when asking for the server's, the server has the opportunity to lie about its version if it decides it can accommodate the client despite their differences. That ability may or may not prove useful as I add more features after a release.

AbiathRPC - Remote ADEPS fragment

AbiathRPC level sets can't have all the fancy configuration available to local level sets because there isn't a real ADEPS file to load. For example, there isn't a way to set the number of infoplane tiles, which is kind of a problem because it's different for different episodes. So I added a server configuration option that has it load an ADEPS fragment from a file. When a client connects, it asks for that fragment and if it's found, the connection routine has the core configuration object load that text. Servers can now share these extra properties - e.g. path tile IDs, Level Inspector and Resource Accountant configuration - with clients.

AbiathRPC - Keep unnecessary menus disabled

Though I had already added code to AbiathRPC's connection routine to disable the menu items that don't make sense for a remote level set, I found today that they get re-enabled when the user switches levels. That's because Abiathar's SetCurLevel looks through the list of lockable menus and sets each to an appropriate state depending on whether there is a level set and whether there are any levels in it. I added an event handler on the TextChanged event of the level name label (which is fired after the level name indicator is updated, which is after menu re-enabling part) to disable those menus again.

AbiathRPC - Debugging level creation

I noticed today that though creating levels in AbiathRPC worked, trying to edit the new ones in the instance that created them would cause a crash. The old level wrapper (one around the plain GalaxyLevel) was still held in the current tool. Even after I made ScanForLevelSetChanges refresh the current tool when necessary, the watchdog render planes were still holding onto the wrong wrapper. There isn't a convenient way to make sure they always have the right kind, so AbiathRPC now keeps a table of plain level instances and tagged ones, updating it when a new level is created and consulting it when looking for a given level wrapper's level ID. Editing new levels now works.

Opening a file from Task Scheduler

Some might think that to open a file (e.g. a plain text document) from Task Scheduler, you need some sort of script to use Windows's "open this document with whatever it needs" mode. Interestingly enough, you don't - and since you don't have to, you don't have to see the flash of a command prompt or whatever you use to launch the final thing. Instead, you can just put the full path to the target document as the "program" in a "start a program" action and sure enough, when the task is run, up will pop that document in its default application. (Caveat: Task Scheduler doesn't seem to like spaces in the path, even if you quote it.)

A guide to installing Abiathar on Ubuntu 14.04

Last time I tried Ubuntu, I never got anything interesting running on it. Today I tried again, and after a ton of fiddling I managed to get Abiathar running on it. Note that I barely know anything about Linux and I'm just compiling a bunch of instructions into one place.

First, you'll need to download the update client (EXE link), which fetches the newest version. Your Downloads folder is a good place for that. Fire up a Terminal and navigate to wherever you put it. Mark it executable with chmod +x AbiatharUpdater.exe.

Now you need Mono, which is like .NET for Linux. Open the Ubuntu Software Center application, then find and install the Mono Runtime (searching for "mono" should pull it up). You need some extra packages:

sudo apt-get install mono-vbnc
sudo apt-get install mono-devel

Unfortunately, those don't include the SSL certificates that Mono needs to fetch HTTPS files, and mozroots is no more, so we need to add another repository?

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb http://download.mono-project.com/repo/debian wheezy main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
sudo apt-get update
sudo apt-get install ca-certificates-mono

For reasons unknown, installing that last certificates package takes a really long time and pokes at a bunch of other packages. Once it's done, you can finally run the Abiathar updater.

mono AbiatharUpdater.exe

If everything goes well, it will download three files. It fails with "The method or operation is not supported" at the very end, but that's OK; it was trying to launch Abiathar using a Windows function that doesn't exist on Linux.

OK, now we need WINE, which is a Windows emulator. (Abiathar needs a UI, which means window handles and stuff that isn't .NET, so just Mono won't do it.) Before we do that, mark Abiathar executable with chmod +x Abiathar.exe.

sudo apt-get install wine
sudo apt-add-repository ppa:wine/wine-builds
sudo apt-get update
sudo apt-get install winehq-staging

That installs the normal WINE package, then it gets the staging (development) branch, which has features we need. Now we need to configure stuff to get the .NET framework running inside WINE.

rm -rf ~/.wine
export WINEARCH=win32
export WINEPREFIX="/home/USER/.wine"
winecfg

Replace "USER" with your home folder's name. The last command produces a GUI configuration utility. While it's loading, it might ask to install some things; install them. Once the dialog comes up, change the Windows version dropdown to Windows 7, then OK out. Now we need extra WINE-related utilities to get .NET.

mkdir Winetricks
cd Winetricks
wget -r -c -N https://raw.githubusercontent.com/Winetricks/winetricks/master/src/winetricks

That downloads a script from the Winetricks GitHub repository. cd down through all the new folders.

chmod +x winetricks

sh winetricks

When you get a GUI, choose to "Select the default wineprefix", then choose "Install a Windows DLL or component." Scroll through the list and find dotnet40. It'll warn you about lacking support, but that's OK. Soon, you'll get the normal .NET 4 installer. Go through it as usual, and once done, cancel out of Winetricks.

Finally, cd back to where you put Abiathar, and run it.

wine Abiathar.exe

Abiathar running on Ubuntu

AbiathRPC - Level creation message size

While testing AbiathRPC's new ability to replicate newly created levels, I found that the client "received no meaningful response" from the method call that sent the level's contents to the server. I recalled that this had also happened when downloading the initial levels from the server, I remembered that I need to adjust the MaxReceivedMessageSize on the server end as well to allow it to handle the big packages that include full levels. With that done, the crash went away.

AbiathRPC - Scan for level set changes

Today I wrote all the parts of AbiathRPC that are involved in level creation and deletion. When the level set is marked as changed, AbiathRPC compares the current level set against its internal bookkeeping to see if any levels were deleted or added. If anything is found, it notifies the server. When a notification comes in as a result of another client doing something, AbiathRPC pokes at the main form to insert or delete the level as appropriate, then updates its bookkeeping.

Noticing deletion was more difficult than expected because with explicit interface implementation and inheritance, callers that know a real class that implements the interface will always call the original method. Therefore, I had to add an event handler to the level deletion menu itself to run the scan for changes.

I also disabled the Reload Graphics menu item for remote level sets because it would just break things.

Finding the physical sector size of a drive with no volumes

There are plenty of ways to get the logical sector size of any attached drive on Windows (e.g. wmic, msinfo32), but it's not very obvious how to get the physical sector size without there being a volume on the drive. Recently, I stumbled upon a PowerShell cmdlet that produces that information: Get-Disk. The output has both LogicalSectorSize and PhysicalSectorSize properties. Unfortunately, this cmdlet appeared in Windows 8, so I don't know a way to get this data on Windows 7 or earlier.

Based on my Super User answer.

AbiathRPC - Close, don't save

It's important that AbiathRPC be involved in the process of closing a level set because it needs to update its own bookkeeping to note that it's no longer connected to a server. (And, of course, it should also actually close the connection.) So today I added a part in the "add menu items" event bus stop that attaches an event handler to the Close menu item's Click event. When that runs, AbiathRPC disconnects.

I also noticed that it would be bad if the Save button worked as normal, creating a local copy that wouldn't work because of the lack of real graphics sources. Saving manually is also pointless, since changes are replicated to the server and other clients immediately. Therefore, during the initial connection, AbiathRPC now disables the Save menu item. I made sure it gets re-enabled if the user closes the connected level set and opens a local one.

AbiathRPC - Live updating

Today I implemented the "tiles changed" notification receiver in the AbiathRPC client. It runs mostly in an Invoke call on the main form to prevent threading headaches. While processing the tile updates, it sets a variable that signals the watchdog notification receivers that render plane updates shouldn't be treated as real local changes. Then it just updates the level data appropriately, marks the tiles dirty on the render planes, and causes a screen repaint.

I tested this with two clients connected to the same server, and changes from one client are indeed replicated in real time to the others. Now I need to detect and replicate full-plane and level metadata updates.

Markeen - Try removing the originality constant

Markeen's generation order has changed quite a bit since the original weighted random function was edited to include an originality constant (which biases the results toward or away from commonly-used tiles). That constant for a long time was set at 1.2, slightly favoring more common tiles. Recently I wondered whether that was actually useful now that Markeen has a smart fill order. I tried removing that adjustment - by setting it to 1 - and the levels certainly did change. The change made it much more common for multiple different platform styles to occur in the same level and therefore collide, but it did make other things fit together better (e.g. poles). I wonder if I could get the advantages of this without the platform collisions.

(I would upload pictures, but currently my Internet is super unreliable; it's losing at least half the packets sent.)

You can get Myst Masterpiece Edition from the Internet Archive

The Internet Archive has Myst Masterpiece Edition available for download. To actually use it, you'll want the BIN and CUE files from this directory. To convert that BIN into an ISO for free, you can use the demo edition of WinISO. Finally, if you have Windows 8 or higher, you can mount that ISO as a virtual CD drive; otherwise, you can extract it with 7zip or WinRAR.

To play it on a modern computer, you'll want ScummVM. Copy the necessary Myst files from that ISO into a new folder, then run ScummVM. Select the folder where you put those files, and you're good to go.

AbiathRPC - Writing to the server

Today I wired up all the previously-stubbed parts of the AbiathRPC client that get notified of local changes. They now keep records of pending updates, and when MarkChangesMade is called, those changes get pushed to the server.

I decided to abolish the server-side checks for out-of-order tile updates, since they just made things more complicated, especially if I wanted to make them work on full-plane updates. "Last writer wins" makes a lot of sense anyway. Speaking of full-plane updates, full-level updates are temporarily removed because they're basically the same as either three full-plane updates or a level deletion/creation cycle. (The latter will be used in metadata changes.)

After fixing a bug that left the original stub level in the memory of the first tool, AbiathRPC can successfully write to a server from one client! Changes are indeed persisted on the server. The next thing to do is write the parts of the client that receive server updates so some real-time collaboration can happen.

AbiathRPC - Watchdog planes

A very important part of AbiathRPC is the ability to detect changes. Today I started working on making that possible. I could have patched the view state class at runtime like I did the state manager, but I'd prefer to avoid reflection if possible. Render planes are always notified of changes, so a bunch of invisible dummy planes ought to do the trick. I added a stop for the event appropriate bus that injects those few render planes (one for each real plane, plus one that's only hit when the entire level is marked dirty).

The new watchdog plane class notifies the controller extension when a tile change is made or when the whole plane is invalidated. I've added an event listener for the main view's painting; that will be when the batched updates are delivered to the server. Waiting until a visual refresh can be done guarantees that the last tool finished whatever it was in the middle of doing when the planes were notified of the changes.

You can customize the Recycle Bin more if you redirect folders

The Recycle Bin allows you to tweak its settings (maximum size, whether to permanently delete) for a few specific folders, but only if those folders are subject to redirection to a network share. Usually that redirection is set in an Active Directory environment, but you can whack the Registry to make it work on a standalone computer too. The relevant values are here:

HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders

Only the folders shown there are eligible for redirection. Actually, the set of folders the Recycle Bin tweaking will work on is the intersection of that list and the list of known folders. Once you've redirected the folder, you can move to the BitBucket Registry key under Explorer

and add a new subkey KnownFolder. Inside there, create another subkey with the curly-braced {GUID} of the target folder. The DWORD values NukeOnDelete (0 or 1) and MaxCapacity (in megabytes) control the behavior of the Recycle Bin for files deleted from that folder.

Based on my Super User answer.

Allowing users to create things in a folder but not modify other things

Sometimes people want to allow users to create new items inside a given folder but not mess with the other things people have put there already. This can be done with Windows access control lists.

The trick is to give all users write privileges on the directory itself, but not apply that entry to subfolders and files. (Read permission to everyone can be granted on contents if desired.) Then, a full-power Allow entry for CREATOR OWNER can be added that applies to subfolders and files only, not the top folder. When a new object is created in the folder, its creator gains all privileges on that object only.

Based on my Super User answer.

Markeen - Tharify test

For the sake of seeing how it would go, today I ran Markeen on some Keen 3 levels after putting them through Tharify, the Vorticons ←→ Galaxy level converter. They all turned out basically the same: a mostly empty level with solid ground at the bottom and a few blocks and point items scattered through the sky.

(Imgur album of all the above.) These were generated at a profile depth of 2. Though multiple passes were allowed, only one was taken because of the tiny amount of failure tiles.

I believe these turned out this way because of the dramatic simplicity of the Keen 3 tileset compared to the Keen Galaxy ones - there's usually no difference between the different faces of a platform here, so Markeen doesn't have much information to go on on how it should form structures.

Markeen - Keeping options open

I noticed that Markeen still has issues with structures colliding with each other. Therefore, I thought it would be helpful to make the tile chooser avoid creating impossibilities. So I wrote up a change to the branch where there are multiple options and knocked out choices that would increase the number of unfillable spots. Unfortunately, making this work at higher orders seems very challenging, so it currently only works at one level. And, of course, failure is sometimes unavoidable at this depth. This feature has successfully altered some choices for improved quality, but it affects way less than one percent of the tiles in each level.

I also saw that some tiles should have been manageable, but were blocked for poor reasons (like a point item being too near what could easily be a pole cap). So at the end of the last pass, I now have Markeen iterating over all the failed tiles and placing the singular one that fits one of the sides - something of a resurrection of the now removed rare structure generator.

If you look closely, you'll find some placed tiles that don't quite fit together. Those are the products of the fallback placer; they would otherwise be normal failures. These levels were generated with a target of 9 maximum failures, 3 passes, and a profile depth of 2.

Markeen - Regeneration passes fixed

Last time, I was very pleased with the quality improvements from the smart fill order. After looking harder at the images with the regeneration passes done, though, I realized that they didn't seem any better or worse than the ones with no extra passes. Looking at the code, sure enough, when I disabled the rare structure generator, I accidentally caused every spot to be flagged as failed.

Once I fixed that, the process ran much faster, and produced some pretty nice levels.

These were generated by a 2-deep profile with a maximum of 4 passes.

Markeen - Good at low profiling depths

I wondered today how much Markeen's new slowness is increased by each level of profiling depth. So, for curiosity, I tried it at a depth of 2 as opposed to 5. It went a lot faster, completing each level in a handful of seconds.

And, to my shock, they also looked good!

Those are very, very good compared even to yesterday's. And only at one pass! I set the number of passes to three and tried again.

Still not perfect, but pretty good. I have a couple new ideas for performance and quality, so I think things can get even better.

Markeen - Smart fill

I just wrote a smarter method of determining the fill order for Markeen. Previously, I already had a method to calculate the probabilities of each tile in a given space. I used that to determine a "diversity of options" for each not-yet-filled tile. Instead of using a predetermined order, the filler now deals with the spaces with less diversity first, since the more diverse ones can presumably accommodate more choices.

I did my best to include some caching, but handling of all those probabilities for every place is expensive. Markeen is at least a hundred times slower now, even after I temporarily disabled the rare structure handling (which is difficult to integrate with the cache updating). The levels seem alright, but this new strategy isn't a magic bullet. I do think there's still more to explore with it; I haven't tried to fine-tune the diversity computer at all.

Here are all ten levels I generated with this new version:

(Click to enlarge.)

AbiathRPC - Logging

I haven't had much time today to develop any of my fun projects, but I did slightly adjust AbiathRPC. It now logs the basic user actions: connection, disconnection, authentication attempts. Instead of using Console.WriteLine everywhere, there's now an actual logging method that includes the timestamp in the output. In the future, I'll probably have this also write to a log file.

(This "Anonymous" is the one with the publicly available VeriMaps certificate. AbiathRPC users need to provide a username to connect.)

AbiathRPC - Infrastructure for editing

Today I wrote the parts of AbiathRPC Server that allow clients to actually edit the levels. That consists of a few WCF method contracts: one to add a level, one to remove a level, and one to edit tiles in a level. That last one is a little interesting, because it's possible that a client could have edited a tile in between when a different one issued a write and when the server received it. Currently, I have the server not change the tile again when it sees that's happened, but I'm not sure whether that's a useful check.

Anyway, calling any of those methods results in the server taking note that changes were made. It saves the levels to disk at ten-minute intervals if there are any unsaved modifications.

Since clients will need to occasionally encode levels into LevelInformation instances (when creating a level with existing data, e.g. by resizing or loading an ASLEV), I refactored the level translation code out of the server and into the bindings library so it can be used by both server and client.

It looks like the server doesn't need to be as complex as I thought. Getting the client to hook all the right places in Abiathar will be the next thing to do.

When an FTC robot controller won't move the motors

Today I was helping an FTC team with its programming. The programming went perfectly fine, but getting the robot to act on it was more difficult than expected. The joysticks were connected to the driver station and through that to the robot controller, but no motors moved. I was super baffled, because I was told that a very similar program had worked before, and I saw the states of the joystick controls on the robot controller's screen.

After much Googling, I found something that suggested rebooting everything and connecting the robot controller's USB cable last. That worked! Both the original program and the new one were able to operate the motors.

StartCom is probably going away

StartCom is a root certification authority (CA) that has historically provided free SSL certificates. I've used them for a couple of my domains, and I've been very happy with the experience.

A while ago, though, it was discovered that StartCom's new owner WoSign had engaged in some sketchy behavior (intentionally backdating certificates). They also had severe bugs that allowed people to get valid certificates for domain's they don't own, which is a Very Bad Thing^TM. Google Chrome has started phasing out its trust of those two CA's. I believe Firefox is also doing something similar.

This is basically a death sentence for that company, as it was for DigiNotar.

Policy Plus - Take ownership when downloading ADMX files

Today I tested Policy Plus's Acquire ADMX Files feature on a new computer. It successfully downloaded and unpacked the MSI, but when it tried to move the ADMX files, it died. I had forgotten that the PolicyDefinitions folder is by default owned by TrustedInstaller, which is the only principal that can write to it. I hadn't noticed this before because on my main computer I temporarily moved PolicyDefinitions for testing so I wouldn't overwrite it.

The solution is, of course, to adjust the ACLs on the destination folder before trying to move things there. The trick in this case is that administrators don't have write-DAC privileges on it, so SeTakeOwnershipPrivilege has to be enabled first, allowing the program to set the owner to the Administrators group. I refactored the privilege-enabling parts out of PolicyLoader into a new Privileges class for ease of enabling any privilege. (In this case, I also enabled SeRestorePrivilege so the DACL could be updated at the same time as the owner.)

I also found a bug that caused the move to fail if an overwrite was needed. That was easily fixed by first deleting files that need to be replaced. Everything works now.

The changes are live on GitHub.

Opening the Keen 4 demo version in Abiathar

A while back, someone asked me how to open the Keen 4 demo version in Abiathar. That's tricky because it doesn't come with an external EGAHEAD, EGADICT, or MAPHEAD. I had an interesting time getting those because the executable is compressed with PKLite instead of LZEXE, so UNLZEXE didn't work. Eventually I did get it decompressed with UNP.

After a few minutes of fiddling around in a hex editor, I extracted the three necessary files. You can download them here. Interestingly, I saw that the EGA header has 4-byte entries unlike the 3-byte entries in the full game. Even after I got that file, though, I didn't know where this version starts its tiles. To find that out, I used KeenGraph, which has the really awesome ability of auto-detecting chunk layout.

All in all, this is the Abiathar New Project Wizard configuration for the Keen 4 demo version:

• Template: Keen 4 with advanced settings
• Level Format: Carmack
• EGA Layout: 4, 478, 1296, 1774, 2916
• Tile Partitioning: defaults
• Containing Folder: pick your folder
• Level Source: start from existing files
• Level Files: original GAMEMAPS and my MAPHEAD
• Graphics Source: EGA resources
• Graphics Files: the original EGAGRAPH and my EGAHEAD and EGADICT
• Tileinfo Source: never use tileinfo

I confirm that those settings work. To me, the demo levels look extremely similar to the final ones, except that most of the ones present in the full game are missing here.