FMod - Foreground highlight

Yesterday, I made little headway in implementing a "front" highlight mode for the Tile Properties view. I realized today that the best approach - in terms of ease of implementation now and of maintainability later - is to have a per-tile overlay system in addition to the existing per-property system.

Every foreground tile now has a per-tile overlay computed for it when called for. Currently the only kind of markup that can go on those images is the foreground highlight. That highlight is a translucent silhouette that only covers the area covered by the foreground tile itself.

The highlight in action

FMod - Attempts at foreground highlight

Somebody mentioned that it would be helpful to have a mode that highlights "front" foreground tiles, i.e. those that always appear in front of Keen and sprites. It's extremely easy to put a constant overlay on such tiles, but that's not very helpful in visualizing what parts will definitely appear in front. The obvious approach would be to get the image of the tile being marked up and make a translucent constant-color version of it, but in the current architecture, the function that creates the overlay image doesn't know which tile it's working with, only its Abiathar Tile Property ID. Multiple tiles can have the same properties, so the cache and the code paths will need to be reworked for this to happen. I do think it's worthwhile, so I'll need to figure something out.

Constructing generic-typed objects in PowerShell

Suppose you wanted to use an object of a generic .NET type in PowerShell, e.g. Dictionary. You might look for an extra switch on New-Object to specify the generic type's specialization, but there isn't one. The correct way to do it in modern PowerShell versions is this:

New-Object 'System.Collections.Generic.Dictionary[string,int]'

You can use either the full .NET names or the short PowerShell type names for the type parameters. The square bracketed part here is equivalent to the angle-bracketed part in C#. Also note that the quotes around the type name are required; if they're missing, PowerShell will interpret the parameter as the value of the -ComObject switch.

Volunteering at an FTC League Championship

I spent today volunteering at the local FIRST Tech Challenge league's championship meet. My first role was Robot Inspector. Previously, I had been a Field Inspector, making sure the robots could operate on the field and connect wirelessly. My new job entailed making sure the robots were physically acceptable, e.g. sized appropriately and made out of allowed components. That was very easy given the checklist. We completed inspections ahead of schedule, except for one team that needed to adjust their robot a bit to pass.

Then, I worked as a Scorekeeper, which was completely new to me. I expected that I would be sitting by the field and keeping track of the scoring events, but Scorekeepers actually manage the computer systems in the play areas. My job was to operate the timer on one of the two fields. When the emcee counted down to the start of the match, I pushed the button to make the big timer display start ticking. That was also very easy to do once I learned to navigate the special software they have.

We finished the matches almost on schedule without any major mishaps.

Copying only directories in PowerShell

Suppose you wanted to copy only a directory structure, without the files inside. In PowerShell, you might try to use Get-ChildItem with the -Directory switch to get only directories, then pipe them into some script block that copies them. The trouble with that is that you need to compute the resulting path by replacing the source path component with the destination one, which is doable, but a little messy.

Unfortunately, Copy-Item has no -Directory switch. It does, however, have -Filter, which technically takes a string, but you can almost think of it as receiving a script block. In this case, -Filter {PSIsContainer -eq $true} solves the task, as does simply -Filter PSIsContainer. That variable is only true for containers: directories, in the case of file system objects.

Registry redirection and the WOW6432Node key

Like IO operations from 32-bit applications on System32 are redirected to SysWOW64, 32-bit programs accessing certain parts of the Registry are redirected to a different location. For example, operations on HKLM\Software will go to the equivalent path under HKLM\Software\WOW6432Node. There are some exceptions, where structures are shared between the two Registry views. MSDN has a list of affected locations.

If a Registry change in the normal parts only seems to affect some applications, you probably need to make it in the 32-bit section as well.

PowerShell's $ConfirmPreference

Some PowerShell cmdlets ask for confirmation before doing their work. (You can always ask for them to confirm by passing the -Confirm switch.) Clearly, different operations have different levels of possible damage, so it makes sense that PowerShell allows each cmdlet to declare a "confirm impact": None, Low, Medium, or High.

You can choose how impactful an operation should be before PowerShell prompts them for confirmation. The $ConfirmPreference variable can be set to one of those four values; confirmation will only be required for cmdlets with an impact higher than the preference.

To override the confirm preference to say "do not confirm this one action", pass -Confirm:$false.

Based on my Super User answer.

Dot-sourcing can be slower than reading in and then evaluating

Someone had a ton of PowerShell functions to declare and for version control reasons put them each in a separate file. They found that dot-sourcing the files to run the script and include the functions in the current scope took notably longer than opening each script file and invoking the contents like so:

dir *.ps1 | % { iex (gc $_.FullName -Raw) }

Understandably, they were curious about this. I did some investigation (profiling) and found that dot-sourcing eventually involves a call to WinVerifyTrust, apparently to verify some resource's integrity. This is probably to look for a digital signature on the script files, even when the execution policy allows running unsigned scripts. This doesn't happen when getting the script text manually and then invoking it.

So, if you're running a ton of scripts, you might consider using Get-Content and Invoke-Expression.

Based on my Super User answer.

When a user doesn't show up on the logon screen

A user wondered why a Windows account vanished from the list of users in the logon screen and Control Panel. After doing some quick experiments, I found that at least these two conditions need to be satisfied for the user to appear:

• The user account must be enabled. If it's disabled, use net user username /active:yes
• The user is a member of at least one of the Users or Administrators groups. If it's not, add it to one of those with net localgroup Users username /add

FMod - Extensible patching

For a really long time, Abiathar has supported extensions injecting their own data structures into project files. Extensions that do keep some user data (not that there are any at the moment) might conceivably need to adjust the patch file based on data they manage.

Today, I added an event bus for patch generation. The event just allows extensions to add lines, which get added by the patch generator into the Abiathar-managed section.

"On connection to user session" trigger not firing? Run as SYSTEM

The "on connection to user session" trigger in Task Scheduler can be used to run a task whenever a user connects or reconnects to their logon session, either at the workstation or by Remote Desktop. Bizarrely, the task never seemed to run for one person. I got the same behavior when I set up the task as they did.

After I set the task to run as SYSTEM and adjusted the parameters to powershell.exe to provide a full path to the script they were trying to run, it worked. Both of those changes had to be made for me to see it work; I tried all four combinations. I'm puzzled as to why SYSTEM is special here (running as an admin or even a normal user should have worked with my test setup), but it's something to try if you're having Task Scheduler problems.

Adjusting process ACLs with PowerShell

Suppose you wanted to stop a process from being terminated. To do that, you could adjust the process's ACL to deny the "terminate" access right.

(Insert spiel here about how processes should be run under a different user than the one you're trying to protect them against. It will be extremely difficult if not impossible to completely stop a user from messing with a process that runs in their own security context and in a desktop they control.)

Downfalls aside, it is possible though not terribly convenient to control the access to a process with .NET and therefore with PowerShell. Another person went to the trouble of implementing a ProcessSecurity class in C#. Using Add-Type, we can take advantage of all those defined types in a PowerShell session. The full script is very long and a superset of that code, but you can see all of it and an example usage - denying the terminate right on given processes - in my Super User answer.

Installing the Remote Server Administration Tools on Home editions

Installing the Remote Server Administration Tools normally on a non-Pro-ish edition of Windows will result in an error saying that the update is not applicable to the computer. However, you can still get RSAT installed.

First, expand the MSU file into some preexisting empty directory:

expand RSAT.msu -F:* C:\msuextract

Inside it, you'll find at least one CAB file. These contain the actual update resources. Apply the ones mentioned in PkgInstallOrder.txt to the computer with DISM:

dism /Online /Add-Package /PackagePath:C:\msuextract\cabfile.cab

It will ask to reboot; do so. You may have noticed an EXE file also from the MSU, which is mentioned in PkgInstallOrder.txt. If so, run it as administrator if you'd like. When I did that, it exited immediately and I'm not sure if it did anything.

Anyway, once that's all done, the Remote Server Administration Tools should work as expected.

Mod hat

My moderator diamond hat from Stack Exchange just arrived in the mail:

Pardon the cell phone photo quality

Thanks, Stack Exchange!

Markeen - The candidate problem

Markeen has trouble with the combination of more-than-one-deep profiles and the scarcity of original levels. There are plenty of more-than-one-away tile placements for most tiles that would work perfectly fine but don't appear anywhere in the originals. The candidate scoring algorithm devalues (instead of deletes) not-found options once the depth is more than 1, but the starting list of candidates is taken from only the first present neighbor, usually the top. This isn't terrible, since a tile should have to be seen before next to all its immediate neighbors. It does perhaps get the probabilities of the final candidate list wrong, though. Biasing the selection toward the preferences of one side seems incorrect, so I'll have to see about changing that.

Markeen - Single-thread for seeded generation

When I implemented the multithreaded generation for Markeen, I inadvertently broke the feature that lets the user specify a seed to get a predictable level set. When there are multiple generations going on at once, the random number generator is used from multiple threads in unpredictable orders, completely destroying the whole "same output for same seed" thing.

To fix that, I reshuffled the outermost generation loop to fall back to the old behavior (one level at a time, sequentially) if a seed was specified. Now, most people will enjoy the fast multithreaded generation, but a predictable set can be created if necessary.

FMod - Don't flood the user with image export errors

After writing the part of Abiathar that allows all levels' pictures to be exported at once, I noticed that if for some reason there's a problem that hampers the export (e.g. a bad character in the config file or an access-denied condition), the user will get an error message for every single level. Clearly, getting the same pop-up error window a dozen or so times for a single problem would be irritating. So today I rearranged some code to ensure that only one error message will pop up if there are problems.

I also gave the Inspection Results window a reasonable tab order, since it was weird and arbitrary before.

FMod - Export all images

One user asked for an Abiathar feature to export all levels' images in one shot. That seems like something that would be legitimately useful every once in a while, so I added a little feature to make that possible. If an automatic filename is set in the config file, holding Control while clicking Export Image checks whether the user actually intended to dump all the images and then runs the normal exporter for every level.

FMod - Zip To

An Abiathar user suggested on PCKF that the Level Inspector be able to take the user directly to the site of potential problems. My first thought was that I didn't have a nice way to highlight a tile and still have normal tools work. So instead, I wrote a method to automatically pan the level so that a given tile would be under the current mouse position. I called the command Zip To; it appears under the View menu and can be called with the backslash key on the keyboard.

Then I took advantage of that functionality to add a Zip To button to the Inspection Results window. Using that or double-clicking an entry closes the dialog and moves the level to put the spot in question under the cursor. That's much easier than taking note of the coordinates and mousing around to find them in the level.

Thoroughly disabling OneDrive in Windows 10

The Windows 10 OneDrive app really likes to pop up my OneDrive folder every so often. Since I don't use OneDrive on most of my devices, I like to turn it off to minimize annoyance. There isn't a very convenient way to do it, but it can be disabled with Group Policy. The relevant setting is this one (Policy Plus unique ID):

Microsoft.Policies.OneDrive:PreventOnedriveFileSync

In the normal hierarchy, that's Computer Configuration → Administrative Templates → Windows Components → OneDrive → "Prevent the usage of OneDrive for file storage."

Enable that, and OneDrive will stop bothering all users on the computer.

BIOS refusing to boot a hard drive? See if it's disabled

Today I looked at a computer that wouldn't boot: it said there were no bootable drives, even though I was sure the drive in it had an OS. I swapped out the drive to another bootable one and got the same problem. I poked around a bit and ended up in the BIOS settings, where I found a section that allows individual SATA positions to be enabled or disabled. (It was a Dell BIOS, and I believe the specific settings were in the Drives section.) Apparently, someone had disabled all the positions, and therefore the computer found no drives to boot. I enabled all of them, which produced some warnings about there not being drives in the extra four positions, but it booted anyway.

KB3213986 may cause "delayed or clipped screens" in 3D applications

I was just reading up on the newest Windows updates before pushing Install, and I noticed that KB3213986 has an entry in the "known issues" section that says this:

Users may experience delayed or clipped screens while running 3D rendering apps (such as games) on systems with more than one monitor. 

The article notes that running in windowed mode solves the problem, as does having only one monitor connected when starting the program. Still, this is something to take note of if you depend on fullscreen 3D programs working smoothly.

Ghost works on slightly damaged volumes

I recently rescued a dying drive that refused to accept writes. It had some corruption, but chkdsk couldn't do anything about that thanks to the inability to write. Therefore, I had to move the data to another drive before fixing the file system. I fired up Ghost, receiving some warnings about there being problems with the file system. Nevertheless, it successfully created a disk image on an external drive; I then wrote that image to a new disk. I was afraid that Ghost would try to interpret the damaged data structures in a way that made things worse, but it seems to have handled it well.

FMod - Fixing small bugs

I recently fixed a few small bugs in Abiathar:

• Export Image used to fail if an automatic naming pattern was specified and the dependency file had not yet been written to disk. Now, it ignores the problematic placeholders if necessary.
• An extra Tools menu used to be left behind when closing a project when Simultaneous Tileset was on. I added a bit of code to the close routine to reset all the menus to how they originally appear.
• The title bar used to show "<new project>" after using Project Settings even on a saved dependency file. That bar is now updated correctly after reloading the edited project.

FMod - Moving away from Dropbox

Dropbox recently announced that it's removing the Public folder feature; all sharing will now be done through individually managed links. That's a serious problem for Abiathar updates, since Abiathar requires files to have predictable names and those links include a big random component. Therefore, I'll be stopping my use of Dropbox for Abiathar hosting.

Since I recently purchased some hosting for the PCKF and Keen Modding, I can use that. Today I copied the Abiathar files from my Dropbox to that server and changed the Abiathar code to look for updates in the new place. I currently have the files accessible at a subdomain, but I'll probably move them to a subdirectory instead so that if I get an SSL certificate for the forum, the updates will also be secured.

FMod - Unmanaged patches

One concern I and some Abiathar users have is that it's easy to forget that Abiathar is managing the patch file and then go on to modify the generated file; those modifications then get overwritten next time Generate Patches is used. There has always been a big warning at the top of the file about that, but even so, it's inconvenient to have to open up Abiathar and click a couple menu items just to tweak the patches.

Today, I added some logic to detect external modification of the patch file. If any new lines are added in the clearly marked spot, Abiathar offers to add them to the dependency file (ADEPS). If the user accepts, the unmanaged patches are added to the normal user-created patch section in the Patches window. The user can decline, in which case Abiathar will continue preserving them when writing out the automatic and in-Abiathar user-specified patches.

I also changed the patch emitter to not write out the CKPatch executable every time. If a file already exists with the right name and size, Abiathar assumes the patcher program is already ready to go.

2016: Rise of the Super User posts

Looking back through the archives while compiling the Best of 2016, I noticed that this year, I wrote a lot of posts based on my Super User answers. That is, I first answered a question on Super User, then thought it would make an interesting blog post, so I trimmed it down a bit and posted it here. Browsing Super User - and Stack Exchange sites in general - is a great way to run into interesting problems that actually need real-world solutions. Evidently, my answers have helped some people (my "impact" counter on the activity page is over one million people now), and I hope republishing them here helps a few more.

I present the collection of SU-based answers.

Markeen - Threaded generation

Markeen can take quite a long time to generate a full level set now that the generator takes multiple passes and has more complex logic. The level generation itself takes place on a single thread, so the problem of generating several levels for one batch is clearly amenable to multithreading. So today, I rearranged the outermost loop to instead queue one thread pool work item for each level. As the levels come in, they are added to the level set object.

Sure enough, building a level set now goes a lot faster. The progress indicators in the console are now intertwined with each other, which looks pretty crazy and conveys no information; I'll need to figure out a system to show the progress. At least Markeen can now take full advantage of the system's processing power.

Fleex's Lab: Best of 2016

The following posts are, in my opinion, the most interesting or useful ones I made on this blog in the past year.

• SSL certificate acquisition security and Outlook Online groups
• When "An attempt was made to query the existence of a blank password for an account" is logged
• Reversing the Sybase SQL Anywhere "SET HIDDEN" action
• The correct format of the fSMORoleOwner field on the Infrastructure object
• Renaming libraries on Windows 10
• Changing Windows display fonts in Windows 8 and newer
• When the Windows System Image Manager fails catalog creation with 0xC1420127 or 0xC1420117
• Suppressing special folder views from networked machines
• What are those \??\ (backslash double question mark backslash) paths in Windows?
• ntuser.pol: How Windows knows which Registry settings are from policies
• An arbitrary dive into assembler and certain Windows internals
• Where Windows 7 stores pinned files and folders
• You can customize the Recycle Bin more if you redirect folders
• A guide to installing Abiathar on Ubuntu 14.04
• Using Chrome's --disable-extensions-except
• The surprising OWNER RIGHTS principal

Welcome to 2017.

Abiathar Online - Remember the scroll position

The real Abiathar remembers the viewport positioning of each level when switching between them. That's especially important when going to the tileset to fetch a tile; resetting the level view back to the upper left upon return is jarring. So today I made Abiathar Online keep track of the scrollbar positions for each level and tileset. It required a little refactoring to make sure that all "change the current level" calls go through the part that stores the current X and Y in the level, but overall it was an easy and worthwhile change, and is now live on GitHub.

Abiathar Online - Dropbox Saver now supports data URLs

Over a year ago, when I last tinkered with Abiathar Online (the web-based Keen Galaxy editor), I was greatly inconvenienced by the Dropbox Saver's inability to deal with data URLs. Without those, I had to put up a server to store the files until Dropbox can download them.

As of several months ago, that feature has finally been added! I already updated Abiathar Online to make use of it, so I can dispose of the echo server's code. The only parties involved in Abiathar Online I/O are now just the client web page and Dropbox.

The changes are live on GitHub.