Invoke functions by pointer

Previously, my P/Invoke command-line tool (which is in desperate need of an actual name) was unable to work with COM objects because the tool requires a library and name for functions it calls. COM objects appear from CoCreateInstance and their methods are accessible in their VTable, which is a member of each object/struct. To use them, I implemented the ability to call methods by a function pointer held in a slot. Currently, doing that requires some seriously ugly script, but at least it's now possible at all.

As a not-super-useful but concept-proving demonstration, this creates an IFhConfigMgr and calls AddRef:

newslot native fhPtr
call ole32.dll!CoCreateInstance /return uint (blockptr(guid {ED43BB3C-09E9-498a-9DF6-2177244C6DB4}), nullptr, int 1, blockptr(guid {6A5FEA5B-BF8F-4EE5-B8C3-44D8A0D7331C}), slotptr fhPtr)
newslot native fh
copyslot fh = fhPtr dereferenced
newslot block vtbl = nullptr, nullptr, nullptr, nullptr
copyslot vtbl = fh dereferenced
newslot native addref
copyslot addref = vtbl field 1
call funcat addref /call thiscall /return uint (slotdata fhPtr)

The funcat (that's "function at", not "fun cat", even though cats are pretty great) keyword on the call command is the new addition that does all the heavy lifting.

FMod - Multiple tour watchdog planes

When I last left off working on the Abiathar tour, I noticed that since the watchdog render plane can only be registered for one level plane, it won't pick up the user's tile placing if it doesn't happen to affect that plane. So today, I made the tour extension register three watchdog planes, one for each level plane. The foreground is special-cased to be the only plane that triggers advancement past the task that asks the user to adjust the foreground only.

.NET bitness matters when using ODBC drivers

.NET assemblies can be compiled in the AnyCPU configuration, in which they can work on either 32-bit or 64-bit machines but have a preference for one or the other. For most applications, this probably doesn't matter too much. However, it definitely can cause issues if native modules are in play, like ODBC drivers. Some ODBC drivers are only registered for one bitness, so trying to start a connection using one might mysteriously fail on a machine with a different architecture. This caused some trouble for me in a recent deployment.

In these cases, it's probably easiest to compile for only the architecture with known-working ODBC drivers in the deployment environment.

Where PowerShell stores the Ctrl+R history

One user wanted to know where PowerShell stores the console history that's searched by the Ctrl+R feature so that it could be brought across to a new machine. PSReadline stores it here:

%APPDATA%\Microsoft\Windows\PowerShell\PSReadline\ConsoleHost_history.txt

Migrating the console history is as simple as copying that file over (though some directories might have to be created).

SQL Server Management Studio might need to be run as administrator

Today I needed to attach a database to an SQL Server instance. When I had it attempt the attachment, the operation failed, claiming that there was an access denied error about the database file. That seemed strange because my user account was the one that had extracted the database file, so I clearly had the rights to modify it. Nevertheless, restarting the management tool elevated made the operation succeed.

FMod - Tour fixed

Today I figured out why the presence of the tour panel made keyboard events not go to the main form. As I suspected, the "exit tour" link gets focus when it's visible. The solution was to respond to the GotFocus event of that link (and also of the Proceed button) and set the focus to some inert label instead.

With that sorted out, I continued writing the tour. It now instructs the user to switch to a different level (since actual levels are more interesting than the world map), pick up tiles, place them, and adjust plane states.

Interacting with elevated windows from Chrome Remote Desktop

Under most circumstances, the assistant in a Chrome Remote Desktop connection cannot interact with windows from elevated programs. This is due to Windows's User Interface Privilege Isolation mechanism. This makes system configuration tasks difficult through Chrome Remote Desktop. I suspect disabling UAC would do it, but there is another workaround if the host would prefer not to do that. These steps apply to the host (nothing special needs to be done by the helper to initiate the session):

1. Make sure Chrome is closed
2. Launch Chrome Remote Desktop from the Start menu as administrator
3. Connect as normal
4. If necessary, Chrome can be reopened now
5. Launch an elevated command prompt
6. Accept the UAC dialog on the host machine
7. Launch any necessary administrative tools from that command prompt

The helper cannot access the secure desktop, so any elevation prompts will render the session inoperable. Since child processes of elevated processes are also elevated, all elevation must be done by using that command prompt.

FMod - Struggling with the tour

Today I wrote a barebones watchdog render plane that allows the user to proceed to the next step in the tour after the viewport has been moved around a bit. That part works great, but while testing it, I discovered a problem caused by the presence of the tour panel (probably the "exit tour" link specifically): keystroke events are no longer delivered to the form. I remember going through similar issues back when I added the zoom slider, but the previous solutions do not appear to do anything here. I'm sure it's surmountable, but for the moment it's putting a damper on the whole tour thing.

FMod - Starting the tour

Today I began writing the tour for Abiathar. It will consist of several "stops", each of which has a blurb of text in the left pane that explains things and asks the user to try them out. The extension that manages the tour does so by listening to various event bus stops, then advancing to the next tour stop if the instructions for the current one have been carried out. So far, it can listen for the "register file config" event, which indicates that a new project has been created.

FMod - Help menu adjustments

In keeping with the next Abiathar version's theme of being more helpful to new users, I recently reviewed and updated the contents of the Help menu. The GitHub issue tracker was never used and isn't as convenient as just posting to the forum, so the Report Bug item has been removed. The PCKF Thread link was still pointing to the temporary domain for the forum, so I changed it to now go to the classic one. The manual PDF is extremely out of date, so the Documentation link has been replaced with a link to the KeenWiki page on Abiathar, which has a brief intro to the editor.

FMod - Need a new OOBE

I realized a while back that Abiathar's out-of-box experience, introduced a few versions back, could stand to be improved. Currently it throws up a box with sixteen (!) checkboxes and a paragraph of text. Asking the user to look over that many settings, especially when unsure what effect they have on the editing experience, is probably not very welcoming. Also, I have still not solved the problem of how to guide users in the basics of Abiathar.

Today I decided that all of those issues need to be fixed. I'm introducing a "tour" that will replace the OOBE config screen. On first launch, the user will be given the option to start the tour. Abiathar will then walk the user through the process of creating a level set, getting around the editor, and making some changes.

So far, I've designed the new OOBE window (which is much prettier than the old one), rearranged the contextual help window to also accommodate the tour guide, and set up the infrastructure for tour mode. When the user starts the tour, Abiathar will load an extension that it implements itself (like the standard toolkit). That allows Abiathar to not have all the event bus stops registered all the time, maintaining performance.

Getting a Unicode code point from a .NET string

It's easy to convert a .NET Char value into a number, but that gets UTF-16 code points, which are not necessarily Unicode code points. The code points for emoji, for example, don't fit in two bytes, so UTF-16 (used by .NET strings) stores them as two Chars, a high and low surrogate, neither of which are valid Unicode characters on their own. To get the real Unicode code point that starts at a given position in the string, you can use Char.ConvertToUtf32, supplying the string and the starting index.

Relevant Super User answer.

Keeping cookies in PowerShell web requests

PowerShell's Invoke-WebRequest cmdlet can be used for all kinds of HTTP requests. For stateful web applications, it might be necessary to retain and retransmit cookies in subsequent requests. Conveniently, the cmdlet has features for maintaining a session.

The first request to the site should specify a variable (by name) into which the session should be saved:

iwr https://example.com/Page1 -SessionVariable session

Then all later requests supply that object as the session:

iwr https://example.com/Page2 -WebSession $session

Based on my Super User answer.

Checking whether the domain is available in PowerShell

One user wanted to know how to run a startup script only if no domain controller was available. My solution was to run a PowerShell script that checks if the domain is reachable. That's easily done by running this line:

[System.DirectoryServices.ActiveDirectory.Domain]::GetComputerDomain() | Out-Null

If it throws an exception, the domain is unavailable. If it proceeds without error, a DC could be contacted. A try/catch block can be used to make decisions based on whether the exception appeared.

Searching all event logs with PowerShell

For this Super User answer, I needed to search all event logs - not just the five standard ones under Windows Logs - for a bit of text. PowerShell's Get-WinEvent does the job:

Get-WinEvent -ListLog * | ? { $_.RecordCount -gt 0 } | % { Get-WinEvent -LogName $_.LogName -MaxEvents 100 } | ? { $_.ToXml().Contains('text') }

If you need to look at events far in the past, you can adjust the -MaxEvents 100 or remove it entirely; the process will just take a while.

FMod - Level 0 detection fix

A new Keen Modding user wanted to know how to open a variety of mods. On one of them, Abiathar didn't show the world map and saving the levels caused the game to crash when going to the map. That is because Abiathar didn't see that the first level slot (level 0, the world map) actually had a level; the MAPHEAD pointer to the header was zero because the header was at the very beginning of the GAMEMAPS (which usually doesn't happen). FMod/FleexCore2 has some heuristics to deal with this, but they were not working in this case.

The problem was that, when writing the level 0 detection code, I used Location instead of Length in one spot, so it checked the plane pointer rather than plane length. That check was intended to make sure that the plane array wasn't absurdly long, which would indicate that the "header" was actually random garbage. Fixing the typo (and upping the maximum viable length a bit just for safety) solved the problem.

Extracting information from certificates on disk using PowerShell

One user wanted to determine some properties of a certificate that resides in a CER file on disk rather than in a certificate store. (If it were in a store, it would be accessible from the Cert:\ pseudo-drive.) I'm not sure if there's a cmdlet for this, but it's very easy to do with the X509Certificate2 class, since that has a constructor that takes a filename of a certificate on disk.

[System.Security.Cryptography.X509Certificates.X509Certificate2]::new('C:\path\to\cert.cer')

Keen Modding Live - Import existing levels

Today I implemented the Import Current Level feature for Abiathar Live Studio. This allows a level belonging to a non-KML level set to be transformed into a new KML level held in a new ADEPS with new level files. Importing uses the same UI as creating a new level, but the episode dropdown is automatically set correctly by default by consulting the DefaultSet.

Keen Modding Live - Found an Abiathar UI bug

Abiathar Live Studio, the desktop client for Keen Modding Live, uses Abiathar's DoVisibleLongOperation API to accomplish work while showing a progress spinning instead of freezing the UI. That dialog was originally written for the AbiatharOS installer (which can take a while to do its work) and only later made into an API. However, I apparently never changed the form's title from "Installing AbiatharOS" to something more generic. Since it would be a little strange to publish an Abiathar update to only make a fix that doesn't affect vanilla Abiathar, I instead used some reflection in Abiathar Live Studio to make the form say "One Moment" instead. When I next update Abiathar, I'll incorporate the real fix.

Keen Modding Live - Level properties window

Today I implemented a way to alter a Keen Modding Live level's properties from the Abiathar Live Studio extension. Since only levels already published to Keen Modding Live have any properties, the option is only enabled after publication; the ADEPS stores the KML level ID.

This provides an easy way to mark a level as ready for public consumption from inside Abiathar - just change the zone from "developing" or "beta" to "showtime." One need not even visit the website.

There's not currently a way to view previous versions or manage the currently endorsed version, but that's one of the next things to do.

Keen Modding Live - Edit level metadata

Today I implemented an API endpoint for Keen Modding live that allows users to change the metadata of levels they own. It supports altering the name, canonical version ID (after checking that the specified version actually belongs to the level), description, and zone ID (again, after verifying that the zone exists). The next step is to add a level properties window to Abiathar Live Studio so that these settings can be changed from the desktop. I should also eventually implement similar functionality on the site.

Policy Plus gets a mention

I got curious about the recent uptick in Policy Plus issue filings, so I did some searching to see if it was mentioned anywhere. Sure enough, I found a fairly new mention on the Microsoft Partner Network. A few posts down the page, a Microsoft support engineer noted that Policy Plus can be used on Windows 10 Home to make a relevant Group Policy adjustment. Apparently, at least one person inside Microsoft knows about the tool.

Modern Windows apps can be launched by protocol

Today I stumbled upon the fact that even simple, non-Internetty modern/Metro Windows apps can be associated with URI protocols. For example, the Calculator will come up if you run calculator:// from the Run dialog - or even from Chrome's address bar.

You can explore protocol registrations in the Registry under this key:

HKEY_CLASSES_ROOT\Extensions\ContractId\Windows.Protocol\PackageId

Each application's subkey has an ActivatableClassId, which contains one subkey for each protocol. The protocol name is stored in the Name value of the CustomProperties subkey.

Policy Plus - Fix crash after searching for policies

A helpful Policy Plus user reported a crash bug today. If the search feature is used to locate a policy before the user opens any categories and the setting editor window is closed with the OK button, Policy Plus will crash with a NullReferenceException. This happens because the UI-smoothness-enhanced ShowSettingEditor function tries to determine whether the current category is still visible, but if no categories have been visited, that variable is null and things break.

Adding a null check fixed the problem. The Debug build has been updated.

Recovering a GPO from the Registry with Policy Plus

If the POL files become corrupted, as they did for one user, Group Policy tools may be unable to open them. In particular, Policy Plus will be unable to start. It is still possible to use Policy Plus to save the policy data from the Registry and incorporate it back into a POL file.

First, move the Registry.pol files out of the folder(s) corresponding to the affected section(s) in C:\Windows\System32\GroupPolicy. If you're not sure which sections are affected, do them both - it doesn't matter where; we're just making a backup in case you want to do something with the originals.

Policy Plus will now be able to launch; it will create a blank POL file to replace the ones we moved. Use File | Open Policy Resources to open the Local Registry for the affected section(s). Use Share | Export POL to gather the policy-relevant Registry branches into a POL file for each section. This works because even though the POL is damaged, the Registry settings contained therein are likely still safe in the Registry. Make sure to save different POL files if you're doing both sections.

Use File | Open Policy Resources again to open the Local GPO. Use Share | Import POL for each affected section to bring in the appropriate POL file you created just now. Then you can use File | Save Policies to commit the newly restored GPO POLs.

Policy Plus - Fixing things when refreshing

Today I noticed that there are a handful of other events in Policy Plus that should have the possibility of moving up the category tree if the current category no longer contains any policies visible under the filter. Fixing that was just a matter of replacing an UpdateCategoryListing call with one to MoveToVisibleCategoryAndReload.

I also noticed recently that OKing out of the setting editor dialog refreshes the category listing and puts the scrollbar back to the top, losing the focus on the selected policy. This is somewhat jarring, so today I fixed it. UpdateCategoryListing now preserves both the scroll position and the selected policy if the current category is unchanged.

These changes are live on GitHub.

Policy Plus - Resizable setting editor

Today I finished implementing this feature request on Policy Plus, making the setting editor dialog resizable. I considered using a fancy layout panel to handle the expansion of the two halves of the form when the form gets wider, but instead I just went for a bit of math in the Resize handler. It determines how much extra width is present now relative to the original size, then evenly distributes that extra to the two halves. Finally, it calls the method that reassigns the maximum size of each element-based control. This all leads to a very smooth resizing experience, even more so than in the original LGPE.

Policy Plus - Remove horizontal scrollbars

A Policy Plus user noticed that, unlike that of the official Local Group Policy Editor, Policy Plus's setting editor window cannot be resized. Resizing Policy Plus's might be desirable because a horizontal scrollbar sometimes appears in the extra options panel, which requires the user to scroll it to see the ends of the controls. I investigated and found that this happens when there are so many controls that a vertical scrollbar appears, taking away a little piece of horizontal space for the already-dimensioned controls, creating a horizontal scrollbar. I addressed this by keeping a list of resizable controls and adjusting their MaximumSize when the table size changes and after the extra options space is set up. I have not yet made the full form resizable, but this change should provide most of the convenience.

Policy Plus - Filter by configured policies

A Policy Plus user suggested that the filtering feature be extended with the ability to show policies that are configured - either enabled or disabled, but not unconfigured. This makes a lot of sense, e.g. one might want to easily look over all the customizations applied via Group Policy. Today I implemented that request. There is now a Configured option in the Current state dropdown in Filter Options that works as requested.

The changes are live on GitHub.

Measuring page size with Edge

Today I needed to determine how much total bandwidth it took to load a given page. Conveniently, the developer tools in Microsoft Edge can show this. On the Network tab, there is an item in the status bar that shows how much data has been transferred. You can also filter the requests by type (e.g. image) to see how much is taken by each kind of resource.

Keen Modding Live - Log files cleanup

I noticed recently that using the Test feature in Abiathar Live Studio (to actually play the level one's developing) results in the DOSBox output and error log files being sprayed into the project working directory, not the test environment. The problem was that Abiathar keeps the current directory the same as the directory containing the ADEPS, and the current directory is inherited by subprocesses like DOSBox unless another is specified. Fixing the problem was just a matter of manually setting the new DOSBox process's current directory to the test directory.

Keen Modding Live - Studio settings

Previously I wrote some code for Abiathar Live Studio that detects the DOSBox installation and, if it can't find the EXE automatically, asks the user to specify it in Settings. Until today, there was actually no way to use a custom DOSBox path because the Settings item did nothing. Now, there's a little dialog that allows the user to change the DOSBox installation used or the temporary directory.

Keen Modding Live - Play any version

For a while now, Keen Modding Live has supported multiple versions of a given level, but there was no way to actually play any version except the current one. Today I adjusted the player page to take a version ID rather than a level ID. Since people might want to share links to the level in its most recent version, the page can also take a level ID, in which case it automatically redirects to the current version. Now I need to implement a way to explore the versions of a given level.

Policy Plus - Check for functional Group Policy infrastructure

Last time I worked with Policy Plus, I found that there isn't a clean way of detecting whether the current Windows installation has functional Group Policy infrastructure. That led to Policy Plus failing to actually apply the POL settings to the Registry in certain environments. I was hesitant to use a table of SKUs because new Windows editions might add new entries, so my table would need to be continually maintained. Today, though, I noticed on that MSDN page that GetProductInfo will map the current operating system product to a product that existed when the supplied OS version did. I took advantage of that get a product that Vista knew about, thereby covering all Windows versions that Policy Plus can run on.

I also found another bug: PolFile.WillDeleteValue incorrectly interpreted a deleter value's presence to mean that the value will not be deleted, which is obviously a problem. I corrected that and published these fixes to GitHub.

Keen Modding Live - Publish

A pretty important part of Abiathar Live Studio is the ability to actually publish the level to Keen Modding Live without the friction of going to the web site and picking the files. Today I implemented the Publish feature, which prompts for the level title, the zone, and the description, then uploads all the necessary files. On subsequent publications of the same level, it doesn't prompt for anything and instead uploads a new version. I think this is pretty slick.

It was a little tricky to implement the POST upload because .NET 4.0 doesn't seem to have a way to do multipart/form-data requests without crafting the entire body manually, which I really didn't want to deal with. .NET 4.5 makes this pretty easy, but Abiathar targets .NET 4.0 so that it can run on Windows XP. I discovered today, though, that loading a higher-targeted DLL at runtime actually works fine if the system has the more updated Framework. Abiathar Live Studio is now targeting .NET 4.5, but can still be loaded by Abiathar on Windows Vista and newer.

Keen Modding Live - Zones

My vision for Keen Modding Live includes the concept of different stages of level readiness. Some authors might have a beta version ready to be played, but not suitable for listing in the main marketplace. Today I started implementing "zones" to which levels can be assigned. There are currently three: Showtime, Beta, and Developing (names yet to be finalized). Showtime will be the primary zone for players browsing the site; people will be able to explore the others if they want. At publication time, level authors select the zone - this is implemented now. Soon there will be a way to reassign the zone of a level, e.g. once it's ready for the general public.

Policy Plus - Windows 10 1709 ADMX definitions

I realized a few days ago that with Windows 10 1709 released to the public, people might want to use the new Group Policy settings it introduces. So I went and found the official MSI package, made sure it still has the same directory structure as the previous packages, and updated Policy Plus to get the new MSI.

In other news, a user actually ran into a problem that I speculated about: installing the Local Group Policy Editor on Home editions with DISM results in a nonfunctional version of the official Group Policy editor that nevertheless confuses Policy Plus into thinking it's off the hook for applying the policies to the Registry (because secpol.msc then exists, even though it shouldn't on Home). I have not yet found a reliable way to check whether the current machine supports Group Policy, so I asked a question on Stack Overflow.

PowerShell surprise: script current directory is not the process current directory

PowerShell supports paths expressed relatively to the current directory. You can do things like gc .\mydoc.txt to get the contents of mydoc.txt in the current directory. Sometimes, though, using relative paths will mysteriously fail with a file-not-found error. This happens when a .NET I/O function is called before the path is resolved by PowerShell. Changing the current directory in script (the one that displays at the prompt) doesn't necessarily affect the process's current directory, which you can get with [System.IO.Directory]::GetCurrentDirectory().

Most commonly, the problem is that a script calls a .NET Framework function directly, but some third-party modules may forget to do the resolution before trying to use the path. To set the process's current directory, you can use [System.IO.Directory]::SetCurrentDirectory.

Keen Modding Live - Log in on the desktop

Today I implemented the Keen Modding login flow in Abiathar Live Studio. Just like on the site, the user can enter their name and password. If there is no Keen Modding account by that name linked to Keen Modding Live, the program offers to link them and then proceeds to log in. The API key received from logging in is stored in the Registry so that the user doesn't have to log in at every Abiathar run. There's also a Log Out menu item if the user wants to invalidate the API key (or all API keys for their account).

With login implemented, the next step is to allow the user to publish their levels to the site.

Keen Modding Live - Better login experience

Today I rearranged the authentication part of Keen Modding Live a bit. Now the Keen Modding authenticator indicates whether an account needs to be linked or the logon just failed. The login page uses that information to only show the Link Accounts button if makes sense to even try. This adjusted API will make for a convenient login experience in Abiathar Live Studio as well, which is my next thing to implement.

Keen Modding Live - Easily test levels

Today I continued working on Abiathar Live Studio, the Abiathar extension that will serve as the desktop client for Keen Modding Live development of Galaxy and Dreams levels. When preparing the development environment for a new level, it now also downloads and unpacks the full episode package, staging it in the testing directory. I have it do this eagerly (at level creation) so that all necessary data is stored locally in case users want to test their levels when they don't have Internet access.

Speaking of testing levels, it now automatically detects the DOSBox installation so that zero configuration is needed in most cases. The user can select a difficulty under the Test menu; DOSBox immediately launches into the level. Under the hood, Abiathar Live Studio saves the project, copies the level files to the testing zone, writes the DIFFICU.LTY file, and starts DOSBox.

The list of which files are level files is stored in the project file in case the Internet goes out and the episode metadata becomes unavailable. Extra configuration options are stored in the project file through Abiathar's configuration section registration API: when the level project is created, Abiathar Live Studio injects the new configuration section, then gives Abiathar a config template object when the project file is opened later.

Keen Modding Live - Downloadable development environments

A goal of the Keen Modding Live editor-integrated desktop client is to make it extremely easy to start building a level. The user should just have to pick an episode, choose where to save their files, and be ready to go. Today I finished implementing the New Level feature. After the user makes their choices, it fetches the development package corresponding to the selected episode, extracts the package to the folder containing the new ADEPS file, and opens the ADEPS.

Development packages are different from normal episode packages in that development ones contain only the bare minimum complement of files needed to make Abiathar work: the ADEPS and nearly empty map files (a template). The other files needed to play the game are in the episode package. So far, I've just made the Keen 4 development package as a proof of concept; doing the others will be easy.

Keen Modding Live - Keeping session info

Dropbox and a few other websites have a section that allows the user to view and invalidate sessions that are logged in to their account. They keep track of the user agent used to sign in so that the user can tell which browser the login is from. That kind of fine-grained session management might be a little overboard for Keen Modding Live (the "log out" API endpoint already provides a way to end all sessions), but keeping track of user agent is very easy, and might be useful for the occasional curious user.

The "log in" API endpoint now remembers the user agent and the logon method (linked Keen Modding account vs. the not-yet-implemented e-mail/password option).

Output of event handler script blocks goes to the event object

One user was wondering how to output from a script block registered to an event to the script that set the event handler. Register-ObjectEvent returns a job object corresponding to the new event, which the setting script can store. That object has an Output property that is an array of the values produced. When the event handler block returns a value (just like normal functions do), PowerShell stashes it in the Output array.

Keen Modding Live - .NET API use

Today I continued sketching out the Keen Modding Live extension for Abiathar. It now has some infrastructure to make requests of the server via HTTP. To deal with the JSON responses in .NET 4.0, I'm using the JavaScriptSerializer class. In terms of actual functionality, the extension can now fetch the list of episodes from the server and present the supported ones in a list.

After I started adding login-related menu items, I decided to keep all the extension's menu items in a dedicated top-level Live menu rather than sprinkling them around the File menu. This way, it's much clearer which items refer to KML levels. Speaking of login operations, I realized that the user experience of always being asked after a login failure whether to try linking a Keen Modding account is kind of weird. I might change the login flow on the web site to only provide that option if there is no linked Keen Modding account.

Keen Modding Live - Internal reorganizing

A lot of the server-side API endpoints for Keen Modding Live included various helper files. Since these other files can't stand on their own, it doesn't make sense for clients to be able to request them directly. It was also cluttering up the main folder, so I wasn't sure which files were importable and which were public pages/endpoints. So today, I moved the includes to their own folder, denied direct HTTP access to those files, and adjusted the remaining pages to point to the new relative path.

Besides the web site, another important component of the Keen Modding Live system is a desktop client to make testing and publishing easy. For Galaxy and Dreams levels, that will be an Abiathar extension. Today I started the skeleton of the extension, with a bit of a road bump from this laptop's screen being 4K - designing forms in the Visual Studio designer at high DPI makes them display incorrectly on lower resolutions, so I had to disable Visual Studio's DPI awareness.

Keen Modding Live - Version chains

I put together the level versioning system on the assumption that the time uploaded would be sufficient to identify which versions are based on which, but today I decided it would be better to have that information stored explicitly. That way, there can be a tree of versions rather than just a line, and the correct chain can be preserved even if the original author publishes a new version in between someone else starting and releasing an alternative one. The "upload level" API endpoint now takes a baseversion parameter if an existing level ID is supplied.

Keen Modding Live - Dreams live

Today I did some quick tests on the live-ified version of Keen Dreams before registering it in the Keen Modding Live episodes table. I noticed that keys were not zeroed when the level was restarted, so I repurposed the EKA-score-zeroing code to zero keys instead. (Since I disabled the Extra Keen At mechanic, its high word never went above zero and therefore didn't need to be reset.) Then I put in the normal map loading patches, packaged the files, and uploaded it.

When testing, I noticed that I had the MAPDICT size wrong in the level upload API endpoint and that I had forgotten to implement the "use the Abiathar default dictionary" feature on the server side. I corrected both of those oversights and Dreams is now live.

Keen Modding Live - Dreams patched

Today, armed with the relative-patch-enabled version of KDRPATCH, I made a patch that makes Keen Dreams reset all stats to their "new game" values when running the complete level code. Since I changed the death action to be the same as the level completion action, the reset happens either way a level is restarted. The call to the stats initialization routine ($03C00003RL) replaces the call that plays the level completion sound, since it would be strange to play that on death. I also changed the default lives to be zero, since there is no concept of extra Keens in live games - the player gets infinite fresh starts. This should be sufficient patching for Keen Dreams. I'll probably get it up as a usable episode tomorrow.

Keen Modding Live - Still working on Dreams

As a follow-up to the main "start level 1 immediately" patch, Levellass helpfully provided another that skips the B800 text screen at the beginning of Keen Dreams. With the assistance of IDA, I figured out how to disable the Extra Keen At mechanic. Now I just need to determine how to zero the score and lives when restarting. I suspect that will involve a call to the function that sets up the blank game state. To prepare for that, I had to hunt down a version of KDRPATCH that supports relative values - the one available from KeenWiki doesn't even support word literals; I had to split the words in Levellass's patches into bytes to make them work with my copy. Eventually I found the updated version bundled with Abiathar. I have no idea where I got that from originally, but it works.

Keen Modding Live - Starting to live-ify Dreams

Since I was unable to figure out how to patch Keen Dreams to skip the menu and go straight into the first level, I posted a patch request, which Levellass answered today. It works great; the only undesirable side effect was easily worked around. I used some other patches from KeenWiki to make the level always restart on death or completion, just like the live Vorticons episodes. I still need to figure out how to zero various stats when restarting.

I also added a new API endpoint that returns information on the versions of a given level. This is necessary because version records are the only place with the filename of the level package, so it would be impossible for API clients to get the actual levels without this. While testing it, I noticed a bug in the "get levels" endpoint - if no filters are supplied, a PHP warning spew precedes the JSON result in the response. I have corrected the problem that caused the warning.

Keen Modding Live - Search levels

Today I added an API endpoint that returns information on the levels in the system, optionally filtered by ID, a substring of the title, or the owner user's ID. Allowing for multiple filters turned out to be a bit tricky because it required piecing together an SQL query at runtime, which isn't difficult if using raw SQL, but that seems treacherous, so I had to figure out how to get parameter binding to work with an unknown number of parameters. call_user_func_array worked great for making the actual bind_param call, but getting the parameter references into an array was tough because array_push doesn't like references as of PHP 5.3. Weirdly, array_merge is fine with them, so I used that to build the array of parameters.

Keen Modding Live - Versioning

One of the goals for Keen Modding Live is to allow people to suggest adjustments to others' levels. That requires a notion of multiple versions of a single level, which seems like a nice thing to have anyway so people can publish updated versions of their own levels. Today I did some rearranging of the existing infrastructure to allow for multiple versions.

The levels table no longer has any information on the level package file, but rather has the ID of the current/suggested version in the level versions table. The level versions table stores the version's package metadata, the version creator's user ID (which may be different from the level owner's user ID), and whether the version is an official revision of the level. My plan is to allow level owners to merge in changes that they like, thereby making the revision part of the official level history. Currently, though, the level upload API endpoint just assumes that revisions by the original author are official and revisions by others are not. The level deletion endpoint currently torches all versions, official or not, of a deleted level, which was easy to implement but might be worth reconsidering.

Determining the volume of a specified file in PowerShell

NTFS makes it possible to mount one volume in an empty directory of another. Files on the mounted volume will be accessible through paths that appear to be on the host volume. One might wonder, then, how the true volume can be identified given a full path. I wrote a script in this Super User answer that does exactly that. It works by listing the mount points via WMI and finding the one that accounts for the largest chunk of the file path. Then it just issues a Get-Volume on the determined volume so that a volume object is returned.

Do your batch scripts expect directory changes across volumes?

I have a batch script that copies some files around, invokes a PowerShell script to transform them, and deletes extra copies of the originals. The script resides in the same folder as the source files, executing a cd to an output folder before running the script that will change them. That way, the alteration and eventual deletion will apply to the copied files and not the only originals. I keep the source folder in my Dropbox, which on the machine where I wrote the script is on the C drive, but on this computer is on D, separate from the rest of my profile. When the script tried to cd to the output folder (under my profile, on C), it didn't actually change the current drive or the current drive's current directory because the /d flag was not specified. It then went on to transform and delete the originals. Fortunately, since it was in Dropbox, restoring them was easy, but I would still have preferred that it didn't happen. In the future, I'll be careful that my batch scripts are prepared for cross-volume operations.

PowerShell script properties cannot be parameterized

PowerShell classes don't support properties with getter/setter methods, but Add-Member can add script properties with script-bodied getter and setter methods. Unfortunately, there isn't a way for these properties to take parameters like properties can in VB.NET, nor can the custom object be indexed like in C#. I did some poking around in the PowerShell source and determined that these downfalls cannot be worked around - arguments are never passed to the getter and setter, other than the current object (scriptThis) and the new value to the setter, of course.

The only workaround that I know of is to use Add-Type to generate a class from C# or VB.NET code. PowerShell script can then index the object with the normal bracket syntax, or use indexed properties with parentheses, almost like calling a method.

Keen Modding Live - /nowait

Today while looking for a tedlevel equivalent for Keen Dreams (which I did not find), I learned about the nowait switch for Galaxy episodes. It causes the game to skip the B800 text screen at startup and therefore load a little quicker. It certainly did make a little difference when I tried it locally, but on the web there's still a little B800 interlude, probably since JavaScript emulation of machine code isn't super fast and therefore needs some time to get things set up. Nevertheless, every smoothing out of the experience helps.

I still haven't managed to devise a way to make Dreams begin a level immediately, so I posted another patch request on the forum. Once I have that, getting Dreams live should be fairly easy.

Keen Modding Live - Almost all episodes live

Today I finished live-ifying and testing Keen 6. It's now a usable episode, so that completes the Vorticons and Galaxy series. To put on the finishing touch, I added a patch to all Galaxy episodes that removes the "if" ammo placement, since it was erroneously always added after one failed attempt at the level due to a side effect of some other patches. All three Galaxy episodes' packages have been reuploaded. This shows the benefit of the two-package system: I can fix problems with the episode package without needing to fiddle with every single uploaded level's package.

To get started on Keen Dreams, I wrote the server-side code for handling the Dreams (three-file) genre. It was a fairly easy adaptation of the Galaxy file-combining code. Now I need to figure out all the necessary patches for Dreams, which might be a bit tough since I don't know if it has a tedlevel mechanism.

Keen Modding Live - More live episodes

Levellass very helpfully provided all the Vorticons patches I was still missing for Keen Modding Live, so today I finished off Keen 2 by editing the map Keen sprite and world map level to be completely black while the real level is loading. Armed with that experience and all necessary patches, live-ifying Keen 3 was a breeze. All Vorticons games are now permissible episodes for upload and play. Currently the only way to install new episodes into the system is to upload some files directly into the server and then manually tweak the database to register them, but it's not terribly difficult and I won't have to do it very often. There was a tiny wrinkle in the database management - the "tell me about all your episodes" API endpoint returns the episodes sorted by their ID in the database, and the JavaScript in the upload page just displays the episodes in the order it gets them. Since I added Keen 4 out of order, that presents the episodes in a bizarre order. So I added a sprinkle of JavaScript to sort them by DisplayName.

With Vorticons handled, I moved on to the remaining Galaxy episodes. Using my previously fashioned live version of Keen 4 as a guide, I browsed for the necessary patches from KeenWiki. I had to make my own tedlevel difficulty setter once again, but my Keen 4 version of that made me perfectly comfortable figuring it out with IDA. With that, Keen 5 is live.

Keen Modding Live - Patch finagling

Since Keen Modding Live is about individual levels rather than whole games, I want to remove as much of the inter-level bookkeeping as possible. Ammo, score, and pogo should all reset on death. All the necessary patches existed for Keen 1 on the KeenWiki, but not for Keen 2 and 3. I tried to translate some more of them, but was unsuccessful. Fortunately, Levellass came to the rescue and made the "don't keep score gained in level when dying" mechanic work for all episodes. I now have a complete patch set for Keen 2, so all I need to do to get that live is make the world map completely black. For Keen 1, I accomplished that simply by painting the entire map level with black tiles and graphics-editing map Keen's original sprite to also be black - that way, nothing else shows up while the real level is being loaded.

Keen Modding Live - Galaxy level handling

Keen Vorticons and Galaxy are very different, so much so that even though Keen Modding Live mostly just has to keep track of files, it needs to treat Vorticons and Galaxy levels differently. In the episodes table, there's a column for "genre," currently set to either a V or a G.

When uploading a new level, the client specifies the episode ID, and the genre determines which files are expected. For Vorticons, that's easy - it only takes one. For Galaxy, it needs both GameMaps and MapHead files, but the player infrastructure expects a single level-specific package. Therefore, when a Galaxy level is uploaded, the system checks that the MapHead is the right length, concatenates the GameMaps onto the end of the uploaded MapHead file, uses that as the package, and generates a JSON manifest explaining to Emscripten DOSBox where the GameMaps begins. Dreams handing isn't currently implemented, but it'll basically be just the same plus a MapDict.

It would be convenient if the system accepted Abiathar single-level (ASLEV) files; then people wouldn't have to worry about keeping the level in slot 1 in the GameMaps. That would require quite a bit of work, though, since ASLEVs are LZW-compressed and that takes a lot of bit twiddling, which doesn't look particularly fun to do in PHP. Perhaps a desktop client (.NET, which can then load FleexCore2) could implement that client-side.

Introducing Keen Modding Live!

Modern Keen mods are getting more and more complex, so releases are infrequent. I think it would be really neat if people felt free to release small creations, especially single levels. It would also be good if players didn't have to fiddle around with extraction and DOSBox setup - optimally, they would just find the level in a repository and click to play. Such a system could also open doors for collaboration by keeping track of a chain of inspiration/editing.

For a while, I've been doing a little bit of poking around to make sure that these ideas are feasible. Though I'm pretty busy now, I think it'll be possible to build it, albeit slowly and at low priority. The site will be called Keen Modding Live. Currently it's possible to register and log in with Keen Modding forum credentials, upload Keen 1 or Keen 4 levels, and play uploaded levels. To allow for level editors to facilitate easy upload, I'm building a web API. To make sure the API is actually useful, and to avoid duplicating effort, my plan is for the web site to use client-side script to hit those same API endpoints.

Fixing an almost-working patch

Last time, I attempted to translate the "don't check whether Keen actually has the pogo before pogoing" patch from Keen 1 to Keen 2. It almost worked - Keen could always pogo starting from the ground, but not the air. My start was this:

%patch $6958 $EB $05
%patch $6BD0 $EB $0C

First I needed to find which of the two lines affected which check. When I commented out the first, ground-based pogoing stopped working, so that one was fine; the second line was the problem. Since I can't read most machine code, I fired up IDA and jumped to 0x6BD0. Bizarrely, I found that it wasn't actually near a comparison with the pogo variable - it was right on the stack-pointer-incrementing tail of some other function. Down a little bit was a pogo check (which I identified by having the variable named in IDA based on the correct half of the patch).

Having found the actual jump instruction at $6BF3, I tried patching the jump condition to be unconditional ($EB). That did something in that the pogo never worked in the air, even after acquiring it with C+T+Space. I don't know if there's a jump condition that never jumps, but since it was a check for equality ($74), I replaced the previous byte - the number the variable is compared to - with something ($FE) that the pogo variable never gets set to. The fixed second line is this:

%patch $6BF2 $FE

So, why didn't the patch at $6BD0 work, even though I found the right sequence of bytes? Turns out I had transcribed the location from XVI32's left column, which shows the address at the start of the line, not the address of the selected byte, for which I should have looked at the bottom left in the status bar. Oops - turns out patching right before sleeping might not be the greatest idea.

Translating a patch from one episode to a similar one

I recently wanted to patch Keen 2 to always allow Keen to pogo, even if the "have pogo" flag wasn't set. (In the original Keen 2, Keen starts with the pogo, but a side effect of another patch removed that.) Conveniently, that patch already existed, but only for Keen 1 and 3. Since it looked pretty simple, I decided to translate it myself.

I started by loading a Keen 1 dump into XVI32 and jumping to one of the two relevant addresses. Starting at $3B16, the original bytes are $83 $3E $9A $AA. Searching for that string in a Keen 2 dump unfortunately turns up nothing. However, since I knew that the code has something to do with the pogo variable, I consulted the game stats table. $9A $AA refers to the data location $AA9A. The analogous location in Keen 2 is $9AB0. Searching for $83 $3E $B0 $9A finds two locations in Keen 2: $6958 and $6BD0.

So I replaced the Keen 1 patch locations with those and ran it and it appeared to work, but it didn't quite feel right. Careful observation showed that the pogo only worked if Keen wasn't in the air, but it should work either way. We'll pick it up next time to see what went wrong.

You can't elevate an existing PowerShell prompt, but you can relaunch PowerShell in the current directory

One user wanted to know how to elevate an existing PowerShell prompt to administrator. Unfortunately, this is not possible because the primary token of a process cannot be replaced after it starts running. It is, however, very simple to prompt for the elevation of a new PowerShell window, and the current directory can be sent over via a startup command.

Start-Process powershell -Verb runas -ArgumentList "-NoExit -c cd '$pwd'"

Note that the $pwd (an object which stringifies to the current directory path) is expanded in the original process. Usually, starting PowerShell with startup commands would cause it to exit after finishing those commands, but -NoExit leaves it around for interactive use.

Securing the PHPSESSID cookie

One of my PHP web applications uses PHP's session functionality to remember the current user. I noticed, though, that the PHPSESSID cookie that's generated for me isn't marked Secure or HttpOnly even though I'm exclusively serving the site over HTTPS and I never need JavaScript to do anything with the cookie. Therefore, as a defense-in-depth measure, I looked into setting those flags on it. Using php_flag in .htaccess didn't work; it caused a 500 error noting in the server log that the command was unknown. The ini_set approach didn't seem to have an effect on my server. Using set_session_cookie_params did the job, though. I'm including this line before session_start:

session_set_cookie_params($cookieTime, '/', $myDomain, true, true);

Acquiring Let's Encrypt certificates on Windows

For a while, I've been getting my Let's Encrypt certificates (in manual mode) in an Ubuntu VM or in the Bash on Ubuntu on Windows environment. Some recent update, however, seems to have broken the BoUoW approach, and since I didn't want to hassle with a VM, I looked for an alternative method. I remembered that someone mentioned ACMESharp, a PowerShell client for Let's Encrypt. It comes with a very helpful quick start guide. I used the HTTP challenge with the manual handler. The only issue I encountered was that Update-ACMEIdentifier with a specified -ChallengeType always returned "pending" even after waiting a while, but supplying only the challenge alias worked as expected, returning a "valid" status very quickly. I was then able to generate a certificate and get it signed.

Emscripten DOSBox has to wait for the filesystem to load

While implementing a difficulty selector for Keen Galaxy levels in Emscripten DOSBox, I needed to inject a file into the virtual filesystem at runtime - not from a downloaded, manifested package. At first, I tried calling FS_createDataFile right before using the pre-supplied loadPackage function, but something necessary was missing, because the file wasn't added and an error was logged to the console. Inspecting the loadPackage function carefully, I found that it checks calledRun on the DOSBox module and, if that's not set, queues the operation with Module["preRun"].push (initializing Module["preRun"] to an empty array if it's absent).

I refactored that pre-run queuing out into its own function, altered the loadPackage function to use that, and called the new function with the data file creation as the callback.

When wireless mice start scrolling very slowly

Recently, my wireless mouse decided to switch to an extremely slow scroll speed. Each roll would go about a tenth as far as it should, which made reading long pages rather inconvenient. Turning the mouse off and back on didn't do anything. Unplugging and replugging the USB dongle, however, did. The problem immediately went away and scroll speed returned to normal.

Setting Keen Galaxy difficulty with Emscripten DOSBox

Today I set up Keen 4 in the Emscriptenified version of DOSBox with the tedlevel parameter supplied so that the game goes straight into the specified level. Unfortunately, using that mode always sets the difficulty to Normal; attempts to change it by starting a new game in the running program just deadlock it. I first figured out how to patch the tedlevel difficulty: it's a word at $3BBE from 1 (easy) to 3 (hard). The patch script uses %patchfile to load a given file at that address. That file, one byte long, is injected into the DOSBox filesystem at page load using FS_createDataFile with the data appropriate for the user's selection.

PHP error spews may lead to bogus data being passed around

Today I was debugging a web page wherein a JavaScript function uploaded data to an API endpoint written in PHP and then received data from that endpoint to use in a page redirection. After adding a feature to the endpoint but making only small changes to the client-side script, I was puzzled to receive this error in the Chrome console:

[Deprecation] Resource requests whose URLs contained both removed whitespace (`\n`, `\r`, `\t`) characters and less-than characters (`<`) are blocked. Please remove newlines and encode less-than characters from places like element attribute values in order to load these resources. See https://www.chromestatus.com/feature/5735596811091968 for more details.

It was logged, as far as I could tell, after the sending of an AJAX request, but I was very confused because the URL that request was POSTed to definitely didn't have any whitespace or less-than characters. When I tried the request in Postman, however, I found that I received some errors from the PHP script that my JavaScript was interpreting as part of the URL to redirect to after the action completed. Sure enough, the HTML tags used to format the notices had less-than characters, and there were newlines in the spew. I had forgotten to step through the callback's setting of window.location.href. Fixing the PHP script led to the return of valid data for a URL fragment, which fixed the problem.

gpedit.msc only edits the GPOs, which might not actually get applied

Today I tested a pretty neat-looking method of getting the Local Group Policy Editor onto Home editions of Windows. It uses only DISM and package files that come with the Home edition. Sure enough, gpedit.msc exists and runs after using it. Changing the Administrative Templates settings persisted across runs of the LGPE, so it was clearly saving them somewhere, but they never wound up in the Registry and therefore had no effect. Therefore, installing the LGPE correctly isn't enough; you also need to have the infrastructure that transcribes the POL contents into the Registry. (Policy Plus does the transcription itself if necessary.)

This DISM approach does work nicely, though, for the things outside the Administrative Templates branch that I tested. Therefore, it's a great complementary strategy to Policy Plus, which currently has no support for anything outside Administrative Templates. The only disadvantage is that it also installs secpol.msc, which is what Policy Plus checks for to determine whether it's on a Pro edition, so that's something I should probably change for users' convenience.

Setting an HKCU Registry setting for all users on the machine

Suppose there's a per-user Registry setting that, for your users' convenience, you'd like to set for everyone on the machine. If there's no policy for it, the standard Group Policy tools won't do the job. Nevertheless, the Group Policy infrastructure will be happy to set any Registry setting you specify, with the caveat that if it's not in a policy branch, it won't go away if you change your mind and unset it in the GPO.

Normal tools won't let you set arbitrary settings, but Policy Plus will. You can use the Import REG option (either the menu item under Share or the button in the POL editor) to apply a REG file to the GPO. Alternatively, you can skip the file and use File | Edit Raw POL directly. If you have a Group Policy-enabled edition of Windows, the change will be applied to your user account and to all others when they next log on.

Unfortunately, on Home editions, there's no way to disseminate user settings except by manually loading each user hive and changing them individually, but Policy Plus will happily apply them to your account when you click Save Policies.

Internet Explorer has to be run before Invoke-WebRequest will work

While testing a web API endpoint with PowerShell's Invoke-WebRequest, I encountered this interesting error message:

The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer's first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again.

Sure enough, supplying -UseBasicParsing made the request work, but the parts of the response object that required Internet Explorer components were expectedly absent. Apparently, this happens because I had never run Internet Explorer on this computer (I used Edge, not IE, once to download Chrome). Launching it, answering its question about security settings, and closing it made Invoke-WebRequest work as normal.

Checking a phpBB login from a new page

Suppose you want to use a phpBB forum's database of users as an authentication method for some other service. It might be possible to figure out the database format and connect directly, but then you miss out on the automatic tracking of failed logins and checking that the account is enabled. So, you might instead create a new PHP script in the existing phpBB site to take advantage of the phpBB infrastructure:

define('IN_PHPBB', true);
$phpbb_root_path = './';
$phpEx = 'php';
include('common.php');
include('includes/functions_user.php');
include('phpbb/passwords/manager.php');
$provider_collection = $phpbb_container->get('auth.provider_collection');
$provider = $provider_collection->get_provider();
$result = $provider->login($username, $password);
if ($result['status'] === 3) {
 echo 'Login successful.';
} else {
 echo 'Login failed!';
}

The first three lines are important even if they don't immediately appear to do anything - the included phpBB files check them. Also note that if you're getting the username and password to verify from GET or POST parameters, you'll need to extract them into variables before including the phpBB files because phpBB seems to somehow disable superglobals; attempting to use them results in an error page.

Some of the above script is from this forum post.

Notification icons need an icon if they are to appear

While helping this user, I briefly worked with notification icons (NotifyIcon) from PowerShell. It's possible to load the icon image from an ICO file, but this is suboptimal for that user because there weren't any desirable icons in a consistent place on all machines. Omitting the line that sets the Icon property's setting results in no notification at all - apparently there has to be an icon. Fortunately, there is a SystemIcons class that provides a handful of normal system icon objects. Applying one of these makes the notification work and removes the need to keep a loose ICO file around.

Apparently there can be multiple nameserver sets for a domain

I recently tried to get the Public Commander Keen Forum back to its classic pckf.com domain after one user noticed that the redirect wasn't working. (Background: the previous owner of the forum didn't own the pckf.com domain, so we couldn't get it transferred during the migration, so a redirect page was placed on the old server instead.) I found that the nameservers had been updated to point to my hosting, so I flipped a couple switches to make the forum available at pckf.com. That appeared to work, but intermittently accessing it would go into redirect loops or throw certificate errors. When there were problems, the domain was resolving to the old server. Some investigation with Google's DNS lookup tool revealed that sometimes the old set of nameservers was returned (so the browser went to the old server), but sometimes the new set was provided (so the browser found the right place). Evidently, there's somehow two differing opinions on what the correct list of nameservers is. I'm not sure how exactly that works, but it's causing problems - I'll contact the domain owner to see if we can get it resolved.

Converting embedded images in PowerPoint to JPG

One user wanted to know how to convert a bunch of PNG images already embedded in a PowerPoint presentation to JPG so that lossy compression could be used to reduce the file size of the overall presentation. I'm not aware of a way to do that in VBA, but I know that PPTX files (like all modern Office documents) are actually ZIP archives, so I took advantage of that fact to write a script that opens a PPTX, converts all PNG files in /ppt/media to JPG, and puts them back into the archive. Interestingly, PowerPoint doesn't seem to care that the extension doesn't match the image format, so that made the job easier. I also included an optional -Quality parameter that specifies how good the resulting images need to look. The full script can be seen in my answer.

Unplugging extra devices might help the system boot

This morning after getting the main machine partially back up and running, I found that it again declined to boot properly. However, after unplugging both the 3.5mm headphones and the webcam, it passed POST and then came all the way up. It even allowed me to plug the headphones back in without causing a power failure, though I haven't tried the webcam yet. I'm quite sure there are PSU problems, but for now I can limp along by minimizing the power draw at POST time.

The official repair shop on campus apparently only services Apple devices (?), so I'll need to find some other place to get a new PSU acquired and installed.

Mostly back up and running

Last night, my computer suddenly stopped working. Today I fiddled with it some more, first by removing an old parallel port card that was installed quite a while back in an attempt to get an old printer working. After pulling that out, the system actually made a beep shortly after boot-up, and turned the screen on when I plugged in an HDMI cable! It wanted to run a disk check, so I let it, which only took a moment since it was only concerned about the EFI system partition. With excitement, I shut the machine back down to plug in all the rest of the peripherals, then brought it back up. It failed to boot (spun the disks a bit then reset itself) once or twice, but soon it came up to the logon screen. The keyboard didn't work at all despite Num Lock being lit, so I unplugged an unnecessary peripheral (the webcam) and rebooted. After another boot failure or two, I managed to get logged in. Everything now appears to be normal.

It's unclear how much of the fault lies with the parallel port card, but considering that it never accomplished anything other than blue-screening my computer, I decided to discard it. There are clearly remaining issues, though, which I am pretty certain are with the power supply. Alas, I have neither the tools nor the ability to make a definite diagnosis of which part of the PSU is problematic. For now, I'm trying to avoid strenuous workloads for the machine (no VMs for a while). Hopefully soon I'll be able to take the machine to someone more qualified to double-check that the PSU is indeed the problem and get a new one installed if so.

Sudden failure

Tonight I was firing up a virtual machine when the physical computer suddenly froze one screen, switched to mostly black on the other, power-cycled itself, and refused to boot. The hard drive is active for a little while, then all activity ceases; the monitors report "no signal" and no beeps are emitted. I tried various things for no improvement: rebooting again, holding down the power button with no power connected, inserting a bootable USB, and reseating the RAM.

I'm pretty sure the RAM is fine, since there are no beeps. The hard drives are probably OK as well; even if the OS one had completely died while the system was up, it should have shown a bugcheck upon detection or an error upon rebooting. There does not appear to be any physical damage to any components. Currently I suspect a failed graphics card or CPU. Alas, I have no VGA cable to test the integrated graphics, but the graphical artifacts shown on-screen at the first failure make me suspicious of the GPU. I'm concerned about the CPU because the computer seemed to feel slower as of late, though that might just be my getting used to the speed.

My next thing to try is to gently remove the graphics card to see if the boot process will continue with only the integrated graphics. I won't be able to see anything for lack of an appropriate cable, but I'll be able to tell from the hard drive activity. If the problem persists, the CPU is probably at fault. In any case, I'll be sharper at investigating things in the morning.

Delete slots in the P/Invoke tool

My P/Invoke command-line tool previously had no way to remove or redefine a slot once it was created. There currently isn't much use for that since scripts proceed only directly forward, but eventually I'd like it to be possible to write loops and branches. If there's looping going on, it might be necessary to reallocate a buffer. Also, in interactive mode, it would be nice to be able to redefine a slot with a different type if a mistake was made in creating it. So, I recently added the deleteslot instruction, which removes a given slot from the current workspace and frees the shred of memory used to store its data.

Figured out the EGAHEAD problem

Last time, I found a set of Galaxy graphics files that Abiathar had trouble dealing with. Today I had time to dig a little deeper. It turns out that though the EGAHEAD had real markers until its end, it ended too soon - the last mark was not the end/length of the EGAGRAPH file. The final two foreground tiles' ending points were simply absent. Therefore, there's no conceivable way any tool could be expected to bring that data into existence, so it's not an Abiathar/FleexCore2 bug.

I ran the game and went to the place where I expected the missing tiles to cause problems. To my surprise, there were no display issues! Then I checked with the Tile Property Modifier and found that the last two Keen Dreams tiles (part of the animation cycle for the left edge of the flames), are not actually an animation target of anything, nor are they placed in any level, so they are never called on and their absence has no visible effect.

When EGAHEADs end too early

Today I helped someone get a tricky set of files opened with Abiathar. The graphics failed to load, even when using the graphics files specified by the patch file and the correct EGADICT. Some debugging revealed that the search for the next real EGAHEAD entry (to calculate the length of the chunk) was reaching the end of the the marks array without finding a non-placeholder entry. Looking at the EGAHEAD in a hex editor, I'm confused as to why that is, since there are real entries all the way to the end of the file. Perhaps there's something wrong with the optimization intended to reduce the number of chunk decompressions - I will need to investigate further.

Reducing the reported number of foreground tiles made the load work. I don't know whether the game is able to access those last two, but if it is, that's definitely a FleexCore2 problem. I'll have to check.

Using multiple packages in JavaScript DOSBox

There exists a JavaScript port of DOSBox, suitable for running DOS applications and games in a web browser. The file system is delivered to the client browser as one big file with the filenames and offsets included in the page as a JSON object. I thought it would be neat if the data package could be split into multiple files so that levels could be easily swapped out without needing to duplicate the other files.

That turned to not be super difficult. I think I spent the most time just trying to get the build to work on Windows and then in WSL, but then I decided it would be easier just to work with an existing JS compilation. I wrote my own minimal packager, which was very easy since it just has to bolt one file on after the previous and keep track of where each starts, 31 lines of PowerShell in all. Then I went to work adjusting the loadPackage function and its associates to deal with multiple packages. I just stripped out some unnecessary stuff (mostly the enormous embedded SVG), added a filename parameter to loadPackage, and sprinkled in some PHP to fetch different packages based on the query string.

Sure enough, changing one part of the query string leaves the game the same except for replacing one level, which has its own package.

Print block slots somewhat helpfully

I noticed today that if my P/Invoke command line tool is instructed to print the value of a slot (readslot) that is a custom kind (i.e. defined with newslot block), it spews an error message that's almost certainly 100% incomprehensible and useless to most users. So today, I added a check for block slots in the readslot implementation to mention the length of the slot instead of attempting to marshal and stringify the value. The architecture is not currently set up such that the fields of a block can be printed; the information on each field's kind is not preserved, only the .NET Framework type, so it couldn't tell the difference between some arbitrary pointer-sized field and a string. Nevertheless, not spewing strange errors is an improvement.

More units for the P/Invoke tool

In a handful of places, my P/Invoke command line tool accepts a unit for a size, so allocslot can take a size measured in chars rather than bytes. It might be helpful, though, to measure size using any kind, e.g. when allocating an array. To allow that, I added two keywords to the unit parser function. sizesof takes the name of a kind (e.g. native). This makes chars somewhat unnecessary since sizesof short will work the same, but chars is more memorable and one of the more common expected units, so it's reasonable to have a shortcut. blockssizedlike takes the name of a slot and looks at its kind; this is helpful for working custom structures defined with block.

Explorer has trouble copying symlinks across volumes

While testing my answer to this question, I noticed that Explorer's copy/move operation ground to a halt when copying a symbolic link from one drive to another. It pegged one CPU core and wedged itself so severely that I had to terminate the process. This happened multiple times, even when I just moved the symlink without the surrounding directory hierarchy. My workaround was to perform the move of the directory structure using the robocopy tool:

robocopy C:\source\root E:\dest\root /e /move /sl

Controlling "Animate windows when minimizing and maximizing" from script

One user wanted to know how to configure the "Animate windows when minimizing and maximizing" performance setting from the command line. There is a Registry entry that directly represents that checkbox, but tweaking that value doesn't cause a behavior change until after a logon/logoff cycle. The correct way to programmatically adjust the setting is via the SystemParametersInfo function, so this calls for some PowerShell and P/Invoke. (It would be a great opportunity to showcase my P/Invoke command-line tool, but it's not ready for release yet.) The call is made somewhat tricky by the need for a special structure, so jamming the entire thing into one PowerShell run would be unwieldy, but it's fine as a script:

Add-Type -TypeDefinition @"
    using System;
    using System.Runtime.InteropServices;
    [StructLayout(LayoutKind.Sequential)] public struct ANIMATIONINFO {
        public uint cbSize;
        public bool iMinAnimate;
    }
    public class PInvoke { 
        [DllImport("user32.dll")] public static extern bool SystemParametersInfoW(uint uiAction, uint uiParam, ref ANIMATIONINFO pvParam, uint fWinIni);
    }
"@
$animInfo = New-Object ANIMATIONINFO
$animInfo.cbSize = 8
$animInfo.iMinAnimate = $args[0]
[PInvoke]::SystemParametersInfoW(0x49, 0, [ref]$animInfo, 3)

It takes a Boolean value specifying whether to enable window minimize/restore animations.

Scripts and interactivity in the P/Invoke tool

For some complex operations, it might be unwieldy to place all the necessary P/Invoke instructions in the command line of my tool. It would be preferable to write out such things in an external script file and then simply run that script. Recently, I added a command that does exactly this. The currently executing script is tracked in a stack, so error messages can specify which script file contained the instruction that caused the problem.

Also in complicated scripts, it might be helpful to have an interactive environment to poke around the current execution state. There's now an interactive command that enters interactive mode, in which instructions are read from the console one line at a time and errors are nonfatal (since it would be inconvenient to have to restart the program every time one mistyped a command).

Adding custom Visual Effects performance options to the Performance Options dialog

The Performance Options dialog (accessible from the Advanced tab of the System Properties dialog) loads the list of visual effects from the Registry, specifically from here:

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VisualEffects

Each subkey is one option in the dialog. It's possible and fairly simple to add new entries. Most of the default ones use the SPIActionGet and SPIActionSet values to specify the uiAction parameter to SystemParametersInfo. A couple, however, are backed by Registry values, so new checkboxes can be added to control Registry values elsewhere.

These values are necessary to define a Registry-backed checkbox:

• HKeyRoot, a DWord that specifies the common Registry handle to use as a root. This should probably be 0x80000001 (for HKCU).
• RegPath, a string specifying the key path under the root hive.
• ValueName, a string specifying the name of the value in the specified key to control with the setting.
• CheckedValue, a DWord containing the data to set into the specified value when the box is checked.
• UncheckedValue, likewise if the box is unchecked.
• DefaultValue, a DWord with the default data for the backing Registry value if it does not exist.
• Type, a string set to checkbox.
• Text, a string holding the display name of the setting. This can be the literal setting title or, if it starts with @, a resource from a DLL.

Configuration for a new setting

The resulting setting

Checking and unchecking that box does indeed set the corresponding Registry value.

Error reporting in the P/Invoke tool

Previously, if my P/Invoke command line tool encountered an error (like bad syntax from the user), the entire program would crash and exit, giving no feedback about what caused the problem other than the exception stack spew to the standard error stream.

In the interest of user-friendly error messages, today I added an exception handler that prints the current instruction number, current instruction text, and exception message when an exception is thrown as a result of processing an instruction. For parse errors, the parser also supplies the fragment currently being parsed (which can be smaller than the current instruction) and prints a caret pointing at the current position within that fragment.

Wacom tablet pen clicking when it shouldn't

Yesterday, the Bamboo tablet that's worked fine for years suddenly developed a critical problem: simply hovering the pen above the tablet causes a click after a second or so. This makes the device basically impossible to use, but fortunately I have a wireless mouse around for back-up. Some research reveals that the issue is probably caused by the pressure sensor in the pen being somehow stuck or out of position. Removing the tip with tweezers and blowing any dust out has worked for other people, so I will try that when I find tweezers or some other instrument that can remove the pen tip.

Policy Plus - Download ADMX Files for non-included languages

A couple weeks ago, an issue report was filed on Policy Plus stating that the Download ADMX Files feature was unable to move the ADMX files. Today I discovered that the user account this was happening on was set to en-GB (with pt-PT probably the system language). The ADMX package from Microsoft only includes a few common ADML languages, but Policy Plus always attempts to move the ADML subfolder named the same as the current culture. If the current culture's code isn't an included subfolder, that fails.

The solution was to first check that the ADML subfolder exists, and if it doesn't, just fall through to the English files. Language-related bugs are one of the perils of developing only on an English system.

The fix is live on GitHub.

Output of formatting cmdlets is not suitable for use as the represented object

New PowerShell users sometimes get confused between an object and its textual, formatted, on-screen representation. PowerShell provides a handful of formatting cmdlets (those with the Format verb), whose output can be much more informative than the default stringification. Of course, formatting an object still creates a string, which is not the same as the original object even if it prints identically. Even putting a string through a formatting cmdlet might get extra whitespace onto it that can confuse other cmdlets. Format-Wide with -Property does this, for instance.

Policy Plus - Antivirus false positives

I noticed yesterday while downloading Policy Plus in a VM for testing that Edge's (and/or Windows's) SmartScreen check marked the EXE as malicious. Surprised, I uploaded the file to VirusTotal, which informed me that one engine (SentinelOne) considered it bad. There was an automated comment from PayloadSecurity with a link to an an also probably automated analysis report.

The report listed a handful of "suspicious indicators", all of which are confusing to me. "Reads the active computer name" and "Reads the cryptographic machine GUID" are listed, but Policy Plus does neither, at least not directly; it's conceivable that the .NET Framework does them for some reason. The "Tries to sleep for a long time" section notes that it tried to sleep for 1566804069 milliseconds, but that's over 18 days, and Policy Plus doesn't intentionally delay at all except when waiting for some other relevant operation to complete, which should take at most a couple minutes, not days. "Reads information about the supported languages" makes more sense, since it does consider the current language for ADML management purposes.

More concerningly, the report alleges that Policy Plus contains the ability to listen for incoming connections, but in reality the program has no remote control or network server features whatsoever. The closest it has is the ability to download a setup package from Microsoft to get the newest ADMX files. The report says it contacts one host, but the IP given resolves to a handful of things under googlevideo.com and gvt1.com, both of which seem to be controlled by Google. I don't know why Policy Plus would ever contact a Google server, because there's nothing in the code to do anything like that.

All in all, the report is baffling to me - if it weren't for the screenshot, extracted strings, and assembly information I'd think they got the wrong file. I filed a SmartScreen false positive report with Microsoft and am waiting to hear back.

FMod - WPF surrender

After considering all the advantages and disadvantage of Abiathar's WPF transition adventures, I decided today to revert the Abiathar code base back to before any of the WPF changes. While it would be nice to get the speed boost from a complete conversion, a lot of the Abiathar internal structure is based (directly or indirectly) on graphics working in immediate mode, which is simply not the case with WPF. The current speed is good enough on modern machines, and there are much more interesting advancements to be made than rendering finagling. So for now, I'm sticking with Windows Forms and GDI+.

Classic Windows Backup is interrupted by hibernation

A month or so ago, I looked into getting a full automated backup for my Windows 10 computer. The clearly recommended choice from the OS is the new Backup section in the modern Settings app, but it didn't seem to want to back up all the folders I specified. Some got backed up and I could see them on the target volume, but others just weren't copied. So I went for the classic Windows 7 Backup and Restore, accessible from the non-immersive Control Panel. That appeared to run as I wanted.

Since my machine has a lot of space used, backup takes quite a while, especially the system image phase. I don't like to leave my machine on overnight, and it's rare for me to stay up late enough for the whole image to finish. I regularly hibernate the machine, but I found once I brought it back into wakefulness that the backup was considered cancelled. That appears to be the major downfall of this strategy - the computer must remain on for the system image to be taken.

Dangerous commands in Bash on Ubuntu on Windows can affect the host

One user wondered whether it was possible for dangerous commands run inside a Bash on Ubuntu on Windows prompt to affect the Windows system. It is possible, because there are mount points to the host file system. For example, /mnt/c goes to the Windows C volume.

I ran a little experiment to see how much a command inside WSL could damage the host. In a Windows 10 VM's Bash prompt, I ran the system-destroying sudo rm -rf / --no-preserve-root. After it finished, Linux was broken and all the test Windows user's personal files and folders were deleted, but the Windows system itself was still fine. Then I rolled the VM back to before the command and tried it again, but with Bash run as administrator. This time, more Windows files were deleted, and after a reboot, the VM bluescreened with CRITICAL_SERVICE_FAILED.

Clearly, dangerous commands inside WSL can damage the host Windows system. Untrusted code shouldn't be run inside the Bash prompt.

What does the SuperHidden Registry value control?

There is a value called SuperHidden under this Registry key on some versions of Windows:

HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced

It's reminiscent of Hidden (which controls whether the user sees hidden files in Explorer listings) and ShowSuperHidden (which controls whether the user sees system files in those listings). One might wonder what SuperHidden controls.

And the answer is... nothing! Using Process Monitor, I observed I/O to it and found that it only gets written when the View tab of Folder Options is shown. It's never read, even when Explorer is starting, so it can't control anything. So why is it there?

Investigating the stack provided by Process Monitor led me to shell32.dll, which issues a write to that value in CachedShellState::SaveAdvancedSettings. The corresponding function CachedShellState::_GetAdvancedSettings, however, issues a read from ShowSuperHidden, the value that actually does something. I also noticed that on Windows 10, the inert SuperHidden doesn't exist, and sure enough the Windows 10 version of shell32.dll writes to ShowSuperHidden.

Therefore, I strongly suspect that the existence of SuperHidden was a typo that was later corrected.

Based on my Super User answer.