PSThar's ForEach-LevelTile command creates a new scope

I noticed while writing PSThar scripts, most recently this one, that the ForEach-LevelTile cmdlet (abbreviated %t) has different scoping behavior than the normal PowerShell ForEach-Object (abbreviated %). The script block that runs for each level tile is more of a distinct scope from the parent than is the script block run for each object by %. Using variables from outside the %t block in the block pretty much requires explicitly scoping those variables with script: every time they're mentioned, otherwise the variable name inside the block doesn't refer to the variable of the same name outside the block. This is very inconvenient and I will look into tweaking PSThar to fix it.

Keen Modding Live - Level comments API

Today I started on a comment system for Keen Modding Live. For logged-in users, comments will appear at the bottom of the player page and possibly the level information page; I haven't decided yet. Comments record what version they were created on (to reduce confusion from level changes across versions) but the same set will appear for all versions' player pages, since not all changes will necessarily affect all comments. I'm currently planning on a flat chronological list of comments without any fancy threads. Replies (to notify a given comment's author) could still be possible.

So far I've written the four necessary API endpoints for comment support: add, edit, delete, and list. At the moment I'm not storing revision history, but the time of last edit or deletion is recorded.

Setting the hotkey for context menu items created with the Registry

One user was experimenting with creating file context menu entries with the Registry and encountered some interesting behavior about the hotkeys for activating them with the keyboard. If multiple entries have the same hotkey (shown with the underline under a letter), pressing that letter cycles the focus through them; if the hotkey is unique, pressing it immediately activates the corresponding entry.

The hotkey for a custom entry can be set by putting an ampersand before an instance of the desired hotkey letter. Interestingly, for that to work properly, the entry's name has to be set in the default value of the entry's Registry key, not just with the key name. If an ampersand is included in the key name, the hotkey letter will be underlined, but pressing the key will do nothing.

Start-Process with -Credential specified may have problems with interactive programs

One user encountered a problem while using PowerShell to start a command prompt as another user. They used the -Credential parameter to pass a credentials object (acquired with Get-Credential) to the Start-Process cmdlet, but found that the resulting command prompt didn't accept keyboard input. I don't know why it's doing that, but I do have a workaround: use the .NET Framework's Process.Start method directly:

[System.Diagnostics.Process]::Start('cmd', $creds.UserName, $creds.Password, '')

Keen Modding Live - Approve/disapprove versions

Today I made the approve/disapprove toggle in Keen Modding Live's version tree actually do something. First I added an "edit version" API endpoint that currently only allows changing the approved status. Originally I was planning on having it be impossible for an approved version to be derived from an unapproved one, but then I thought it made sense to be able to hide uninteresting/broken intermediates. So I adjusted the grid preparer to show descendants of hidden versions as direct descendants of the hidden versions' parents if disapproved versions are not shown. Then, of course, I wired up the toggle link to the API endpoint.

An unapproved version showing in the chain leading to the current version

The same chain with the unapproved version hidden

Keen Modding Live - Version tree

Today I wrote a bit of JavaScript to render the version tree for a given Keen Modding Live level. It's still rough from a UI perspective, but it exists:

Lower records are newer/derived versions. The two at the very bottom are both derived from the one with the green box. (Connecting lines/arrows are not yet implemented.) The green box indicates the current version. Faded out records are "unapproved" versions that only appear if "show all" is checked. The "fork" link goes to the upload-level page with that version preselected. "Set current" sets the level's current/default version to that one. "Approve"/"disapprove" toggles whether the version is approved - this is not yet implemented.

Keen Modding Live - Starting the version tree

Today I started working on the version tree: the bit of UI on the Keen Modding Live level info page that should show the versions of the level and the derivation relationships between them. This is turning out to be quite difficult to do in HTML. The approach I've settled on is to first translate the version information into a two-dimensional array like this:

A
B   b
C c
D

("A" is the base version, the main chain of work is all the capital letters, "b" is an alternative version to "B" also forked off of "A", and "c" is an alternative to "C" still derived from "B".) I think I have the grid part done; now I need to render it into an HTML table and make it look nice/understandable.

Keen Modding Live - More info on player page

Previously, Keen Modding Live had no way to get from a player page to the level's information page. So today I populated the player page with some useful information: level description, version comment, level owner, version author (if different than the level owner), and version creation time. While I was at it, I made the player page return an HTTP error and bail out if the level or version ID can't be found instead of spewing dozens of PHP errors. The link that goes back to the level information page reads "see other versions of this level," so I suppose the next thing to do is actually render the version tree on the level page.

Keen Modding Live - Level info page

I've recently continued fleshing out the Keen Modding Live level information page. It now shows the level's name, author, description, creation time, episode, and zone. If the current user owns the level, a "delete level" link appears that, unsurprisingly, deletes the level. Administrators have a link that sets the level's owner to the ID Software pseudo-user. I plan to use that to upload the original levels while making it clear that they're not mine. While testing, I noticed that the delete-level API endpoint is broken: it did not in fact delete the level. There was an issue with foreign key dependencies in the database; setting the level's current version ID to null before cascade-deleting it and the level versions fixed the problem.

Adding the Backup Operators group to a Home edition

Home editions of Windows have fewer built-in groups than Pro editions. Some of those missing groups, however, still have their SID appear in various default ACLs and privilege lists. One such group is Backup Operators.

Creating a new group with the same name as a missing built-in group won't have any effect because the SID will be different. There is also no way to specify the SID of a new security principal... without editing the SAM database in the Registry. To access the SAM of a live system, we need to open the Registry Editor as the SYSTEM account. That can be accomplished with PsExec:

psexec -s -i regedit

Information on built-in groups is stored under this key:

HKEY_LOCAL_MACHINE\SAM\SAM\Domains\Builtin\Aliases

It has one subkey for each group. The subkey names are the zero-padded eight-digit hexadecimal representation of the last fragment of the group's SID. The Backup Operators SID is S-1-5-32-551, so its subkey is 00000227. The Aliases subkey also has a Names subkey, which likewise has a subkey for each built-in group. This time, the subkey names are the group names. The default value of each name subkey is an empty binary value - what matters is the type code of the default value, which stores the last SID fragment.

Both keys are necessary for the group to work properly. They can be exported from a Windows installation that has a desired group and imported into one that doesn't. The group will appear after a reboot.

Most privileges will be removed by UAC

Extra privileges can be assigned to non-administrative users with User Rights Assignment or the NTRights utility. However, UAC will remove all privileges from the user's token except the standard five (shutdown, change-notify, undock, increase working set, and change timezone). Just like with normal administrator accounts, getting the extra privileges requires elevation. If a user is not a member of any UAC-protected groups, though, the UAC elevation dialog will prompt for the credentials of an administrator instead of elevating the current account, making it impossible to take advantage of the privileges. It may also be necessary to be interactively logged on, as opposed to started in a different user's session with "run as different user."

DCOM configuration permissions are Registry key permissions

The DCOM Configuration section of the Component Services snap-in allows modifying the permissions of COM applications. The ability to change the permissions on the Security tab is governed by the ACL in the Configuration Permissions section. Even going into the Advanced dialog on the ACL editor, though, doesn't allow administrative ownership-taking, so even administrators can't always adjust the permissions.

The Configuration Permissions are actually the permissions on the Registry key describing the COM application:

HKEY_LOCAL_MACHINE\SOFTWARE\Classes\AppID\{GUID}

The GUID is the application ID from the General tab of the application's properties window. The Advanced ACL editor in the Registry Editor can be used to take ownership and adjust configuration permissions. After a reboot, the DCOM Configuration snap-in can be used to graphically adjust Launch and Activation or Access permissions.

Backup Operators may not be able to create shadow copies

Yesterday I learned that SeBackupPrivilege and SeRestorePrivilege are not sufficient to create a volume shadow copy. I assumed that membership in the official Backup Operators group would do the job, but after doing some testing today, that may not be the case. Historically, I've used the code in this Stack Overflow answer to create shadow copies while running as an elevated administrator. When I tried it as an elevated backup operator, the first line didn't throw an error as it does for normal users, but the response object it returned was blank and indicated an access-denied condition. No shadow copy was created.

Evidently, even though membership in Backup Operators allows connection to the Volume Shadow Copy Service (or so it appears from the ACL in DCOM Configuration), only administrators can create shadow copies. That matches the response to this other Stack Overflow question.

SeBackupPrivilege may not be enough to use the Volume Shadow Copy Service

A new answer to a Super User question I addressed a while back informed me of something I had missed. I thought that the backup-related abilities of the Backup Operators group were all from the backup and restore privileges granted to it, but apparently use of the Volume Shadow Copy Service requires real membership in that group (or in Administrators). It looks like this might be enforced by the security in the DCOM configuration - only SYSTEM, Administrators, and Backup Operators are granted launch or activation permissions on the application. Users (even administrators) are not allowed by default to edit the ACL on that application, so altering it will be tricky. I will continue to investigate.

Using an existing CSR in Let's Encrypt

I recently needed to get an SSL certificate for a site whose hosting provider generates the CSR. I use Let's Encrypt to get free SSL certificates. I was happy to find that the Certbot client is easily able to request certificates based on a CSR:

certbot certonly -d example.com --csr /path/to/file.csr

Viewing an execution plan for SQL Anywhere 11

I recently needed to investigate what part of a query to an SQL Anywhere 11 database was taking so long. The database software comes with a program called DBISQL, or "Interactive SQL." Today I discovered that it has a section for doing exactly what I needed: Tools | Plan Viewer. Paste/type the query in the top box and press Get Plan, then information will appear below. The most helpful part - the tree - is in a tiny left pane by default and probably not visible, so that will probably need to be expanded. Nodes in the tree can be clicked for more information, like the table being accessed.

Using SetOwner on an access control list may require SeTakeOwnershipPrivilege

One user was having some trouble gaining control of a file. Taking ownership and adding a Full Control access rule to a file's ACL can be done with PowerShell, but there's a significant wrinkle. The SetOwner function on the ACL object, which would allow an administrator to take ownership of the file, requires the process to have turned on SeTakeOwnershipPrivilege. Running elevated gives the process the permission, but doesn't enable it. As far as I know, there's no pure PowerShell way to enable the privilege - a mess of embedded C# to P/Invoke the native API is the only way I've found.

Keen Modding Live - Player page cleanup

I've continued to tweak the design of Keen Modding Live's player page. I realized that it's not really ever useful to keep the mouse pointer around in fullscreen, so the checkbox that controls that has been removed. The difficulty and keybinding controls have been moved up next to the DOSBox widget. (I had to struggle a bit with CSS to keep the DOSBox centered and get all the controls floating where they should be.) I also made the entire game-and-controls section hidden until all downloading and initialization is finished - that way, the user won't see weird half-loaded elements jumping around.

Finding the path to a file mentioned in the USN journal

One user was going through the output of fsutil usn readjournal and wondered about the "file ID" and "parent file ID" rows. The output of that command appears to correspond (with the rows somewhat out of order) to the USN_RECORD_V3 Windows API structure. The IDs mentioned are NTFS object identifiers. They can be converted to a full path with fsutil file queryFileNameById. For example:

fsutil file queryFileNameById C:\ 0x0000000000000000001200000019ab0e

Keen Modding Live - Unifying player page style

I made a couple minor adjustments to Keen Modding Live's web site today:

• The player page now has the same template as all the others. There's still a lot of design that needs to go into that, since it's probably the most important page.
• Level information pages now have an "upload a new version of this level" link that goes to the upload page with the level selected.

Guessing the type of an absent Registry value from Process Monitor logs

One user was trying to use Process Monitor to figure out what a program is looking for in the Registry. The relevant value was not present, so the RegQueryValue operation's result was NAME NOT FOUND. When the query is successful, Process Monitor reports the data type, but it does not when the value is not found. This is because the API call used to query the Registry does not take the desired data type - the Registry just stores chunks of data for the application to interpret. When a value is found, the Registry returns its kind, but if the value isn't there, there's no kind to report.

There is one piece of information that can allow a guess to be made about the value type (in some cases). Applications need to pass in a buffer for the returned data and the buffer's length. Process Monitor reports a "length" even for unsuccessful Registry queries, but (at least on 64-bit systems), its number is 12 bytes more than the application supplied. So subtracting 12 from the "length" gives the size the application is expecting for the results. A DWord occupies 4 bytes and a QWord occupies 8 bytes. All the other kinds (mostly strings) are variable-length.

Setting the icon for an entry in the New context menu

Entries in the New submenu of the Explorer right-click menu are generated by ShellNew subkeys in the Registry. One user wanted to know how the icon for those entries is determined. By default, Explorer will use the file type's icon, but this can be overridden by placing an IconPath string value in the ShellNew key. The format is the same as most resource identifiers: the path to the file containing the icon, a comma, and the numeric resource ID. For example, %ProgramFiles%\Windows Mail\wab.exe,10 is an open book icon. Changes to the New-related Registry entries require a restart of Explorer before taking effect.

Making a shortcut to troubleshoot compatibility for a specific program

One user wanted a fast way to open the Program Compatibility Troubleshooter and start troubleshooting a specific program. The PCT window is provided by msdt.exe, but Process Explorer reveals that it doesn't get passed the path to the program to be troubleshot. Rather, it receives a path to an XML file written by the very short-lived pcwrun.exe. That process exits so fast that I couldn't get a command line out of it, but simply passing the path to the incompatible program's EXE works just like clicking the "run compatibility troubleshooter" button in the Properties window.

Policy Plus gets its first pull request

This morning, a Policy Plus user submitted a pull request. Apparently, some ADMX files (like the Office ones) have blank values for some XML attributes specifying numeric default values. I'm pretty sure that's not compliant with the ADMX specification, but if the Local Group Policy Editor supports it, Policy Plus should too. The pull request fixes the problem by assuming the appropriate type's default (in the .NET sense) value if the attribute cannot be parsed into that type.

Keen Modding Live - URL parsing expansion

The only part of Abiathar Live Studio that needs to know about the non-API parts of the Keen Modding Live web site is the URL format of the level and version pages. The dialog for selecting a level is supposed to be able to extract level or version information from the URL on the clipboard, which it can only do if its idea of what the web site is doing matches up with reality. So today I expanded the pasted-URL parser to deal with level information pages (in addition to version player pages).

While testing, I accidentally chose a Keen 1 level and found that the level picker was happy to go ahead with it. The level search dialog filters out levels of non-Abiathar-compatible episodes, but the URL-pasting route did no such checks. I'm sure all kinds of bizarre behavior would occur if a Vorticons level was selected, so I added some logic to stop that.

Keen Modding Live - New versions via web

A conspicuously absent feature from the Keen Modding Live web site up to this point was the ability to upload new versions of existing levels. It was possible in Abiathar Live Studio through the API, but no such program exists for Vorticons levels. Today I set to work adding that feature to the web site.

I decided that, to avoid duplicating effort, I would extend the existing level upload page to optionally take a level ID as an URL parameter. I started off with a light sprinkle of embedded PHP to pre-fill and disable the fields that were already determined at the level's creation if a level is specified. While I was at it, I added fields for version comment and level description, which weren't possible to set via the web before.

To populate the list of versions (from which to choose the base version for the new one), I embedded a larger chunk of PHP that echoes a JSON array into a JavaScript fragment that then populates the dropdown with it. I would have just used the API from JavaScript like I did with the episode and zone dropdowns, but I wanted to show the usernames of the version authors, and doing that from JavaScript would require at least one more round trip. And I would have had PHP render the <option> tags, but I didn't want to risk messing up the escaping and causing security issues.

I successfully used this new system to upload a new version of my tiny Keen 6 test level, after which I noticed that I forgot to add a way to mark the new version as recommended. So that's the next step.

Keen Modding Live - ZXC fix

Today while poking around the Keen Modding Live site, I found that Keen 1 and Keen 4 levels were completely failing to load when the keyboard preference was Z/X/C (as opposed to Ctrl/Alt/Space). The problem was that I had mistakenly kept the CTLPANEL/CONFIG file in the game folder while packaging those episodes. When the ZXC-enabled configuration file was injected, the DOSBox JavaScript thingy I'm using complained about a duplicate file and keeled over. Deleting extra files and repackaging fixed the problem.

Keen Modding Live - Cleaning up files

With Abiathar Live Studio effectively done, it's time to give Keen Modding Live an actual web site. The pages and scripts that are there made reference to several pages that are either blank placeholders or not there whatsoever. Apparently when I first started arranging things, I had plans for a "dashboard" page for logged-in users, distinct from the homepage. Now I think it would be nicer and simpler to just have a feed of recently published levels be the homepage for guests and members, with another feed showing up on the side for members' personal events. So I went through and cleaned up all the redirects to the nonexistent dashboard page.