Tuesday, October 31, 2017

Output of event handler script blocks goes to the event object

One user was wondering how to output from a script block registered to an event to the script that set the event handler. Register-ObjectEvent returns a job object corresponding to the new event, which the setting script can store. That object has an Output property that is an array of the values produced. When the event handler block returns a value (just like normal functions do), PowerShell stashes it in the Output array.

Sunday, October 29, 2017

Keen Modding Live - .NET API use

Today I continued sketching out the Keen Modding Live extension for Abiathar. It now has some infrastructure to make requests of the server via HTTP. To deal with the JSON responses in .NET 4.0, I'm using the JavaScriptSerializer class. In terms of actual functionality, the extension can now fetch the list of episodes from the server and present the supported ones in a list.

After I started adding login-related menu items, I decided to keep all the extension's menu items in a dedicated top-level Live menu rather than sprinkling them around the File menu. This way, it's much clearer which items refer to KML levels. Speaking of login operations, I realized that the user experience of always being asked after a login failure whether to try linking a Keen Modding account is kind of weird. I might change the login flow on the web site to only provide that option if there is no linked Keen Modding account.

Keen Modding Live - Internal reorganizing

A lot of the server-side API endpoints for Keen Modding Live included various helper files. Since these other files can't stand on their own, it doesn't make sense for clients to be able to request them directly. It was also cluttering up the main folder, so I wasn't sure which files were importable and which were public pages/endpoints. So today, I moved the includes to their own folder, denied direct HTTP access to those files, and adjusted the remaining pages to point to the new relative path.

Besides the web site, another important component of the Keen Modding Live system is a desktop client to make testing and publishing easy. For Galaxy and Dreams levels, that will be an Abiathar extension. Today I started the skeleton of the extension, with a bit of a road bump from this laptop's screen being 4K - designing forms in the Visual Studio designer at high DPI makes them display incorrectly on lower resolutions, so I had to disable Visual Studio's DPI awareness.

Friday, October 27, 2017

Keen Modding Live - Version chains

I put together the level versioning system on the assumption that the time uploaded would be sufficient to identify which versions are based on which, but today I decided it would be better to have that information stored explicitly. That way, there can be a tree of versions rather than just a line, and the correct chain can be preserved even if the original author publishes a new version in between someone else starting and releasing an alternative one. The "upload level" API endpoint now takes a baseversion parameter if an existing level ID is supplied.

Thursday, October 26, 2017

Keen Modding Live - Dreams live

Today I did some quick tests on the live-ified version of Keen Dreams before registering it in the Keen Modding Live episodes table. I noticed that keys were not zeroed when the level was restarted, so I repurposed the EKA-score-zeroing code to zero keys instead. (Since I disabled the Extra Keen At mechanic, its high word never went above zero and therefore didn't need to be reset.) Then I put in the normal map loading patches, packaged the files, and uploaded it.

When testing, I noticed that I had the MAPDICT size wrong in the level upload API endpoint and that I had forgotten to implement the "use the Abiathar default dictionary" feature on the server side. I corrected both of those oversights and Dreams is now live.

Wednesday, October 25, 2017

Keen Modding Live - Dreams patched

Today, armed with the relative-patch-enabled version of KDRPATCH, I made a patch that makes Keen Dreams reset all stats to their "new game" values when running the complete level code. Since I changed the death action to be the same as the level completion action, the reset happens either way a level is restarted. The call to the stats initialization routine ($03C00003RL) replaces the call that plays the level completion sound, since it would be strange to play that on death. I also changed the default lives to be zero, since there is no concept of extra Keens in live games - the player gets infinite fresh starts. This should be sufficient patching for Keen Dreams. I'll probably get it up as a usable episode tomorrow.

Tuesday, October 24, 2017

Keen Modding Live - Still working on Dreams

As a follow-up to the main "start level 1 immediately" patch, Levellass helpfully provided another that skips the B800 text screen at the beginning of Keen Dreams. With the assistance of IDA, I figured out how to disable the Extra Keen At mechanic. Now I just need to determine how to zero the score and lives when restarting. I suspect that will involve a call to the function that sets up the blank game state. To prepare for that, I had to hunt down a version of KDRPATCH that supports relative values - the one available from KeenWiki doesn't even support word literals; I had to split the words in Levellass's patches into bytes to make them work with my copy. Eventually I found the updated version bundled with Abiathar. I have no idea where I got that from originally, but it works.

Monday, October 23, 2017

Keen Modding Live - Starting to live-ify Dreams

Since I was unable to figure out how to patch Keen Dreams to skip the menu and go straight into the first level, I posted a patch request, which Levellass answered today. It works great; the only undesirable side effect was easily worked around. I used some other patches from KeenWiki to make the level always restart on death or completion, just like the live Vorticons episodes. I still need to figure out how to zero various stats when restarting.

I also added a new API endpoint that returns information on the versions of a given level. This is necessary because version records are the only place with the filename of the level package, so it would be impossible for API clients to get the actual levels without this. While testing it, I noticed a bug in the "get levels" endpoint - if no filters are supplied, a PHP warning spew precedes the JSON result in the response. I have corrected the problem that caused the warning.

Sunday, October 22, 2017

Keen Modding Live - Search levels

Today I added an API endpoint that returns information on the levels in the system, optionally filtered by ID, a substring of the title, or the owner user's ID. Allowing for multiple filters turned out to be a bit tricky because it required piecing together an SQL query at runtime, which isn't difficult if using raw SQL, but that seems treacherous, so I had to figure out how to get parameter binding to work with an unknown number of parameters. call_user_func_array worked great for making the actual bind_param call, but getting the parameter references into an array was tough because array_push doesn't like references as of PHP 5.3. Weirdly, array_merge is fine with them, so I used that to build the array of parameters.

Saturday, October 21, 2017

Keen Modding Live - Versioning

One of the goals for Keen Modding Live is to allow people to suggest adjustments to others' levels. That requires a notion of multiple versions of a single level, which seems like a nice thing to have anyway so people can publish updated versions of their own levels. Today I did some rearranging of the existing infrastructure to allow for multiple versions.

The levels table no longer has any information on the level package file, but rather has the ID of the current/suggested version in the level versions table. The level versions table stores the version's package metadata, the version creator's user ID (which may be different from the level owner's user ID), and whether the version is an official revision of the level. My plan is to allow level owners to merge in changes that they like, thereby making the revision part of the official level history. Currently, though, the level upload API endpoint just assumes that revisions by the original author are official and revisions by others are not. The level deletion endpoint currently torches all versions, official or not, of a deleted level, which was easy to implement but might be worth reconsidering.

Determining the volume of a specified file in PowerShell

NTFS makes it possible to mount one volume in an empty directory of another. Files on the mounted volume will be accessible through paths that appear to be on the host volume. One might wonder, then, how the true volume can be identified given a full path. I wrote a script in this Super User answer that does exactly that. It works by listing the mount points via WMI and finding the one that accounts for the largest chunk of the file path. Then it just issues a Get-Volume on the determined volume so that a volume object is returned.

Thursday, October 19, 2017

Do your batch scripts expect directory changes across volumes?

I have a batch script that copies some files around, invokes a PowerShell script to transform them, and deletes extra copies of the originals. The script resides in the same folder as the source files, executing a cd to an output folder before running the script that will change them. That way, the alteration and eventual deletion will apply to the copied files and not the only originals. I keep the source folder in my Dropbox, which on the machine where I wrote the script is on the C drive, but on this computer is on D, separate from the rest of my profile. When the script tried to cd to the output folder (under my profile, on C), it didn't actually change the current drive or the current drive's current directory because the /d flag was not specified. It then went on to transform and delete the originals. Fortunately, since it was in Dropbox, restoring them was easy, but I would still have preferred that it didn't happen. In the future, I'll be careful that my batch scripts are prepared for cross-volume operations.

Wednesday, October 18, 2017

PowerShell script properties cannot be parameterized

PowerShell classes don't support properties with getter/setter methods, but Add-Member can add script properties with script-bodied getter and setter methods. Unfortunately, there isn't a way for these properties to take parameters like properties can in VB.NET, nor can the custom object be indexed like in C#. I did some poking around in the PowerShell source and determined that these downfalls cannot be worked around - arguments are never passed to the getter and setter, other than the current object (scriptThis) and the new value to the setter, of course.

The only workaround that I know of is to use Add-Type to generate a class from C# or VB.NET code. PowerShell script can then index the object with the normal bracket syntax, or use indexed properties with parentheses, almost like calling a method.

Monday, October 16, 2017

Keen Modding Live - /nowait

Today while looking for a tedlevel equivalent for Keen Dreams (which I did not find), I learned about the nowait switch for Galaxy episodes. It causes the game to skip the B800 text screen at startup and therefore load a little quicker. It certainly did make a little difference when I tried it locally, but on the web there's still a little B800 interlude, probably since JavaScript emulation of machine code isn't super fast and therefore needs some time to get things set up. Nevertheless, every smoothing out of the experience helps.

I still haven't managed to devise a way to make Dreams begin a level immediately, so I posted another patch request on the forum. Once I have that, getting Dreams live should be fairly easy.

Sunday, October 15, 2017

Keen Modding Live - Almost all episodes live

Today I finished live-ifying and testing Keen 6. It's now a usable episode, so that completes the Vorticons and Galaxy series. To put on the finishing touch, I added a patch to all Galaxy episodes that removes the "if" ammo placement, since it was erroneously always added after one failed attempt at the level due to a side effect of some other patches. All three Galaxy episodes' packages have been reuploaded. This shows the benefit of the two-package system: I can fix problems with the episode package without needing to fiddle with every single uploaded level's package.

To get started on Keen Dreams, I wrote the server-side code for handling the Dreams (three-file) genre. It was a fairly easy adaptation of the Galaxy file-combining code. Now I need to figure out all the necessary patches for Dreams, which might be a bit tough since I don't know if it has a tedlevel mechanism.

Saturday, October 14, 2017

Keen Modding Live - More live episodes

Levellass very helpfully provided all the Vorticons patches I was still missing for Keen Modding Live, so today I finished off Keen 2 by editing the map Keen sprite and world map level to be completely black while the real level is loading. Armed with that experience and all necessary patches, live-ifying Keen 3 was a breeze. All Vorticons games are now permissible episodes for upload and play. Currently the only way to install new episodes into the system is to upload some files directly into the server and then manually tweak the database to register them, but it's not terribly difficult and I won't have to do it very often. There was a tiny wrinkle in the database management - the "tell me about all your episodes" API endpoint returns the episodes sorted by their ID in the database, and the JavaScript in the upload page just displays the episodes in the order it gets them. Since I added Keen 4 out of order, that presents the episodes in a bizarre order. So I added a sprinkle of JavaScript to sort them by DisplayName.

With Vorticons handled, I moved on to the remaining Galaxy episodes. Using my previously fashioned live version of Keen 4 as a guide, I browsed for the necessary patches from KeenWiki. I had to make my own tedlevel difficulty setter once again, but my Keen 4 version of that made me perfectly comfortable figuring it out with IDA. With that, Keen 5 is live.

Keen Modding Live - Patch finagling

Since Keen Modding Live is about individual levels rather than whole games, I want to remove as much of the inter-level bookkeeping as possible. Ammo, score, and pogo should all reset on death. All the necessary patches existed for Keen 1 on the KeenWiki, but not for Keen 2 and 3. I tried to translate some more of them, but was unsuccessful. Fortunately, Levellass came to the rescue and made the "don't keep score gained in level when dying" mechanic work for all episodes. I now have a complete patch set for Keen 2, so all I need to do to get that live is make the world map completely black. For Keen 1, I accomplished that simply by painting the entire map level with black tiles and graphics-editing map Keen's original sprite to also be black - that way, nothing else shows up while the real level is being loaded.

Thursday, October 12, 2017

Keen Modding Live - Galaxy level handling

Keen Vorticons and Galaxy are very different, so much so that even though Keen Modding Live mostly just has to keep track of files, it needs to treat Vorticons and Galaxy levels differently. In the episodes table, there's a column for "genre," currently set to either a V or a G.

When uploading a new level, the client specifies the episode ID, and the genre determines which files are expected. For Vorticons, that's easy - it only takes one. For Galaxy, it needs both GameMaps and MapHead files, but the player infrastructure expects a single level-specific package. Therefore, when a Galaxy level is uploaded, the system checks that the MapHead is the right length, concatenates the GameMaps onto the end of the uploaded MapHead file, uses that as the package, and generates a JSON manifest explaining to Emscripten DOSBox where the GameMaps begins. Dreams handing isn't currently implemented, but it'll basically be just the same plus a MapDict.

It would be convenient if the system accepted Abiathar single-level (ASLEV) files; then people wouldn't have to worry about keeping the level in slot 1 in the GameMaps. That would require quite a bit of work, though, since ASLEVs are LZW-compressed and that takes a lot of bit twiddling, which doesn't look particularly fun to do in PHP. Perhaps a desktop client (.NET, which can then load FleexCore2) could implement that client-side.

Tuesday, October 10, 2017

Introducing Keen Modding Live!

Modern Keen mods are getting more and more complex, so releases are infrequent. I think it would be really neat if people felt free to release small creations, especially single levels. It would also be good if players didn't have to fiddle around with extraction and DOSBox setup - optimally, they would just find the level in a repository and click to play. Such a system could also open doors for collaboration by keeping track of a chain of inspiration/editing.

For a while, I've been doing a little bit of poking around to make sure that these ideas are feasible. Though I'm pretty busy now, I think it'll be possible to build it, albeit slowly and at low priority. The site will be called Keen Modding Live. Currently it's possible to register and log in with Keen Modding forum credentials, upload Keen 1 or Keen 4 levels, and play uploaded levels. To allow for level editors to facilitate easy upload, I'm building a web API. To make sure the API is actually useful, and to avoid duplicating effort, my plan is for the web site to use client-side script to hit those same API endpoints.

Monday, October 9, 2017

Fixing an almost-working patch

Last time, I attempted to translate the "don't check whether Keen actually has the pogo before pogoing" patch from Keen 1 to Keen 2. It almost worked - Keen could always pogo starting from the ground, but not the air. My start was this:

%patch $6958 $EB $05
%patch $6BD0 $EB $0C

First I needed to find which of the two lines affected which check. When I commented out the first, ground-based pogoing stopped working, so that one was fine; the second line was the problem. Since I can't read most machine code, I fired up IDA and jumped to 0x6BD0. Bizarrely, I found that it wasn't actually near a comparison with the pogo variable - it was right on the stack-pointer-incrementing tail of some other function. Down a little bit was a pogo check (which I identified by having the variable named in IDA based on the correct half of the patch).

Having found the actual jump instruction at $6BF3, I tried patching the jump condition to be unconditional ($EB). That did something in that the pogo never worked in the air, even after acquiring it with C+T+Space. I don't know if there's a jump condition that never jumps, but since it was a check for equality ($74), I replaced the previous byte - the number the variable is compared to - with something ($FE) that the pogo variable never gets set to. The fixed second line is this:

%patch $6BF2 $FE

So, why didn't the patch at $6BD0 work, even though I found the right sequence of bytes? Turns out I had transcribed the location from XVI32's left column, which shows the address at the start of the line, not the address of the selected byte, for which I should have looked at the bottom left in the status bar. Oops - turns out patching right before sleeping might not be the greatest idea.

Sunday, October 8, 2017

Translating a patch from one episode to a similar one

I recently wanted to patch Keen 2 to always allow Keen to pogo, even if the "have pogo" flag wasn't set. (In the original Keen 2, Keen starts with the pogo, but a side effect of another patch removed that.) Conveniently, that patch already existed, but only for Keen 1 and 3. Since it looked pretty simple, I decided to translate it myself.

I started by loading a Keen 1 dump into XVI32 and jumping to one of the two relevant addresses. Starting at $3B16, the original bytes are $83 $3E $9A $AA. Searching for that string in a Keen 2 dump unfortunately turns up nothing. However, since I knew that the code has something to do with the pogo variable, I consulted the game stats table. $9A $AA refers to the data location $AA9A. The analogous location in Keen 2 is $9AB0. Searching for $83 $3E $B0 $9A finds two locations in Keen 2: $6958 and $6BD0.

So I replaced the Keen 1 patch locations with those and ran it and it appeared to work, but it didn't quite feel right. Careful observation showed that the pogo only worked if Keen wasn't in the air, but it should work either way. We'll pick it up next time to see what went wrong.

You can't elevate an existing PowerShell prompt, but you can relaunch PowerShell in the current directory

One user wanted to know how to elevate an existing PowerShell prompt to administrator. Unfortunately, this is not possible because the primary token of a process cannot be replaced after it starts running. It is, however, very simple to prompt for the elevation of a new PowerShell window, and the current directory can be sent over via a startup command.

Start-Process powershell -Verb runas -ArgumentList "-NoExit -c cd '$pwd'"

Note that the $pwd (an object which stringifies to the current directory path) is expanded in the original process. Usually, starting PowerShell with startup commands would cause it to exit after finishing those commands, but -NoExit leaves it around for interactive use.

Friday, October 6, 2017

Securing the PHPSESSID cookie

One of my PHP web applications uses PHP's session functionality to remember the current user. I noticed, though, that the PHPSESSID cookie that's generated for me isn't marked Secure or HttpOnly even though I'm exclusively serving the site over HTTPS and I never need JavaScript to do anything with the cookie. Therefore, as a defense-in-depth measure, I looked into setting those flags on it. Using php_flag in .htaccess didn't work; it caused a 500 error noting in the server log that the command was unknown. The ini_set approach didn't seem to have an effect on my server. Using set_session_cookie_params did the job, though. I'm including this line before session_start:

session_set_cookie_params($cookieTime, '/', $myDomain, true, true);

Acquiring Let's Encrypt certificates on Windows

For a while, I've been getting my Let's Encrypt certificates (in manual mode) in an Ubuntu VM or in the Bash on Ubuntu on Windows environment. Some recent update, however, seems to have broken the BoUoW approach, and since I didn't want to hassle with a VM, I looked for an alternative method. I remembered that someone mentioned ACMESharp, a PowerShell client for Let's Encrypt. It comes with a very helpful quick start guide. I used the HTTP challenge with the manual handler. The only issue I encountered was that Update-ACMEIdentifier with a specified -ChallengeType always returned "pending" even after waiting a while, but supplying only the challenge alias worked as expected, returning a "valid" status very quickly. I was then able to generate a certificate and get it signed.

Tuesday, October 3, 2017

Emscripten DOSBox has to wait for the filesystem to load

While implementing a difficulty selector for Keen Galaxy levels in Emscripten DOSBox, I needed to inject a file into the virtual filesystem at runtime - not from a downloaded, manifested package. At first, I tried calling FS_createDataFile right before using the pre-supplied loadPackage function, but something necessary was missing, because the file wasn't added and an error was logged to the console. Inspecting the loadPackage function carefully, I found that it checks calledRun on the DOSBox module and, if that's not set, queues the operation with Module["preRun"].push (initializing Module["preRun"] to an empty array if it's absent).

I refactored that pre-run queuing out into its own function, altered the loadPackage function to use that, and called the new function with the data file creation as the callback.

Sunday, October 1, 2017

When wireless mice start scrolling very slowly

Recently, my wireless mouse decided to switch to an extremely slow scroll speed. Each roll would go about a tenth as far as it should, which made reading long pages rather inconvenient. Turning the mouse off and back on didn't do anything. Unplugging and replugging the USB dongle, however, did. The problem immediately went away and scroll speed returned to normal.

Setting Keen Galaxy difficulty with Emscripten DOSBox

Today I set up Keen 4 in the Emscriptenified version of DOSBox with the tedlevel parameter supplied so that the game goes straight into the specified level. Unfortunately, using that mode always sets the difficulty to Normal; attempts to change it by starting a new game in the running program just deadlock it. I first figured out how to patch the tedlevel difficulty: it's a word at $3BBE from 1 (easy) to 3 (hard). The patch script uses %patchfile to load a given file at that address. That file, one byte long, is injected into the DOSBox filesystem at page load using FS_createDataFile with the data appropriate for the user's selection.