Markeen level project progress 5/30

Today I finished decorating the ninth level in my Markeen-inspired level pack. While playtesting again I noticed that it was not that hard on the "hard" difficulty setting, so I added a couple creatures. I then selected a machine-generated starting point for the tenth level. I'm starting to run out of good material from the original batch of 100 levels: there were none with the last of the four fuse machine colors, so I had to swap out a machine and give the surrounding area a makeover. I've started cleaning up the platform tiling. So far I haven't been able to come up with a route for the player to take through the level, though I do have a theme planned.

More machine-assisted level design progress

A couple days ago I had started the "fix the glaring tiling errors" phase for the ninth level of my Markeen-assisted level pack. Markeen had produced a neat centerpiece for the level, but I wasn't sure about the route I was going to have the player take through it. Today I punched a small number of holes in the generated structure to create a path for the player that makes me think of convection. The player goes down one side and cannot return up that way, so they must instead return via a door whose path they should clear earlier. I got on a roll and made the level playable, fixed the distribution of point items, and did the background. All that's left to do for this one is decorate.

FMod - Grid size inconsistency fix

While using Abiathar to make some of my own levels today, I noticed a small bug. I enabled the grid to see tile boundaries in the simultaneous tileset, but the grid was too fine: 16x16, even though I had zoomed the tileset to a tile size of 32x32. The grid size was appropriate for the level viewer, though. Apparently I missed a spot when giving the simultaneous tileset its own zoom level. This is fixed now.

Gradle scripts for publishing to Maven Central shouldn't sign specific files

About a year ago, I deployed a Gradle project to Maven Central, which was made difficult by the signing requirement. I found a third-party guide that provided a very long snippet of buildscript to explicitly sign the POM (project manifest XML file) and various other files. After updating the project to Gradle 6, that didn't work anymore; the files that got signed were not the ones actually uploaded, so the signatures were invalid. Gradle 6 also introduced a new file that wasn't getting signed properly. After some poking around I discovered that signing specific files is not necessary when using the maven-publish plugin. The signing plugin can sign an entire publication as long as its configuration is written after the part of the buildscript that declares the publication:

signing {

    sign publishing.publications.getByName("mavenJava")

}

That change also made my check for duplicate signed files unnecessary.

A full working version can be found on GitHub.

Machine-assisted level design progress

Intermittently over the last few years (!) I've been working on a Keen 5 level pack based on levels generated automatically by Markeen. I have a lot of time over the summer, so I have a chance to make some progress. Previously I had fixed all the major tiling errors in the eighth level and made it largely playable, but I wasn't happy with the flow. It's OK to have some optional areas, but it was very difficult to design a route for the player, given the constraints, that would visit even a third of the level. Looking at it fresh today I thought of a new route that I like a lot. It makes the level less sensical as a physical place, but I think a theme of "space station under construction" salvages it. The foreground is done and I just need to finish decorating.

Facebook's /me/feed endpoint can't create posts with just one image

I recently wrote a script to migrate posts from a Facebook download-your-information archive to a new Page, including all attached photos. Since many of the posts had multiple images, I had the script upload each as photo to the /me/photos endpoint unpublished, then attach all of them to one new post created with /me/feed. I then made another request to edit the post to backdate it. This worked perfectly for posts with multiple images, but for posts with just one image, the backdating request failed with "you cannot backdate unpublished posts." Apparently single-photo posts are a different kind of thing than posts with multiple photo attachments. For the former, I had to make a normal (published) /me/photos request and get the post_id from that for backdating.

Facebook's download-your-information JSON encodes non-ASCII as UTF-8

Today I wrote a script to reupload Facebook posts from the JSON files produced by their download-your-information tool. Getting the post text itself was straightforward except for non-ASCII characters. JSON escape sequences for control characters specify 16 bits, but Facebook's never used more than a byte. Apparently they store their strings as UTF-8 and encode each byte to JSON. The .NET Framework (used by my script) represents strings as UTF-16, so Facebook's text was mangled. To get the correct text, I had to take the string from the JSON, map each character to a byte, and decode the byte array as UTF-8 to produce a UTF-16 string in memory.

FMod - Song format flexibility

Today I rearchitected the GalaxyAudio class in FleexCore2 (which represents a Keen Galaxy audio archive) to lazily parse the different kinds of sounds only when requested. If a parsed sound is requested, the stream of bytes is parsed and the sound is stored in a cache. Other components can request raw chunks, which serialize the sounds back to bytes if they've been parsed or just return the original byte stream if not. Abiathar now avoids triggering the parsing of IMF songs unless needed to turn a headerless IMF into a chunk. This allows preserving audio entries of arbitrary format in IMF chunks.

Since FleexCore2 changed, the ImfPreview extension had to be updated as well. While I was working on that I also made ImfPreview use Abiathar's standard "one moment" dialog instead of its own copy.

FMod - Song name migration

An Abiathar user asked for the ability to import a different format of song into the audio archive. I think the best way to implement this would be to allow importing any file directly into the audio archive without having Abiathar try to parse it. Unfortunately FleexCore2 currently isn't designed to load files without parsing them fully. Abiathar also assumes that IMF songs in the audio archive can have a "tag block" after the song data and takes advantage of that area to store the user-provided song name. Not every format will have space for a tag, so today I changed the Music Mappings dialog to store the name in the ADEPS (Abiathar project) file instead. 

That would be a simple change if not for the backwards compatibility concern. Users may already have set song names that existing Abiathar versions then wrote into the audio archive. It would be bad to lose those when updating. I wrote some migration code to detect when the project's audio files haven't been saved since an old Abiathar version and, if so, load them using the old architecture just to copy the names from the tags to the project file. With this working, I can start on a new architecture to allow greater format flexbility.

AdviceAdapter's exit advice may trigger multiple times

I recently tried to use AdviceAdapter from ASM Commons to insert a bit of code that would run before a method exited. It was important that the new code run exactly once no matter what happened inside the original function. Unfortunately the adapter seems to call onMethodExit right before every return or throw instruction. Advising on returns is fine, not so much on throws because the exception could be caught inside the method. Checking for containment in a try block would not help without some serious rewriting because a called function could be capable of throwing multiple types of exceptions, only one of which might be caught by the function being instrumented. So in AdviceAdapter's defense, its job is essentially impossible. I ended up renaming the original method and generating a "bridge" method with a simple try-finally around the call to the original.

Java monitorenter instructions cannot be interrupted

A Java application I'm helping with sometimes needs to terminate untrusted threads that got out of hand. Our sandbox seemed to be working great until I tried running two tasks that tried to synchronize on the same object and spin indefinitely once the lock was acquired. Trying to terminate the locked-out thread failed until the thread that acquired the lock was stopped. Apparently the monitorexit JVM instruction used to enter a synchronized block cannot be interrupted by stopping the thread, much less with the normal interrupt mechanism. We were already doing some bytecode editing for our sandbox, so I extended that to rewrite uses of synchronization-related instructions or methods to call static sandbox functions that simulate the originals using ReentrantLock, locking functions of which can be halted by thread shutdown.

Infinite try-catch loops can crash the JVM

Previously I discovered that the Java compiler sometimes emits exception table entries set to "handle" an exception by jumping back to the start of the covered region. This led to deadlock if an exception was indeed thrown in that region, usually due to bytecode rewriting inserting instructions. It was manageable to fix during bytecode rewriting with ASM by dropping exception table entries (not calling super.visitTryCatchBlock) that had the same start and handler label. Unfortunately there are other circumstances, particularly with while loops inside synchronized blocks, in which the compiler emits an entry with the handler in the middle of the covered region. I saw this crash the JVM entirely, citing an "invalid class file" but providing no additional information with -Xverify:all. Based on the access-violation error code, it seemed to be a stack overflow in the JVM itself. Avoiding this problem is more difficult because ASM doesn't provide a way to know the order of labels until all the exception table entries have been visited. I had to make the ClassReader first accept a visitor that recorded the indexes of problematic try-catch blocks, then use that information during the real transformation that dropped the offending entries.

When the Android emulator says "not enough space to create userdata partition"

Today I helped a student who was trying to set up the Android emulator but received the message "emulator: ERROR: not enough space to create userdata partition." At first they suspected the specified virtual storage device was too small and tried increasing it, but that just made the "required" number in the error message bigger. This message is actually saying that there is not enough free space on the actual hard drive to reserve for the virtual storage device. Freeing up some disk space allowed creating the emulator.