Robolectric doesn't like spaces in the user profile path

Today I helped someone who was trying to run Robolectric tests but found that they crashed with a FileNotFoundException for a JAR somewhere under their user profile directory. This user's profile folder had a space in the name that the error message displayed as %20. Apparently the path was getting URL-encoded somewhere along the way. We worked around the problem by creating a symbolic link (mklink /d) in C:\Users from the mangled name to the actual user profile folder.

When JUnit tasks get stuck in "No tests were found"

Tonight I needed to troubleshoot some Android Studio JUnit run configurations failing with "no tests were found." Interestingly, they would work fine until the user tried to run them in the presence of a compile error, at which point they would stop working until a Build | Rebuild Project, even if the compile error was fixed. It was fixed by adding a before-launch task to the run configuration that runs the clean Gradle task for the module. Later it was found that a plugin was interfering with the build process.

Exotherm - Connection maintenance is working

When I hibernated my computer last night, I had Exotherm running and connected. When I started back up this morning, I found that it quickly reconnected to the server without my intervention. It was also able last night to remain online for a long time without getting confused about its connection state. So it looks like the connection keep-alive code is working.

Exotherm - Maintaining connection

I previously found that Exotherm tended to lose its connection to the FICS after a while of idling. That wasn't even due to the automatic one-hour idle log-out; the bot and the FICS just disagreed about whether it was online. Today I rearranged its FICS interface some more so that it will try to reconnect if it finds that the connection was lost. Also, after 5 minutes of no input or output, it will send a chat to itself through the server just to keep the connection alive. These changes should make it suitable for deployment as a long-running process on my server.

Volley and Robolectric

Yesterday I set up unit testing for an Android project using the Robolectric test framework. The app depends on Volley, which works fine in Android. But trying to run the unit test in the JVM produced a crash stating that the org.apache.http.client.HttpClient class could not be found. I tried adding empty classes with the appropriate names to sidestep that error and similar subsequent ones, but couldn't get the test to start. Eventually I found that Robolectric provides a shadow implementation of the necessary classes, but it's in another dependency. Adding that to the Gradle project file fixed the problem:

testImplementation 'org.robolectric:shadows-httpclient:4.1'

When ClassCastException is thrown from Mockito-mocked functions

Today I needed to mock some methods on a Java static class for unit testing. I used PowerMockito because, as far as I can tell, Mockito cannot mock static classes, but the two tools have very similar functionality. I used when, then , and thenCallRealMethod to run other code and then return the normal result, but found that a ClassCastException was thrown in the mocked method. The stack trace didn't provide a line number for that frame, and the argument types were correct in the calling code, so I was confused.

I had accidentally used lambda syntax to return something from the then handler that was not of the method's return type. When PowerMockito tried to use that as the return value, the exception was generated. Apparently chaining thenCallRealMethod after then is ignored.

Some build.gradle features aren't available for Android modules

Today I tried to merge a Java project into an Android module so that a Gradle plugin could act on the entire codebase at once. I found that many features are provided by the Java plugin for Gradle (apply plugin: 'java') but not the Android plugin (apply plugin: 'com.android.application'). Unfortunately, those two plugins cannot be applied at the same time. It looks like I'll need to rewrite the custom plugin to work with the Android project.

Exotherm - Running on a server

When I'm ready to deploy Exotherm as an official computer on the FICS, I'll need a place to run it  continuously. I already had some Namecheap shared hosting, but when I tried to run my script there, it failed to connect because port 5000 was being blocked. Today I got in touch with support and asked for port 5000 to be opened. To my pleasant surprise, they did so right away.I successfully ran Exotherm from there for a bit, so it looks like this will work.

Exotherm - Minor rearrangement

Throughout its development, the Exotherm entry point (FICS interface code) acquired many global variables that had to be listed in every function that used them. That would be a mess if I kept adding more features, so today I rearranged those into one class. Now the functions only need to declare their use of one variable.

Exotherm - Avoiding stalemate

Previously Exotherm's evaluation function knew nothing about stalemate. I once tried to add that consideration by checking the size of the list of legal moves, but generating that list is very expensive. I've never seen Exotherm stalemate its opponent, so that definitely wasn't worth it. But just to be safe, today I added stalemate detection in a different spot. Now it will retroactively score a position as zero (a draw) if it finds no legal moves when trying to deepen the tree.

Exotherm - Playing itself

Recently I wondered about the possibility of having the Exotherm engine play itself. Since it's designed as a FICS bot, the easiest (though perhaps not most efficient) way is to make a copy of the scripts, run both to connect to the FICS, and have one issue the command to play the other. That worked as expected! I watched several games. Sometimes the game ended with an opening trap; sometimes it went through the endgame. There didn't seem to be a large advantage for either color, but the engine isn't super strong yet.

In the future, I could use bot vs. bot games to determine which version of the code is better: make a change and have the modified script play the original version. If I'm going to do that a lot, though, I should probably make a local test harness instead of relying on the FICS.

The Quick Access Toolbar can refer to macros in a specific document

A long time ago I wrote some macros in an Excel workbook. A user put shortcuts to them in the Quick Access Toolbar. When the workbook was copied, we found that pressing the buttons made Excel open the old copy of the workbook - apparently it was trying to run the macro from the original document. This was confusing because the QAT customization screen referred to the commands as ThisWorkbook.MacroName and even showed the path to the new workbook on mouseover. But deleting the macro entries from the QAT and re-adding them from the new workbook made them work with the new workbook.

Exotherm - Configurability

I realized last night that my work on Exotherm yesterday might have weakened it overall. Even though it could search deeper/faster, its time management strategy led to it cutting off one layer of nodes partway through. Some second-level nodes had all (third-level) responses considered and therefore had their evaluations adjusted, but not all, adding a huge element of randomness. So today I added a "maximum depth" parameter, at which the move determiner will stop expanding the tree. Under PyPy it now moves in one second or less with a depth of 2, which produces fairly strong moves.

Until today it would always tell its opponent some internal information about the evaluation function each move. Obviously that would be bothersome and possibly too revealing in a rated match, so today I added a per-opponent "talk level." It defaults to "normal," at which it only greets and congratulates the opponent. At "silent" the bot doesn't speak at all; at "verbose" (only available for unrated games, otherwise acts like "normal") it provides its internal commentary. Telling the bot "silent", "normal", or "verbose" will make it update its per-user talk level and acknowledge the change.

Exotherm - Time management

After yesterday's intelligence improvements, Exotherm took quite a bit of time to move, regardless of the time control. Today I worked on improving its time management. The main move determination function now takes three time-related parameters: minimum time, minimum depth, and maximum time. It explores the move tree for at least the minimum time unless it runs out of moves to explore. It then continues until it reaches the minimum depth in the tree or runs out of the maximum time. The FICS interface script currently uses 1/40 the remaining time as the minimum, 1/12 as the maximum, and 2 as the minimum depth. This worked as intended - the bot doesn't usually lose on time even in fast games, but of course its play deteriorates when it gets under time pressure.

There are two main factors that determine the bot's strength: move evaluation time and which positions get evaluated. Until today it wasn't very good about reducing the latter: it would bail out if it had win-in-one but otherwise it would search all paths equally. Today I added a notion of "finished" nodes. When a child node is a win-in-one, it sets its parent as finished - when you can win immediately, there's no point in doing anything else. When the main loop encounters a node with a finished parent, it skips growing the tree from that node.

Since I'm new to Python, I don't know how to optimize processing speed. A suggestion I discovered was to run the script under PyPy, an alternative JIT-compiled Python interpreter. So I installed that, ran it, and found an order-of-magnitude increase in evaluations! Exotherm can now evaluate about 17,000 positions per second. Using skipping, it can sometimes even get to a depth of 5 in reasonable time.

Exotherm - It plays decently!

Yesterday I got my atomic chess bot to execute random legal moves. Today I made it choose moves with some intelligence. First I wrote an evaluation function that returns a number describing which player is favored in the position and by how much. My first version assigned very large numbers for king destruction or checkmate, or the material balance otherwise. Then I wrote a function tasked with selecting the best move in the current position. It does a breadth-first search on the tree of possible moves, bubbling the best variation up the tree as new nodes are evaluated.

That kind of worked in that it would blow up the opponent's king when possible, but it was extremely slow and vulnerable to common traps like 1. Nf3. Using a profiler, I found that a lot of time was spend just copying the board, so I switched to a list comprehension instead of deepcopy. That improved the nodes-per-time markedly, but the overall evaluations were still poor. The bug was in the bubbling up: once a child node's value was lowered (due to finding a better response), its parent's value was not. So the bot was effectively playing hope chess.

Fixing that made the evaluation numbers more reasonable, but the bot still fell for the opening traps because it couldn't see far enough ahead in time. Even with 20 seconds per move it only evaluated 3,000 positions. Looking at the profiler again, I saw that generating the list of legal moves is dramatically more expensive than evaluating a move. My search was deferring evaluation of newly discovered responses until it could enumerate their counter-responses, but that lets many possibilities go to waste. So I rearranged the logic so that responses are evaluated as soon as they're generated. That improved nodes per time by an order of magnitude!

But even so, obvious knight traps still tended to work on the bot, and it often made pointless moves (like moving an undeveloped rook back and forth). I made the evaluation function more sophisticated by slightly increasing the score of pawns as they moved toward promotion, which I hoped would also encourage the bot to push 1. ...f6 to stop the knight trap. Then, more interestingly, I increased the score of pieces that are attacking opposing pieces by a tenth of a pawn per attacked piece. That finally made the bot worry enough about the knight to dodge the trap. Around this point, it was able to defeat a couple humans! I further refined the evaluation function by counting king-explosion-threatening pieces as an extra third of a pawn, which encourages the bot to make threats.

Along the way, I added a couple features to the bot's FICS interface. It greets its opponent when the game starts, stating that it's an atomic game and explaining how to cancel the game if desired. (Other variant bots do this too.) At the end of the game, it congratulates its opponent or says "thanks for playing!" depending on the game's result. If the opponent doesn't make their first move within one minute, the bot cancels the game (to avoid accidental denial of service). Currently, just for testing, the bot states its depth, positions considered, and overall evaluation at each move.

Its main downfall now is time management. It's completely unaware of the clock, so it usually forfeits on time in short games. I'll work on that tomorrow.

Exotherm - It plays!

Yesterday I got Exotherm to find all the places a piece could conceivably move. Today I used that to generate a list of all hypothetical moves, and then that to generate the list of legal moves. Castling is a bit of a special case, so castles are only inserted into the move list in the last stage. Legality in general is somewhat tough because of atomic's check rules, e.g. being next to the other king makes it impossible to be in check.

After writing the move list generator but before handling legality, I made the FICS bot script select and play a random move when it's to move in a game, resigning if the server says the move was illegal. With the obvious caveats, that worked! After implementing legality testing, resignations almost went away. There was a bug in the hypothetical move executor such that it thought any king move to the C or G files was castling, so it mangled the board state whenever that happened. Fixing that has, as far as I can tell, made its legality tester perfect.

Exotherm - Chess logic begins

Yesterday I got my bot to connect to the FICS. Today I started writing its chess logic. It can now parse a style12 line into an object representing the game state. That class has a method that executes a given move (without testing legality) and returns the new game state. That'll be necessary for thinking ahead and testing hypothetical moves. Considering en passant, castling, and promotion was pretty finicky, but I think this part works. I also wrote a function to generate the hypothetical move targets for a given piece, again not fully testing legality yet.

Exotherm - Keeping a connection

Yesterday I used the interactive Python prompt to establish a connection to the Free Internet Chess Server (FICS). When I put all the necessary code into one script today, it didn't work so well; there's a bit of delay before responses from the server are available. I tried just sleeping for a brief period, but that still didn't work, so I ended up using expect on the telnet object in the login code.

Once past the login, I used read_until to go line-by-line. FICS is designed to be usable in a plain terminal, so it does its own linewrapping. Alas, there's no way to say "give me lines of unbounded length" (240 characters is the maximum width) so I had to write some finicky logic to reassemble linebroken notifications into logical lines.

With that all done, Exotherm can log in and listen for match requests. Currently it accepts all challenges that match its FICS formula (atomic games of 10 minutes or less) but then just sits there. Now I need to actually start writing some chess logic.

Exotherm - Time to learn Python

Two unrelated things happened recently:

1. My lab supervisor suggested that I learn Python
2. I started wanting to write a chess engine for the FICS

So the obvious thing to do is write the engine in Python.

I know there are tons of very strong computers on the FICS already, but as far as I can tell there are no active computers that play the "atomic" variant. In that chess variant, captures trigger an explosion that takes out all surrounding pieces (except pawns) and the capturing piece as well. Therefore, the amount of material on the board tends to go down fast, which I think should make computer analysis easier. Inspired by the "atomic" term, I'm going to name the engine Exotherm.

I've read a bit about Python before, but haven't written anything in it yet. So the first thing to do was get a Python 3 environment on my server so I have a place to run the bot. That appears to be working. Now I need to make a program that connects to the FICS. I successfully did that in the interactive prompt using the built-in, convenient telnet library.

Markeen maps update 1/3

I've mostly finished the foreground of the eighth Markeen-inspired map. I've fixed all the glaring tiling errors (except pipes), but I still don't know the level flow. I took out one clearly unhelpful gate, but there's the beginnings of one in the leftmost red room that I'm not sure what to do with.

Markeen maps update 1/2

Today I started chipping away at the eighth Markeen-inspired map. Some parts of the level were very good in terms of tiling, but there were some messes too. This is the first level in which Markeen used switchable bridges, two in this case. Unfortunately, the switch it provided is pretty much right next to the end of the level, so I'd have to change the level structure a bit to make it useful. Because of that, I'm not yet sure of the flow of play, so I just focused on correcting the tiling, which is almost done now.

GetAccessControl won't get Registry key audit entries by default

For a recent Super User answer, I needed to manipulate Registry audit entries (SACLs) with PowerShell. After enabling SeSecurityPrivilege I was able to set audit rules just fine, which I checked by using the Registry Editor's permissions editor. But when I tried to use GetAuditRules on the result of GetAccessControl to check my work inside PowerShell, I saw that it always returned an empty list. This had me baffled until I looked at the documentation for GetAccessControl on Registry keys and saw that the zero-parameters overload doesn't request the SACL:

This method overload is equivalent to calling the GetAccessControl(AccessControlSections) method overload with the bitwise combination of the following flags: AccessControlSections.Access, AccessControlSections.Owner, and AccessControlSections.Group.