Robotics - Springing a Surprise

Tonight featured an extra-long meeting of the robotics team. That would have been amazing if we had more than three people there for the majority of the time. Despite our small numbers, we were able to do some poking around with the robot. For the first hour or so, it was just me and an engineering notebook writer. We were unable to replicate the spring insertion that was done the meeting before, so we tried all kinds of alternative means to make springs stop the bar from falling forward. That failed, and we spent a little over a half hour helping the other team repair their robot after a botched rewiring. When some more of our people arrived, we did even more messing around with the springs and came up with absolutely nothing. Near the end, when my father (who has an interest in engineering) arrived, he showed us an excellent way of replacing the torsion springs with expansion strings - or rubber bands.

FMod - Extensible Tools

I finally got around to wiring up the state wrapper for Fudge, allowing tools to be easily made. The first thing I did with this awesome new framework was move the default functionality into a real tool: the Tile Tweaker (formerly called Tile Placer/Picker). The great thing about using my own extension API to do the core tools is that it forces me to make that API powerful enough for even the original program creator to use.

In addition to re-implementing the most basic of level editor functions in a very complicated and roundabout way, I also created a linker tool, a la TOM. It's surprisingly intricate for such a simple thing; you can copy the infoplane link for that tile or create a bidirectional link just by using the right mouse button at various times.

I also (again) changed the infoplane font, this time to a small thin thing.

Robotics - Springs not Strings

The John Deere engineer that sometimes comes in and helps us recommended that we try to use springs to stop the hanger bar from falling forward while we're driving. The capstan motor would easily be able to overcome the springs, so they wouldn't cause a problem for the pull-up. Since we didn't have good springs on hand, we extracted a few from some new mousetraps that the supervisor found. A few people, including me, messed around with putting the springs inside the turning point of the bar, but with no success. Only near the end of the meeting did we discover that the spring we were using is a stretchy spring, not a stereotypical springy spring. Next meeting, we will use pliers to add tension to the springs, which should fix all our problems!

FMod - Interface Infrastructure

Whew, interfaces and dynamic programming are complicated! However, they definitely paid off for Fudge.

With a few more tweaks to the important interfaces (and a new one for level-based plane renderers), Fudge's drawing routines were back up. I also took this great opportunity to abstract the level collection manipulation and give each level its own visual offset. Knowing that every plane rendered for the level viewer is in fact based on a level is a very helpful thing. Another great benefit of this new system is the ability to re-render only one tile at a time instead of redoing the entire thing. I took advantage of that in the links faux-plane to adjust its collection of lines only if a tile is changed that is a link.

That little text in the bottom right displays the current tool. That's right, the Tools menu actually does things now! It currently can only be populated by extensions, but I'll soon fill out a core "extension" to supply the basic tools.

FMod - Render Revamp

The last time I tried to do something simple by setting up an advanced extensible system, I failed miserably. So let's try it again!

I want Fudge extensions to have the ability to draw things on the level viewer, probably in a plane-based way. I also want the Fudge core to not take forever to redraw the level when the user moves it around or places a tile. The solution I imagined is a collection of cache-able graphic planes for each level. Instead of re-rendering all the tiles shifted over a little, I could just keep the same level composite and draw that shifted a little bit. I'm hoping that my strategy will also work with zooming.

So, I created a few classes and an interface to represent planes. The interface allows extensions to define any sort of rendering they'd like to do. Those classes represent some standard types of planes, like a level-based tile renderer.

I haven't been able to test it yet and I'm not sure what to do about the tile palette, but I think this is actually going to work.

Robotics - Diagonal Mount

I've said it before and I'll say it again: Saturday meetings are amazing for productivity. Today's, however, not so much. It's very difficult to be productive when the people responsible for most of the productivity are not present.

For the first two hours or so, the only people present on my team were me and an engineering notebook writer. We were able to do remarkably little beyond moving an out-of-place motor to an in-bound location and bending some L brackets to use as diagonal mounts (at the suggestion of the supervisor). The diagonal mount was meant to stay out of the way of the capstan string while still being inside the allowable dimensions. I successfully attached it, but it was not very stable and was too high.

When the actual build people arrived, we got the mount actually in place. Unfortunately, it was unable to spin the flag because the raiser was still too high. So, we moved it back over to a perpendicular mount that is probably in the way of the string. We're fairly certain it is now at the right height and sure it will fit into the corner, but we must again confront the string issue.

FMod - Zoom

To my surprise, I learned yesterday that The Omegamatic, an existing and very good level editor by Mink, already supports Keen Dreams. Now that I have a competitor, I need to do everything in my power to make Fudge better.

The first thing to do was match Omegamatic's features. Since I saw easy constants to replace with a variable, I went ahead and implemented zooming as a variation of the pixels per tile. It required a little bit of adjusting the GDI+ graphics settings, but Fudge can now do a pixelated zoom!

Another nice thing Omegamatic has is a "show links" mode, which interprets high infoplane values as links and draws lines representing them. However, it doesn't understand that level-entrance infoplane values are not links, drawing lines all over the place to points far outside the map. In my implementation, I checked for that before drawing the lines. Additionally, Fudge will only draw the hex infoplane value composite if there is no link.

FMod - Supporting the Galaxy

Today marks a massively important step in the development of Fudge!

Prompted by the super secret beta tester's suggestion, I went about generalizing the editor to allow Galaxy levels as well. Fortunately, I already had all the things in place from FMod, so I just needed an interface to provide the same, well, interface. At first, I had this grand idea of an abstract superclass handling the similar IO, but that shortly become a pain and I went back to a simple interface. Getting ready for Galaxy also required a system for managing the bazillion "default" files and displaying the right drop-down file name options in the Gather window.

The first problem I encountered with that was the EGA header pointing to a null chunk in the middle of the tiles, signifying an empty or all-black tile. That didn't stop FMod from trying to read a blank chunk and causing all kinds of exceptions. All that needed to be done was add a check to see if the chunk was nothing before trying to pull a tile out of it, and if not, use an empty one.

So yes, Fudge supports Galaxy now. Next up, I need to add a zoom function and make Keen 6 graphics not cause a crash (the EGA header produced by tedsetup has a few extra bytes that FMod thinks are another chunk).

Robotics - Strings Everywhere

Resolving issues is so much easier when people who actually know what they're doing are there! Ignoring the flag raiser for a little while, we went ahead and installed the second capstan shaft twirler thing in the back of the robot. I coded it, wired it, and made sure everything was working. Because there were quite a few distractions at this meeting - namely the engineering notebook people wanting to get a team picture when we already had a perfectly good one from two months ago - we didn't manage to get a whole lot done. Beyond the second string attached to the hanger bar, all we did after that was bend some scrap metal into a battery holder.

FMod - Super Secret Deployment

After waiting almost 36 hours for any kind of acknowledgement from the first person I gave a private beta copy of Fudge to, I selected someone else. This time, I went to a place essentially guaranteed to have responding people: IRC. Since it is a private beta, I launched a PM session with the Keen community's best programmer, who was fortunately online.

I'll admit now that I have never actually deployed an application heavy enough to use external resources beyond those in the System/Microsoft namespaces. Fudge makes use of two external projects, both by me: FMod (the backend) and Fudge.Interop (the extension API). Experienced people probably know where this is going, but I took the compiled Fudge.exe file out of the Release directory, put it in Dropbox, and gave my tester the link. It failed to launch. Twenty minutes and three recompiles later (including trying to embed the types, which resulted in "maximum number of errors has been exceeded"), I discovered that I really have to distribute the compiled versions of those projects, in this case as DLLs.

I gave him a ZIP containing all three files and he was able to launch it. Coincidentally, three is a common number for me on this project: three EGA graphics files, three map files, three tile planes, three state settings, and now three Fudge files. Apparently, I need to make it more obvious that, to load default levels from memory, one must check the "dump" box in the Dependency Conglomerator. It probably wouldn't hurt to have better error messages too. But once he got it working, he described it as "definitely the editor of the future."

He wants me to generalize it to support Galaxy as well, and of course have actual tools to manipulate more than one tile at once, but it seems I have done a good thing.

FMod - Cache ALL the Graphics

I should probably go back through and add a Fudge tag to all the FMod posts that are primarily about it.

The Fudge extension API, though not a critical feature by any stretch of the imagination, is a pretty fun project to work on. I added a dynamically generated menu system that will create "about" windows for each extension based on the FudgeExtensionMetadata object requested in loading and also provide the user with the option of unloading the extension. That menu will later contain extension-generated commands as well.

In an effort to decrease load times (long due to translation of EGA graphics to GDI+ bitmaps), I reworked the tile storage system to always use a cache. Tiles not used in any levels will never be translated because, well, they don't need to be rendered. However, all tiles will be translated when the user switches to the tile palette for the first time, which does now have a little bit of a wait. Still, doing this cleaned up the code and definitely made the initial loading less painful.

I'm now fairly certain the last of the GUI bugs have been squished. Changing any setting will ensure the menu check boxes and status bar reflect the new state and there will never be a way to access a command that requires a state that is not.

FMod - Double Buffer, So Intense

Fudge, the first GUI consumer of FMod, has all the critical functionality required for being a decent level editor! Not only did I fix the flickering issue, I added a few more level-related commands, generalized functionality, and squashed little GUI bugs!

Apparently, one is not supposed to dispose IDisposable objects that were created by the system. My reflection trick that I tried to use to enable double buffering would have worked if I had not disposed the graphics after painting them. It's too late to go back to that - I switched everything over to the dynamically generated subclassed control - but this is probably better programming practice and it works perfectly

Generalizing the level creation dialog, I created commands to edit level properties - name and dimensions - and copy levels. There were a few issues with tile sets not being re-rendered when changing the advanced settings, which I resolved by using my fancy GUI methods instead of just setting the field.

I've always wanted to create an API and this gives me the perfect opportunity to do so. Since I don't really want to be maintaining this for forever, I'd like everyone to be able to create extensions for Fudge and load them easily. To that end, I started on a reflection-based assembly loader. If an extensions directory is present, it will load all files inside it with the FEX file extension, check the assembly for an IFudgeExtensionServer, and ask it for a list of IFudgeExtension to load.

FMod - In a Mirror, Unsafely

The last critical feature of Fudge, the first GUI editor based on FMod, is completed! I added the level adder just today and it works perfectly! While getting that set up, I had to modify some of the updater subs to keep the right level checked in the Levels menu, which included changing the loop to look for the Tag property instead of using a loop counter.

Remembering that there is no Galaxy editor to render any standard tile past the end of the tileset, I generalized the infoplane value editor to work for all planes. To make it obvious that it was a different plane that had strange values, I made use of the ColorMap class in Imaging to tweak the composite hex square thing's background color.

There was still, of course, the issue of flickering when moving about the level viewer. Microsoft, for some reason, made the DoubleBuffered property protected from outside change. Therefore, I tried to use reflection to set it on startup. It worked at first, but the Drawing assembly soon threw the awful inexplicable ArgumentException from nowhere. So, I guess I'll be creating a class to inherit from Panel and expose that property at runtime. I really would have preferred to keep my GUI design visual, because this is Visual Studio...

FMod - Shortcuts

The Fudge GUI for the Keen Dreams part of FMod is coming along very nicely. In fact, it's almost release-worthy, but there are some other tools I want to add to it first.

Today, I made the re-render pass after changing a tile only apply to that little area of the screen. Now, it doesn't cause all kinds of blinking whenever you just change one tile. Then, I added some labels near the tile palette to indicate the status of that plane. Those labels can also be clicked to increase interaction and right-clicked to decrease the interaction of that plane. I also made the Delete Level tool strip item actually do something: delete the current level after asking to confirm. The only absolutely necessary feature left is adding levels (and maybe resizing them). Finally, I tweaked a few shortcuts and added stuff to the key press view shifter routine to make holding Control and Shift cause extra shiftage.

Robotics - More Issues than Anticipated

I assumed that the issues with the string and flag raiser interfering with each other could be fixed by routing the string through zip ties around the flag raiser's spinning area. To do that, we also had to redo the zip ties around the NXT because it's cables were also in the way of the flag raiser. It turns out that turning string and still pulling things with it is physically impossible, so we ended up derailing our treads from their motors and bending the superweak aluminum that Tetrix gives us for building. In addition to that, the flag raiser somehow cannot reach the flag anymore because the robot can't fit into that corner. So, we now have quite a few more issues than anticipated.

FMod - Wow Many Transparent

Apparently, I don't need to call MakeTransparent() on a GDI+ bitmap because I can just set pixels to Color.Transparent. So, I wasted quite a few hours trying to figure out why the transparency was going insane. Well, now I know and I can happily move on to other things.

I really only worked on Fudge for a few minutes today, but in that time, I fixed the renderer, tweaked the controls, added an infoplane value editor (the Hexadecimal property of NumericUpDown was amazing), and started on the tile palette renderer.

It works!

Robotics - Strings and Things

There were apparently more issues with our robot than we had previously thought. Today, we went through the massive tangled mess of wires in the center, removing old connections and testing all the controls. I discovered that the flag raiser spinner thing was connected to a different control port than the code had been using for a while.

We also continued our idea of using a capstand shaft. Re-purposing an axle, we mounted a motor on the front bar of the middle area, tied some string to it, and connected that string to the hanger bar. It has the disadvantage of not letting us get the bar back up if we put it too far down, but we tested it using someone's arm and it did hold! All we have left to do is get the string out of the way of the flag raiser, ziptie the wires to safe places, and do some practice driving!

Robotics - Fully Autonomous

I somehow forgot that there was a robotics meeting yesterday and therefore didn't document it.

Saturday meetings, though inconvenient, are the most effective of the meetings. They are longer than the after-school ones and pretty much everybody can come to them, even if a little late.

A few ideas were proposed for fixing the problem of it falling back down after hanging: increase the gear ratio, brake the motors with magnets, add another hook coming in from the opposite direction, and use a threaded rod instead of a bar. We explored the possibility of the magnet brake by salvaging magnets from a destroyed gear box, but they were insufficiently powerful. While waiting for other people to do whatever they needed to, I worked on the infrared-aware autonomous. It required a little more adjustment of the sensor, but I'm fairly certain it will now work every time - scoring 60 points.

Near the end of the meeting, the builders started work on a capstand shaft, a pole with string around it that would be wound to pull up the robot. It should work, and if it doesn't, we can always decrease the diameter of the shaft.

FMod - Unusual ArgumentException

I think there's something wrong with the default System.Drawing.dll file sent with the .NET framework. When I do things with images, it throws ArgumentException on a different thread than the one I did the rendering on. It's even more of a pain because I am fully certain that I am disposing the graphics adapter object thing before letting it go out of scope, the only thing the Internet has told me to do about this. So, I have been unable to test the renderer or do pretty much anything with Fudge.

I did, however, adjust how it thinks about using in-assembly graphics. Instead of wanting to dump them to disk after using the default graphics, I am having it save that fact with the dependency file, removing the need for keeping unnecessary files around.

FMod - Fudge

With essentially all the basic formats readable and writable by FMod, I decided to take a little break from this low-level stuff and write some GUI code, putting all these classes to use. Since the community has no Keen Dreams editor, that's what I'm doing. I can draw on my experience from CFEdit, a program with a perfectly functioning GUI but nothing to use its levels.

So far, I have a very nice layout and a menu system with a lot of calls to methods that may or may not be working perfectly. Using Visual Studio's ability to embed resources in the assemblies, I can package a default version of all necessary files for ease of starting new projects. With my ArrayStream, I can read embedded game files just as well as file system files.

There's also a pretty impressive framework for rendering the three planes and I even have a way to composite nybble character pictures to display infoplane values, though that may not be necessary in Dreams. However, I can't actually render anything yet because of a strange ArgumentException being thrown somewhere in System.Drawing that is untraceable.

The current working name of this editor is Fudge.

Robotics - Autonomous II

After a long winter break (and a long series about FMod that everyone is probably tired of now), the robotics teams got back together to finish it up before the competition in a month.

Despite my use of the word "finish", nothing was finished or really done at all today. The main builders were away at basketball, so we really couldn't do anything serious. I did manage to move the infrared sensor over to the correct side, which took a surprisingly long time. (It's amazing what things in the way will do to ease of access.) A whole lot of people left early, leaving me the only person on my team left doing anything. I used that time to rewrite the autonomous code to reflect our new design, which needed to be updated since November. It now takes advantage of the daisy-chained controllers, saving three NXT sensor ports. It works pretty well, but I still need to get the servo angle and drive times right.

FMod - One Last Hurrah for Huffman

With full Keen Dreams support in the graphics part of FMod, the last (important) thing left to do about that strange episode was to edit its levels. Since Carmack compression didn't exist at the time of its release, they used Huffman on the map chunks instead. They also applied Huffman to the level headers, giving the compressed length of the main header in the super header.

Since the uncompressed structure is identical, I copy/pasted a lot of code from the GalaxyLevels class into the new DreamsLevels. Since the only existing thing using Huffman was GalaxyGraphics, I centralized some methods relating to IO of the dictionaries. People haven't done a lot in the way of modding Dreams, so I had to manually rip the map superheader out of the executable for testing. I am very confident as to the correctness of the place I took it from; there's no other instance of ABCDh followed by a relatively sparse region. However, I am considerably less certain that I took the maps' Huffman dictionary from the right place; I can't get the decompressed data to make sense no matter what I do to it. I'm almost 100% sure that my decompression and dictionary reader algorithms are right and that it's just a question of using the right dictionary.

FMod - I Have a Dream

Huffman decompression is pretty easy, especially now that I know about that weird bit order reversal that goes on with the dictionary! Compression was a little harder because I need to create a dictionary, but I had it default to the trivial dictionary (no compression) unless a better one is specified. There was also the issue of writing down that dictionary to EGADICT, which I did in a way a little messier than the original. It's OK; it's the same size and is equally fast, just looks strange to the eye.

While trying to add new GalaxyGraphicsChunkSettings instances, I discovered that some episodes have texts in a different order. That immediately presented an issue: it always assumed the order used by Keen 4. So, I changed those variables into a list of strings. Shortly, I discovered that Keen 6 lacks some resources that both 4 and 5 have. So, I went back through and inserted checks to see if various fields of the chunk settings object were zero before trying to read the resources.

Though I didn't get a chance to test reading episodes 5 and 6, I went on to Keen Dreams, that strange in-between episode that is full of exceptions and special cases. It took an hour just to get the EGAHEAD reader generalized to its four-byte arrays instead of the newer three bytes. I also discovered a problem with the hardcoded chunk sizes of the small tiles; Keen Dreams has fewer of them. That was solved by adding new fields for that to the settings instance. I'm still having trouble with getting decompressed sizes for KD, though; they always return zero.

FMod - Denounment

Even though I didn't write very much new code at all today, I did make a lot of progress on FMod! First, I adjusted my bit twiddling in the EGAHEAD writer, repairing the overflow issue and allowing Keen to read my graphics. When I did run Keen with those graphics, I found that everything was blue; only the third and fourth planes had been used.

It took quite a while of scratching my head, stepping through the code, and running tests to determine that I was running a for loop backwards. I was using my "check if value minus two to the n is not less than zero" trick to split the bits, but I was running it with n starting at zero and getting bigger, which would obviously put the bit in the wrong plane. Making it a "step -1" fixed it, and it only took me a little while to find the places where I forgot to reverse the loop bound constants, which resulted in all-black tiles.

Then, I checked the ModdingWiki and found that my question about Huffman tree reading had been answered. Keen apparently likes to reverse the bit order of the character data in the EGADICT, which was the cause of the problem. I will be able to fully reintegrate Huffman compression tomorrow.

The last little issue I had was with the large scrolling texts at Keen startup. In my reader, I had used the wrong variable inside a loop, causing the loop to have no effect. With that corrected, all that remains is to stop Keen from wigging out at the edge of those texts.

FMod - Put It Back

I learned a little bit more about Huffman compression and the dictionaries involved there today. I determined that my way of reading the dictionary is flawed and results in the strange mangled bit-backwards values I encountered yesterday. I found a lower-level guide to reading Huffman dictionaries, which I will mess with tomorrow. Unfortunately, I might not get to use my nice generic recursive binary tree class.

Continuing to ignore the massive issue of putting stuff in an actually compressed format, I decided to keep going and write the Galaxy graphics writer. Given the documents I acquired yesterday and the reader code I wrote based on them, getting the first draft down was easy. Of course, it didn't work the first time. I had mis-renamed one of the mask plane variables after adapting it from the unmasked version. There was also an issue with my byte math when trying to write the locations of chunks to the EGAHEAD, but that was also easily resolved. Keen still can't read an EGAGRAPH written by FMod, but XVI32 says I'm close - there's something wrong with the length of something starting at the fourth or fifth chunk.

FMod - Exporting all the Galaxy Graphics

Until I figure out what's going on with the Huffman dictionary, I have commented out the decompressing part of loading the Keen Galaxy graphics. It appears that EGAGRAPH is already fully decompressed from being run through KeenGraph.

Today, I wrote a little bit more code to finish extracting the other resources found in EGAGRAPH. After a little bit of testing, I determined that my temporary decompression fixed worked with the trivial Huffman dictionary. I also discovered some issues in the masked 8x8 tile reader, which were fixed by using the correct plane array. I also inverted the exported font colors to be more like KeenGraph. After polishing everything up, I wrote some test code to dump some of the resources to my test directory and it worked! Font characters, tiles, sprites and other graphics sprayed everywhere!

In the two days I have left before school resumes, I want to have a way to get all this stuff back together in the EGAGRAPH and get actual Huffman stuff working.

FMod - Galaxy Graphics

Today and yesterday, I did some fiddling with Galaxy graphics for FMod.

At first, I continued my idea of having a complicated chunk management system with different manager types for the various chunk types and interrupt-like interface hooks. Managing requests and dependencies (for the table chunks) became a terrible mess very quickly, so I scrapped that idea and moved onto a simpler allocation table. Since I would have to have different manager sets for different episodes anyway, I created a class that stores the starting and ending chunk ID of each type of chunk and a generator that makes one of those for the requested episode.

As a precursor to actually reading things, I wrote the Huffman decompressor. It was fairly simply based on the example code on the ModdingWiki. With a quick BinaryTreeNode<T> class, it was loading dictionaries and decompressing non-byte-aligned data shortly.

With that done, and file format documentation at the ready, I began writing the readers for each range. It was surprisingly easy; the hardest part was the bit gymnastics to get the EGA plane data in the right order. The code could probably be cleaned up by using lambdas instead of copy/pasting similar code, but that can always be done later. Once I had pretty much everything in place, I actually ran a test. It failed miserably on essentially the first thing: the fonts. The height was something crazy like 64 and the offsets were way past the end of the chunk. Similar crazy number errors appeared in the tables, which only quietly bungled the data.

Apparently, there are still some things about Huffman compression that I still don't understand. I'm using Levellass's "trivial Huffman dictionary", which is said to work on Keengraph-exported graphics, which is what I'm trying to use. Somehow, the Huffman effect didn't get applied to some of the later bytes. FMod doesn't know which to reverse, though Keen uses them perfectly fine.