FMod - Wizard UI Done

Ah, today I finally finished the UI for Abiathar's New Project Wizard. All the screenshots for the left bar have been taken and mapped; all the Next button instances are appropriately wired up. It needs a little bit more testing to ensure that nothing terrible happens if the user selects strange options, but this marks an important milestone in Abiathar. That Dependency Collector was pretty much the last holdout from my time making dense, complicated dialogs - and now it's gone! Well, I'll probably leave it in as an advanced option for users who are used to it and want to get a project started quickly, without dealing with a long friendly wizard.

Next, I'll need to actually write some new code to process the generated dependency file. If I do keep the Dependency Collector, it will need to generate a dependency file as well, instead of just verifying the file names. And, of course, I'll need to implement all the new features promised by the wizard's options.

FMod - Slightly Complicated Wizard

I've finished laying out the New Project Wizard, and it is possible to have to configure a maximum of 12 different screens, 13 if you count the "finished!" page. This makes the wizard way more complicated and long than I had originally wanted, but its steps are fairly easy on their own. I might have to make the old Dependency Collector an option, maybe marked "(advanced)."

It was very difficult to figure out a good way to arrange the tileinfo options. There's the old choice of whether to get tileinfo from a separate file, the MapHead, or memory (or nowhere, I guess), but now there's the choice of how to start it if you're not loading an existing file. Like the level files, this will be doable from a blank, the defaults, or an actual file.

The wizard UI is almost done. I just need to put controls on the Tileinfo File screen and get two more screenshots for the left panel. After that, it's time to code up all these new features the UI promises.

FMod - User Driven Development

Besides the normal development on the New Project Wizard, which hopefully will be all done within a week, I executed a few changes as a result of talking with a very important modder on the #KeenModding IRC channel. This person is still using an old Windows XP computer with barely any RAM, so getting Abiathar to work on it is a challenge.

Abiathar v1.4.1, the last public release, was successful in that it let him open his mod. Of course, due to his RAM situation and general oldness of the computer, scrolling with the arrow keys was extremely laggy. I'm still not sure how this is possible (it should be pretty much instant unless it trips the memory manager), but I did find some optimizable things. First, I moved the scroll detection to the top of the key-down handler. That helped a little, but lag was still happening. I tried changing the repaint call from that handler to use Refresh instead of Invalidate (to force it to happen immediately), but it made stuff slower in some cases, and I reverted it. Eventually, I added a config option to set how many tiles will be shifted per key down event, which should reduce the number of repaints necessary.

Also mentioned was the non-appearance of translucent colors (non-black pixels without the mask bit set). Fortunately, FleexCore already preserved translucent colors, so it was very easy to allow them in the image converter. It's not exactly how the game renders them, but it gives a pretty good impression of translucency.

Finally, he mentioned that it's difficult to get used to the function key controls for the plane states, especially having used other editors for a long time. Since this was also a problem for people whose function keys are permanently bound to other actions, I decided it was time to do something about it. The default is now TED5-style numbers for changing plane states, with the exception that 7-9 call the tilesets instead of the weird Space+number way. The old style can be enabled in the config file.

FMod - Further Wizardry

Wow, the New File Wizard is turning out to be a very challenging thing to implement. Today, I added a lot more of the necessary steps to the wizard, finishing the advanced settings and moving into the normal (easy for the user) configuration. All the new features I want to add are requiring a lot of new options.

The temporary CreationInfo substructure in the file config now uses an enumeration for the initial source instead of just a dump vs. open Boolean; it will eventually allow creating a blank level set in that format, which is easy for me to implement because they all accept the same type of individual level file. The GraphicsFiles structure has fields to hopefully support loading tiles from bitmaps.

Here are the new screens:

Configure tile counts and infoplane allocation, default for Keen Dreams (advanced)

Choose the dependency file's directory

Choose how to start the level project (the dump option only appears if using a template that supports it)

Choosing names of level resources

In that last screen, the names of the resource types change depending on the type of levels. (For example, had "TED5 MapTemp" been selected on the Level Format screen, "GameMaps" would be "MapTemp.") Of course, the MapDict row disappears when not using a level format that requires it. File existence checks are only done if starting from existing files.

FMod - Better Exporter

In addition to making small progress on the New Project Wizard, I did some major improvements and additions to the bitmap exporter! In fact, those additions necessitated the addition of a configuration substructure for it.

First, I added an option to preserve the zoom level. By default, Abiathar will export the level as if it was zoomed to the 16 pixels per tile level. Since one might want to make it look more pixellated (or get a farther-out view), they can now switch BitmapExporter.PreserveZoom to True and do the image export as normal.

Then, I added an option to remove the two-tile-wide level borders, which usually just have bunch of ugly "edge of map" tiles, present only to appease the in-game renderer. When that is turned on, Abiathar will render the level as normal and then grab the appropriately sized rectangle out of the middle.

Finally, I added a way to export the level image without having to type a filename. If BitmapExporter.AutoFilename is not blank, it will be used as a template for the filename. There are a few special "entities" that Abiathar will replace before saving:

• %l is the current level's name
• %n is the current level's ID
• %p is the Abiathar residing directory
• %d is the dependency file residing directory
• %f is the dependency file's name without extension

So, the AutoFilename string %d\%f_%n_%l.bmp saves the file in the dependency file's directory with the depfile's name, level ID, and level name in the file name.

FMod - Wizardry

Despite important things like college happening, I have made some progress on Abiathar. The current focus is the New Project Wizard, which will serve two hugely important purposes: improve ease of use and make it easier for me to expand to new formats.

Today, I eliminated the Back button, which was causing huge headaches for me because users could go back and prompt total reorganization of settings, which could totally blow up everything they had set up. I personally think that the occasional use of a Back button in creation wizards does not justify the huge pain that it is to make work properly through all the possible paths.

I also filled in three steps: Template, Level Format, and EGA Layout. All the format/layout steps will not appear unless the user chooses to start from a blank slate.

This is what they look like:

The default view of the Template step

Default view of the Level Format step after selecting "Keen 4" and "inspect advanced settings" in Template

Default view of the EGA Layout step with the aforementioned Template settings

You'll notice a large "From GFXINFOE" button on the EGA Layout step. This allows users to take a TED5 graphics accessor instead of typing in strange numbers. (This is designed for Bio Menace.)

Also, I added a new menu option under View: "Export Image." This allows the user to save a .bmp of the level being viewed in Abiathar, with all the current view settings except offset, zoom, and grid. Things are rendered in the bitmap only if they appear in the level view - there is no code duplication.

The Race to Artificial Intelligence

Here are some things I know:

• In computer security, there are two groups: the good guys and the bad guys.
• People are trying to develop artificial intelligence.
• Intelligence-singularity machines can modify and adapt themselves.

It hit me while thinking about these things that the race to artificial intelligence is more important than which people/companies are going to get super rich. I believe that the first infosec "side" to acquire real, adapting AI will be able to claim total victory.

It's very well established by now that every piece of software contains bugs. In fact, entropy pretty much guarantees that something is going to go awry if you run a process long enough. And as we saw in Heartbleed, little issues can explode into big trouble when exploited. One vulnerability opens another, and pretty soon you've got everything compromised. (The Walmart credit card stealing malware got in through a noncritical system.)

Artificial intelligence is going to be really good at understanding binary files, even obfuscated ones if it's smart enough. It's also going to be really good at crunching large amounts of information, correlating data with other data in nontrivial mappings (understanding context), and checking every contingency.

The way issues are found (both bugs in a process and errors in code) is by trying every possible path of input to see if it ever does something it shouldn't. Eventually, we will have AI sufficiently advanced to analyze a piece (or a bunch) of software and figure out all the bugs, including a lack of detection by a monitoring application (it'll figure out what causes a malicious program to not trip a virus scanner). This is what the race is towards.

If the good guys get it first, they'll get to analyze every program for holes that would allow an attacker to get it where it shouldn't be possible. Now, since bugs are always going to happen, things are still going to be tough for the heroes. The only way I see them winning is to have perfect integration between the user's definition of "working properly" and virus detection. This is a different application of AI: it needs to be able to tell what is going to produce a state that the user does not desire. The brute-force aspect of decision analysis comes in again: it has to figure out what the change in state will do to every aspect of the system.

There's also the in-betweenish case of the good guys who will be good by virtue of eliminating the bad guys. Basically, fighting fire with fire: use the AI to detect security holes, exploit them to get into the bad guys' stuff, and make it impossible for any computer to run another AI ever. It sounds Communist, but it might be the only way to win.

If the bad guys get it first, they'll be able to (again) analyze all the things, but they're not interested in fixing it. Instead, they'll set up vast networks of zombified computers, analyzing every program ever to break into everything. Once every vulnerability is cataloged, the AI can create programs to exploit it. Mass information theft, large-scale denial of service, and general badness ensue. Since they got AI first, they can analyze the problems that will be in the good guys' AI, thereby staying a step ahead. (I'm assuming that computational power is equal.) This is why time matters: AI needs time to adapt.

Alright, stop reading my blog and go develop AI for the good guys. Time is of the essence!

FMod - The Wizard's Tricks

Today slipped by without me really accomplishing anything. (That kind of day makes me very sad and disappointed with myself.) I did make a tiny bit of progress on Abiathar, in that I theoretically solved a problem I had been thinking about for a long time. Oh, and I made more work for myself.

The problem I had been dreading dealing with was the merge of the Gather and Open subroutines. Currently, they both do a lot of work figuring out what data to put in what stream. This work is extremely similar, but different in that Gather has to deal with files being taken from memory on a one-time basis, while Open can trust that this is the way it will be done every time. Gather starts editing from the settings in the Dependency Collector; Open starts editing from the dependency file. Once they have figured out where to get the stream data, they pass Stream objects to a common FilesOpened subroutine.

Since Open takes an existing dependency file and Gather needs to create one, the obvious solution is to have the Dependency Wizard create an actual dependency file object. This works great, except for one thing: what if the user wants to start from default maps? To solve this, I added a new structure to AbiatharFileConfig that will not be saved or loaded; its only purpose is to inform the new unified file opener that some files need to be created. This has not yet been implemented, but I believe it will work great.

I also noticed a huge empty space to the left of where I'm going to put things in the Dependency Wizard. Other wizards (like installers) have some artwork in a tall panel. Inspired by this layout, I fiddled with the wizard form layout to be more like it. I created a Dictionary(Of TabPage, Bitmap) to display a different image for each tab page. I tried making them screenshots from the game, but [1] those are a huge pain to prepare and [2] they're not super relevant. I might stick with this style, but I might invite people from the community to make relevant artwork.

FMod - Has an Icon

Yesterday, I quickly wrote up an invitation for PCKF members to create an icon for Abiathar, which I would set as the application tile in the next version. There were two responses, each containing multiple images. I picked three of my favorite submitted images, rescaled two of them, and put together a multi-image ICO file. This will show different icons at different resolutions (view settings in Explorer).

The icon and credits will appear in v1.5.

Tales from a Windows 95 Compatibility Engineer

Today, while I was busy not implementing the Abiathar dependency wizard, I started reading "Tales of Application Compatibility", the author of which I do not know because it's a chapter from an eBook. Anyway, it's amazing. From what I can tell, the author was a software engineer on the Windows 95 dev team responsible for making sure Windows 3.1 programs ran on Windows 95.

The chapter is composed of many small anecdotes, each the story of an application whose horrifying abuse of the Windows API and common sense caused headaches for Microsoft. Microsoft couldn't just blame those coders because the general nontechnical population wouldn't understand; it would be assumed that it was a problem with Windows 95.

Anyway, go read it:
Tales of Application Compatibility

FMod - Dependency Wizard Time

I did a tiny bit of tinkering with the GalaxyLevels class in FleexCore in preparation for full Bio Menace support (which, incidentally, I still have not played). I added support for non-Carmackized levels, which skips that part of the compression in the level saving and loading. The levels loaded properly without this, but I added it just to make sure it didn't mistake some infoplane value for a Carmack sequence.

But, to manage all these options, I'll need a better Dependency Collector. I've been trying to fit all the options on that one form, but it's super difficult to understand and still can't hold all the options (like custom entry of EGA offsets). So, for both those reasons, I'll be creating a wizard of sorts, like those you see on installers. It will be a bit tricky to keep the template system useful, but I think a page-by-page wizard with helpful text will be far more easily understandable to new users.

With that done, I'll be able to add features like loading graphics from bitmaps or having a different set of tiles for the infoplane layer than those IDs in the foreground.

FMod - Simpler Than I Thought

Today, I did a whole lot of digging through the TED5 source code, discovering all kinds of interesting things. For example, the tileinfo data stored in MAPTHEAD is RLEB compressed. The flag byte is the low byte of the RLEW flag word stored in the same file. Anyway, I used all that to implement the FromFiles and ToFiles method for loading and saving from MAPTEMP and MAPTHEAD.

After a little bit of testing and a lot of confusion, I figured out that I had wasted a lot of time. The game doesn't actually load MAPTHEAD; it uses the normal MAPHEAD as an accessor to MAPTEMP. I don't know what MAPTHEAD is actually used for, but it probably is supposed to actually be a temporary file. It turns out that I didn't need this giant TedLevels class; Abiathar will happily load the Bio Menace levels with the standard GalaxyLevels class, using the appropriate file names. (I couldn't actually see the level because I didn't bother configuring the EGA layout, but it looked like it had a sensible layout.)

So, basically, FleexCore now supports TED5 temporary files for no reason at all. At least I advanced the community's knowledge of these formats...

FMod - TED5 Temporary Files

One feature requested by the Keen community is the ability to edit Bio Menace levels, which requires understanding the TED5 internal map format. It appears that the BM developers didn't want to wait for TED5 to Carmackize and compile the levels into the standard iD Software game maps format, so they swapped out the TED5 loader for the normal loader. This means I will have to be able to load all that extra stuff and then adjust it properly for custom levels.

There was one problem with this before today: the ModdingWiki had incomplete documentation on the format. Fortunately, I have on hand the TED5 source code (in C), so naturally I went through it to find the MAPTHEAD entries I wanted. While doing that, I uncovered everything I needed and more! There was easy data on the GFXINFOE format and more TEDINFO fields, in addition to a better explanation (and in one case correction) of what was on the wiki. All the things about MAPTHEAD have been put up there, but there is a little bit more investigation (through the actual use of the fields) is required for the others.

I haven't actually implemented the formats, but I will be able to easily implement INextGenLevelSet with this, allowing easy integration into Abiathar.

FMod - Manage that Memory

One major outstanding issue with Abiathar was that it could not load large level sets on 32-bit computers. The view state caches took up so much room that they would overflow the amount of memory granted to a managed process on an x86 platform. This was impacting one of the Keen community's important modders, so I took another crack at solving it.

First, I looked into the MemoryCache class in the Runtime.Caching assembly. At first, I thought I could easily reorganize the view state dictionaries into a MemoryCache accessed with a string like "1/Background" but then I discovered that almost all the planes need continued access to their cache to render changes efficiently. I did more research on an equivalent of Java's SoftReference, but there is no nice solution that doesn't require me to rework everything.

This was when I noticed that every plane for every level is given a giant bitmap when the dependency file is opened. It was very easy to move all the view state creation into the level and tile palette switch segments. That had a great effect: I could open the Atroxian Realm files in 32-bit mode without crashing. Unfortunately, it would still run out of memory and die if I looked at enough levels.

Then, I thought of a brilliant hack. As long as the represented level is not being viewed, it's possible to delete a level view state, thereby destroying and disposing its graphics resources. Another brilliant hack involved the nature of the out-of-memory error: it's an exception and can be caught - and therefore squashed before it affects the user. With a few invocations of Process static methods from System.Diagnostics, I had a cache deleter wired up to the out-of-memory handler. When System.Drawing throws ArgumentException, Abiathar starts deleting level view states until the working set size is below 1400MB. The executing method can then invoke itself from the exception handler, restarting its work with available memory.

I tested this more, polished it up, and released it as Abiathar v1.4.1. Last advantage of The Omegamatic: gone.

FMod - Slim Config

I'm going to start implementing some of the features suggested in the "Abiathar: what's next?" thread on Keen Modding, but today I focused on other things, mainly college. (Priorities!) However, I did do a little bit of improvement on Abiathar.

One of the things I was very unhappy about in v1.4 was the size of the defaults.aconf (template) config file. Generated new, it was 50KB. Most of that was representing the InfoplaneInspectorEntryData structure, which contains seven almost-always-False fields. By adding a few attributes to the FrofConf fields, I was able to stop it from serializing certain fields if they had a certain value. I applied <FleexConfSaveIfNotEq(False)> to all the fields in that structure, cutting the total size down to 32KB. That's still quite a bit, but it's a lot better. I also used these awesome new attributes to remove the MapDictCommand field from the PatchGenerator structure if it wasn't used.

Another attribute, called FleexConfLoadOnly, will be used for backward compatibility and conversion. I plan on removing the HuffCompressedMaps entry and combining it into a very amazing currently-secret structure. To do this and make it not break existing dependency files, I'll need to continue loading it but not resave it. This will allow me to use it in the AdaptVersion subroutine to modernize it and then delete it.

FMod - What's Next?

With Abiathar v1.4 out, adding the ability to generate patch files, I'm not really sure what direction to take the program next. Other than bug fixes (which haven't come in yet), there doesn't immediately seem to be anything more to do with it. I could polish the interface, snazz up the documentation, remove the version number, and seal it forever as simply "Abiathar", but I don't want to kill its development. I learned a lot about programming (especially the world of graphics and rendering) creating it, and I think it's friendly enough to build around.

So, what's next? I don't really know. I've been fantasizing about cooperative editing, Google Drive style, and v1.4 added infrastructure that could be used to make that a real thing (I couldn't help it). The Keen community is small enough that such a feature probably wouldn't get too much use, and it probably a waste of my time. People would be amazed, and it might get used once or twice after the release, but the *.aslev (single level file) system is fine for putting together group projects.

Then there's expansion to other similar games. Bio Menace is one that still has a little bit of interest, but its file format is difficult to get back onto disk correctly. I can load the TED5 not-so-temporary file, stripping out the parts that don't matter for level editing, but saving it and putting all that junk back in the right place would be difficult. I could go all-out and make it compatible with everything TED5 can do, but that would require so much reworking of a lot of the concepts. Abiathar would lose its power as an adapted-to-Keen superpowered, focused program.

Finally, there's expansion to Vorticon levels. This might be the craziest idea. It wouldn't be hard to get a Vorticons editor going, but merging it with Abiathar and its attachment to the Galaxy way would be very difficult. Then again, it would go a long way towards satisfying my vision of one program for all Keen modding ever.

In short? We shall see. Expect me when you see me.

FMod - v1.4

It's here, everyone: Abiathar v1.4! The following is my announcement/sales pitch about it on PCKF:

First up, the most awesome feature: automatic patch file creation! Click "Write Patches" under File and Abiathar will generate a patch file (for maphead, mapdict, and tileinfo where appropriate), a batch file, and a CKPatch executable. Don't have an UNLZEXE'd game file? No problem, Abiathar's "Un-LZ EXE" utility, also under File, has you covered! In fact, you can just drag an EXE onto Abiathar and it will un-LZ it.

I also fixed a whole bunch of bugs, including the File Emitter not working at all and the annoying dependence on the absolute location of the file. That's right, you can now work with the same adeps on multiple computers! In that vein, I also made saving files a lot more convenient; Abiathar starts you off in the place containing the resources.

The horror of persist.aconf's Profiles section is finally over! You can now modify editor.aconf, containing view settings, without being overwhelmed by all that other trash, now locked in defaults.aconf. In fact, the entire profiles system is no longer around; those things are now more like templates. To manage them, I added the Template Manager. (Read more about that in the help file if you want.) Unfortunately, this means you will have to create new adeps files, as the format is not backwards-compatible. (Open them up in Notepad and you'll see.) On a positive note, infrastructure is in place so you never have to do that (or replace a deleted-after-update persist.aconf) again!

In the miscellaneous department, there's now a snazzy splash/loading screen that greets you with a random exclamation on every launch. Also, my beta testers get a nice treat; enter a beta key I give you to download the private beta version in question. Finally, you can drag an ADEPS file onto Abiathar to open it without going through File->Open.

FMod - Those New Features

Today is a great day for Abiathar!

I first changed all the business with RangeConfigEntry, making the AppliesTo entry that appears on every InfoplaneInspector record to be a list of integers. That cut down the file size by 10KB, only reducing the total size to 50KB, but that was still a huge amount of data being wasted.

I then added a PatchGenerator entry to the file config, containing the extension for the %ext line and the directives for adding maphead, mapdict, and tileinfo. The tileinfo is always a %patchfile with an offset, but only KDRPatch is missing the nice %maphead directive which can be used for the Galaxy episodes. There is now a "Write Patches" entry in the File menu, which dumps the appropriate patcher (record added to DefaultResxLookup.GetFileSet), writes the patch file, and creates a short little batch file to run the thing. This should cut down on the number of e-mails I get about Abiathar.

Then, to make distribution of beta versions easier, I added a "have beta key?" menu option under Help. It doesn't appear unless the appropriate option is set in editor.aconf, so it shouldn't confuse less advanced users. The menu command prompts the user for a key, which will be used to fetch an update file from my Dropbox and update in a similar method to the stable version. If the update file does not exist, Abiathar will get a 404 and tell the user that the beta key is invalid - no secrets are stored in the client program.

Next, I created a simple little template manager to facilitate sharing of the profile/template entries. It required a little reworking of the Dependency Collector to load the updated defaults.aconf, but it's working quite well. It even allows the user to create a template out of a dependency binder, which can be useful if sent one with special configuration.

Finally, because this confuses new modders, I started to tackle the absorption of UNLZEXE. I already had the Windows version of the utility packed in the File Emitter, so it was trivial to add an "Un-LZ EXE" utility to the File menu. It just dumps the utility, runs it on the user-specified file, deletes the old version, and renames the "exenew" back to the normal name. Of course, it does checks to make sure that decompression really happened before deleting stuff.

Abiathar v1.4 is looking great! Tomorrow, some more testing, documentation updates, and maybe the release!

FMod - Stable in All of Its Ways

The move to FrofConf is complete! Just today, I wrote the initializer of the ProfilesConfig class, thereby allowing Abiathar to generate new files from the templates there. The representative configuration file is over 50KB, though, mostly due to the heavy use of a numeric range structure. It's definitely not worth it to have that, especially when the biggest range is about 8, with a normal one spanning 3 IDs.

I also fixed a whole bunch of bugs in the stream I/O of the FleexConfig base class. The Dependency Collector is working as normal, now with the ability to have spaces in template names! The Step procedure for selecting resources has been made immensely more convenient with a relaxation of the in-working-directory restriction.

Soon, I'll be adding a way to manage (import and export) templates, which should make the addition of new templates simple. Tomorrow, I'll rip out all the business with List(Of RangeConfigEntry) and just have a List(Of Integer). But for now, Abiathar is stable and ready to be made even more awesome with new features!

FMod - The New Configuration

Abiathar is again compiling! I did a whole bunch more hack-and-slash removal of the old ConfigHelper-based config engine and migrated everything over to the FrofConf (FleexConfig) way. I finished writing the representative structures for the editor, file, and template configurations. All mention of the profile system has been ripped out, even in the Dependency Collector.

The DC now holds onto a selected template, which is duplicated into the file config when opened. All the HasMapDict business has gone away; it's just a Boolean field. Some casting needed to be done to pass the configuration through the Interop module, but it's type-safe now. In other news, Abiathar now throws up a splash "loading" screen instead of appearing to do nothing while loading.

Currently, I'm having a little trouble with it deciding to create a zero-byte file instead of actually saving the configuration. The editor configuration has defaults now, but I still need to write in the defaults for all the profiles - that's going to be the hardest part of this entire upgrade. Once all that is done, new features can happen easily!

FMod - Reconfigurating I/O

I executed the change in file configuration, changing it from one XML document to a dictionary of FROF configuration structures accessed by a string name. This caused more compile errors than I expected. Since I was in triage mode anyway, I went ahead and implemented the new event bus style that I had been thinking about for a while. I fixed up all the broken things and then started ripping apart the first class I ever wrote for Abiathar: the dependency binder.

I deleted the file pointers in that class, moving them to a FrofConf structure. That broke a huge amount of things in the saving and loading routines, which I still have not fully fixed (they're not even building now). The profile infrastructure is something I threw together a few days before release to support NetKeen and quickly became dissatisfied with. Instead of mixing data between the file configuration and the profile in the general configuration, the entire profile in the general config will be copied to the file config, allowing it to be modified without messing with the blueprint. This is possible because of a new Duplicate function on any FrofConf structure.

I decided to place these structures in a separate file, "defaults.aconf", while the settings for less advanced users remain in the equivalent of "persist.aconf", which I may or may not change. A bug in the File Emitter has been fixed, allowing it to correctly output files with sizes greater than 8KB. So far, v1.4 looks to be a promising version!

FMod - Refactor by Triage

As I said last post, it's time to migrate to the new reflection-based configuration system. I did a little modification of the deserialization routine, moving the duplicated type branch into one method, like the serializer. I also added support for null values, which will make things a lot easier with actual use.

Abiathar has two configuration files loaded at any given time: the general editor view settings and the file-specific settings. Today, I changed the signature of the file-specific configuration file to the new config tree and deleted the XML initializer in the dependency binder class. This led, as one might imagine, to a lot of errors in the code, though not as many as I expected. Most of them appeared in the Level Inspector and Resource Accountant methods, which comprise some of the most convoluted code I've ever written. I ripped out the terrible hierarchy of classes and the deserialization from super-opaque strings in the XML configuration. It was replaced with a complicated, but very easy to modify, system of configuration compounds.

Before all that, though, I tried to fix the memory-related error that caused loading of large level sets to fail. I tried various strategies to allocate more memory for Abiathar (specifically the unmanaged graphics part), including AllocHGlobal. That didn't work - threw OutOfMemoryException - so I tried P/Invoke with SetProcessWorkingSize. That also failed with some overflow error inside IntPtr. Eventually, I noticed that I had set the compiler to create a x86-only binary. After changing it to use x64 if possible, I threw in a helpful error message (instead of "error creating dependency file") to the New window and called it OK.

Tomorrow, I'm going to create some sort of event bus-based system for creating new configuration compounds (since extensions can no longer just look around the XML document and tack on entries). If that works, I may also go ahead and change everything to use FrofConf.

FMod - Better Config

To implement the patch generator in Abiathar, I need to store some configuration. Unfortunately, the XML config file has begun to be a giant pain to work with. Not only is it unnecessarily bulky, it is difficult to update (to add fields into) and some areas - especially the inspector settings - are complicated and difficult for users to modify.

Therefore, I invented a new way to store configuration information. It's kind of like JSON, and way more strongly-typed than what I have. Currently, I have to pass the ConfigHelper a "path" to the configuration entry whenever I want to access a variable. There's caches for frequently-accessed config entries, like the hex/decimal defaults. All of this is really a mess and I want to be able to use something like "Config.Colors.Grid" (an actual object) without having giant methods that need to be changed all the time to save/load the config.

My way is Fleex Reflective Object Field Configuration, FrofConf. It identifies configuration entries in a config object with attributes and reflects over them to write a hierarchy. Currently, it supports numbers (of all kinds, loaded using field type), strings, compounds, and lists (of any type, including compounds and other lists). All structure and type data is determined by the object. Addition of new fields is simple: add the configuration field to the object, the default (from the constructor) will be used when loaded for the first time, and all fields will be saved.

Today, I got everything about this system working. Tomorrow, I will actually have Abiathar use it. This will involve a major rip-out of the XML stuff, but it will definitely be worth it for code cleanliness and maintainability.