Policy Plus - Category tree construction

Some time ago, I finished writing the ADMX/ADML parsers for Policy Plus. The next thing to do is turn that data into useful hierarchies of objects (like the categories with parent/child relationships). So today, I started filling out the BuildStructures method with category management code. It can now figure out the tree structure of categories after loading. I also corrected the type of the ClientExtension field - it's supposed to be a GUID (or String), not a Boolean.

While testing the new code, I discovered something interesting about the default Windows ADMX files. Several categories defined in inetres.admx have invalid parents. That is, the parent field references a nonexistent category. The real Group Policy Editor seems to make these "orphaned" categories top-level categories when they try to reference categories in other ADMX files, but ignores them when they reference absent categories without a namespace prefix. Policy Plus's current behavior is to always put orphaned categories at the top level, but this will probably change eventually.

FMod - Number format consistency

"Gridlock" reported to me via IRC that some Abiathar tools, notably the Tile Property Modifier, always show tile IDs and coordinates in decimal no matter what the ShowInHex settings are. That is indeed a problem, since it makes it difficult to see whether you're looking at the right tile/location. I added a new API function to determine the appropriate display string for a number and have started updating the StandardTools extension to take advantage of it.

OBS Surprise: "Reset Size" required after changing source's resolution

While fiddling around with Open Broadcaster Software (a very good free program) for recording a screen and a webcam, I experienced something unusual. After manually adjusting the desired resolution of the webcam to be slightly larger than I had it before, the size of the resulting rectangle got smaller and also moved around a bit. I was very confused. After more fiddling around, I figured out that the Reset Size context menu icon on the source made it take on the correct size.

Bugchecked: In the queues with Ben N

Today I recorded and uploaded the first video of a new series, Bugchecked: In the queues with Ben N. In this series, I review posts on Stack Exchange while commentating, and I have my face on a webcam picture-in-picture too, for extra expression. The purpose of the series is to provide entertainment (to people who are somehow entertained by examinations of post quality?), to help me express and coalesce my personal policies in reviewing, and to help post authors understand what goes on in review.

You can watch the first episode now.

Disabling writes for a program with Low Integrity

Windows processes each have an "integrity level" in addition to a user token. Even if the user has sufficient access to a particular resource, no process can write to an object at a higher integrity level than itself. Most objects have Medium integrity, so processes at Low integrity can barely write anywhere.

To launch a process with Low integrity, you'll need PsExec:

psexec -l -i cmd.exe

That produces a command prompt running at low integrity, which you can see if you do a whoami /all. If you try writing to normal files/folders or Registry keys, you'll get an access-denied error.

Runaway string literals

The newest version of Visual Basic .NET allows multiline string literals, so that the resulting string values can contain linebreaks with no messing around with vbCrLf. The downside is that when the closing quote is removed (e.g. while backspacing out a string), the string literal can just go on and on. When that happens, code is temporarily reinterpreted as part of the string, blowing away syntax highlighting, causing errors in the Error List window, and slowing the machine. There does not appear to be a way to turn off multiline string literals or fix the runaway problem. I sincerely hope Microsoft fixes this soon, and I'll keep on the lookout for workarounds.

Web hosting on Amazon S3

I've been looking around for web hosting - just a place to store a few static files accessible under a normal domain I bought - and a friend mentioned Amazon S3. S3 is a highly scalable file storage service with a price that's fully determined by how much data is stored and transferred. Since I'll only use a tiny amount of data, my price is a bit of loose change per year, and the first year is completely free.

I followed the setup instructions for websites, which involved setting S3 nameservers as authoritative for my domain and adding a few entries to my Route 53 (Amazon DNS) dashboard. Once the nameserver change went through, I had a website. Updating it is as simple as uploading some files into the appropriate bucket.

Highly recommended, at least from what I've experienced so far.

Using ASP.NET master pages as templates to create static HTML pages

ASP.NET has the concept of "master pages", which can contain placeholders to be filled in by normal pages. If you aren't actually running any .NET code, there's no reason to make the web server go through the work of rendering the full page from its parts. Since I wanted to get a bunch of static pages instead of ASPX ones, I wrote a PowerShell script today that compiles ASPX documents into standalone HTML pages (and fixes up links):

$template = Get-Content $args[0] | Select -Skip 1 | Out-String
$template = $template.Replace('asp:ContentPlaceHolder ID', 'asp:ContentPlaceHolder id')
$placeholders = $args[1].Split(',')
Get-ChildItem *.aspx | ForEach-Object {
 $text = Get-Content $_ | Select -Skip 1 | Out-String
 $finaltext = $template
 $contentnumber = 1
 $placeholders | ForEach-Object {
  $masterplaceholder = '<asp:ContentPlaceHolder id="' + $_ + '" runat="server"></asp:ContentPlaceHolder>'
  $valueopener = '<asp:Content ID="Content' + $contentnumber + '" ContentPlaceHolderID="' + $_ + '" Runat="Server">'
  $content = (($text -Split $valueopener)[1] -Split '</asp:Content>')[0]
  $finaltext = $finaltext.Replace($masterplaceholder, $content)
  $contentnumber++
 }
 $finaltext = $finaltext.Replace('.aspx', '.html')
 $newname = $_.Name.Replace('.aspx', '.html')
 $finaltext | Out-File $newname
 Remove-Item $_
}
Get-ChildItem *.vb | Remove-Item
Get-ChildItem *.config | Remove-Item
Remove-Item $args[0]

It assumes that your Content elements are named sequentially. The script takes two arguments: the name of the master page and a comma-separated list of placeholder IDs. For example:

.\compile.ps1 example.master title,text

The script blows away ASPX pages, VB code files, configuration files, and the master page. Therefore, you should copy your site into a different folder before running it. (I had it clean the folder for ease of uploading the site.)

A quick guide to assembling 16-bit programs with TASM

Today I needed to translate some 16-bit DOS assembly to machine code, and I was unable to find an instruction reference that included the binary representation of the commands. Therefore, I did some fiddling around with a DOS version of TASM, which I happened to have laying around.

To assemble a set of instructions, first put it into this structure:

cseg segment 'code'
org 100h

start:
YOUR ASSEMBLER HERE

cseg ends

end start

Save that as a text file, test.asm for instance. Open up your DOS emulator and run this:

tasm test.asm

It produces an object file, which must be linked into a real executable file. For that, there's a different TASM utility:

tlink /t test.obj

You'll then have a COM file which you can open with a hex editor to get your machine code.

I used TASM 3.1 and TLINK 5.1, which are probably available online somewhere.

When IDA Pro 5.0 fails to start

In the past, I've tried to use the IDA disassembler (the freeware version 5.0) on Windows 8, but it always threw some error on starting, and wouldn't let me do anything after that. The problem didn't seem to be helped by compatibility modes on the main program A couple days ago, I managed to get it working correctly. The trick was to run the setup program in compatibility mode for Windows XP SP3. Once setup completed, I didn't have to manually apply any settings to anything else, and it's been working ever since.

Solving the lsass.exe high CPU problem

A comment on the post about lsass.exe taking an unreasonable amount of CPU mentioned that it happened when Chrome ran. My situation also involved Chrome, so I searched for "chrome lsass high cpu" and found this Chromium issue page. One comment there discovered a workaround, which worked for me on the machine in question.

Remove the folder named with a SID that resides here:

%APPDATA%\Microsoft\Protect

Danger! That folder contains things related to a private key of some description that belongs to you. I do not know what will happen to cryptography-related things like EFS if you torch it. Consider backing it up first.

Setting a file's creation or modification date with PowerShell

Windows files each have a creation date and a "last modified" date, but those properties can easily be modified. Any user that can write to the file can set the creation, last-write, or last-access time to anything at all. That can be accomplished with the .NET methods SetCreationTime, SetLastWriteTime, and SetLastAccessTime, respectively.

You can use those without writing a .NET program. Just open PowerShell and type something like this:

[System.IO.File]::SetCreationTime("C:\America.doc", (Get-Date "7/4/1776"))

When critical OS processes consume too much CPU

Today I dealt with the issue of lsass.exe consuming an insane amount of CPU and making the machine run exceptionally slowly. A quick Google search turned up nothing helpful. Since I already had Task Manager open, I tried setting the offending process's priority to Below Normal. (Terminating that process is not an option.) Updating the priority actually worked. It kept using a lot of CPU when nothing else was going on, but the system became vastly more usable.

Of course, this solution is less than ideal, and according to the Task Manager warning, may lead to system problems. It's working for the moment, at least.

Policy Plus - ADML loading

A couple days ago, I wrote the ADMX loader for Policy Plus. Today I wrote the ADML (language-specific resources) file loader. It's fairly straightforward: it loads the XML document and extracts the string table and presentation table. The official ADML documentation was helpful, with only a few mistakes in whether certain attributes or elements are required.

You can read the AdmlFile.vb code on GitHub.

Checking whether a file is sparse in Windows

Windows has a concept of sparse files, which are expected to be mostly zeros and therefore have special handling in NTFS. Only sectors with data in them are written to disk, the rest show up as zeros when read but take no disk space.

If you're interested in seeing whether a certain file is sparse, you can use the fsutil utility:

fsutil sparse queryflag C:\path\to\file.ext

Policy Plus - ADMX loading

Today I wrote the first part of Policy Plus: the ADMX loader and parser. The AdmxFile class can load an ADMX file and extract its referenced namespaces, defined products and their versions, supported-on definitions, categories, and policy settings. Policy settings were very challenging because there can be a lot of Registry entries associated with the policy, both attached to the enabled/disabled switch and individual controls. The ADMX format documentation was tricky to navigate, missing one thing, and, in one or two other cases, actually wrong. Hopefully my code will be helpful documentation until I can write up something better.

I successfully loaded some ADMX files and was able to poke around them with Visual Studio's object inspection tools. The next step is to write a loader for the language-specific (ADML) files and then compile all their references into human-readable identifiers.

Introducing Policy Plus

The tool used to adjust Group Policy settings on a local machine, the Group Policy Editor (gpedit.msc), only works on Pro or Enterprise editions of Windows. You can find Things On The Internet that let you get an old version (judging by the icons) of the tool on Home editions, but that's legally dubious. Therefore, I think it would be good to create a similar program that's free of licensing/edition issues and has more features. Specifically, I'd like these additional capabilities:

• Search for policy settings by Registry entry
• Show the Registry modifications made by a policy
• Edit the local Registry or a .POL file
• Show what items (e.g. products, policies) are defined in each ADMX file
• Load ADMX files from user-defined sources

I don't have a name for the program yet, but its working title is Policy Plus. Today I created the GitHub repository for it, chose a license - Creative Commons Attribution 4.0 - and created a Visual Studio project. I also reviewed the .POL format again. The first thing to do, I think, is implement an ADMX/ADML loader.

FMod - Conveyor tileinfo

I learned today on the Keen:Modding forum that all Keen Galaxy episodes - not just Keen 6 - contain the code for conveyor belts, both right-moving and left-moving. The same tile properties work in all games. I had only added the right-moving property for Keen 6 to Abiathar's Tile Property Modifier, since that was all I had known about before.

Today, I updated the defaults.aconf generator to include both directions in all Galaxy episodes, and added corresponding code to the in-place upgrade procedures. That some K:M post mentioned another property that results in bizarre jiggling around, so for the sake of completeness, I added that one too.

Suppressing special folder views from networked machines

Some folders in Windows have a special entry in their desktop.ini that tells Explorer to treat them differently than a directory with a bunch of files. The Fonts folder is like that; so is the Recycle Bin. When such folders are examined from across the network, Explorer sees the same desktop.ini, but the special folder views are usually not aware of the remoteness of the machine. Therefore, Explorer ends up displaying the corresponding information for the local machine, no matter which computer's files are being browsed.

To avoid that problem and make the folder show up as a standard holder of files, you can deny permissions on desktop.ini to network users:

icacls desktop.ini /deny NETWORK:F

(Command prompt required, since you can't see the raw files if there's a special view!) That rule will apply only when accessing the file remotely; local use of it will be unaffected. The special folder view will apply to the local machine, while remote machines will see the raw directory listing. The rule can be removed with:

icacls desktop.ini /remove NETWORK

Based on my Super User answer.

When a folder looks empty from the network but has files

I just saw an interesting thing. A folder appeared to be empty when viewed from a different computer on the network, but had files in it when viewed on the local machine. I made sure both computers were showing hidden files, but that shouldn't have mattered because I was using dir /a in an administrative command prompt.

The administrativeness of that prompt mattered because the folder in question was only accessible to administrators. I understood what was going on when I remembered that UAC always limits the tokens that result from non-domain network authentication. (I was just accessing a different computer on a workgroup with username and password.) It's kind of bizarre that I just got an empty directory listing rather than an error, but that's how it is, apparently.

Changing the security settings on that folder let me access it from the network.

When the Windows 10 upgrade fails with "Operation failed: Mount WIM file"

I continued attempting to upgrade a 64-bit Windows 8.1 laptop to Windows 10 today. It would consistently get to about 24% in the pre-reboot setup screen (when the old OS was still running) and then fail with the generic "Something happened" message box. This appeared at the end of the setuperr.log file:

2016-06-09 11:37:53, Error                 SP     CMountWIM::DoExecute: Failed to mount WIM file C:\$WINDOWS.~BT\Sources\SafeOS\winre.wim. Error 0xC1420127[gle=0xc1420127]
2016-06-09 11:37:54, Error                 SP     Operation failed: Mount WIM file C:\$WINDOWS.~BT\Sources\SafeOS\winre.wim, index 1 to C:\$WINDOWS.~BT\Sources\SafeOS\SafeOS.Mount. Error: 0xC1420127[gle=0x000000b7]
2016-06-09 11:37:54, Error                 MOUPG  MoSetupPlatform: ExecuteCurrentOperations reported failure!
2016-06-09 11:37:54, Error                 MOUPG  MoSetupPlatform: Using action error code: [0xC1420127]
2016-06-09 11:37:54, Error                 MOUPG  CDlpActionImageDeploy::ExecuteRoutine(329): Result = 0xC1420127
2016-06-09 11:37:55, Error                 MOUPG  CDlpActionImpl > > >::Execute(441): Result = 0xC1420127
2016-06-09 11:37:55, Error                 MOUPG  CDlpTask::ExecuteAction(3243): Result = 0xC1420127
2016-06-09 11:37:55, Error                 MOUPG  CDlpTask::ExecuteActions(3397): Result = 0xC1420127
2016-06-09 11:37:55, Error                 MOUPG  CDlpTask::Execute(1631): Result = 0xC1420127
2016-06-09 11:37:55, Error                 MOUPG  CSetupManager::ExecuteTask(2024): Result = 0xC1420127[gle=0x0000013d]
2016-06-09 11:37:55, Error                 MOUPG  CSetupManager::ExecuteTask(1987): Result = 0xC1420127[gle=0x0000013d]
2016-06-09 11:37:55, Error                 MOUPG  CSetupManager::ExecuteInstallMode(804): Result = 0xC1420127[gle=0x0000013d]
2016-06-09 11:37:55, Error                 MOUPG  CSetupManager::ExecuteDownlevelMode(389): Result = 0xC1420127[gle=0x0000013d]
2016-06-09 11:37:58, Error                 SP     CDeploymentBase::CleanupMounts: Unable to unmount the directory C:\$WINDOWS.~BT\Sources\SafeOS\SafeOS.Mount. Error: 0xC142011C[gle=0xc142011c]
2016-06-09 11:38:12, Error                 MOUPG  CSetupManager::Execute(232): Result = 0xC1420127
2016-06-09 11:38:12, Error                 MOUPG  CSetupHost::Execute(371): Result = 0xC1420127

The mention of WIM mounting made one user in the Super User chatroom recall the WIMMount Registry entry. I removed the subkeys of this:

HLKM\SOFTWARE\Microsoft\WIMMount\Mounted Images

After I did that, setup ran successfully to completion! (I had also run dism /online /cleanup-image /restorehealth along the way, but I'm not sure that did anything relevant.) That laptop is now running Windows 10.

When the Windows System Image Manager fails catalog creation with 0xC1420127 or 0xC1420117

I am currently in the process of setting up an unattended installation of Windows 10. Therefore, I'm using the System Image Manager to create the Unattend.xml file, and to do that, I need a catalog file that corresponds to the WIM I'm deploying. The SIM can generate a catalog from a WIM, but in my case, it failed with codes 0xC1420127 or 0xC1420117, depending on what I tried. Eventually, I discovered that for the SIM to create a catalog, you need to satisfy three conditions:

• Use install.wim from the \sources folder on the original installation media. A captured WIM does not work because, evidently, it doesn't actually contain the stuff that goes into a catalog.
• Update your ADK to the version corresponding to the OS you want to deploy. I had previously deployed Windows 8.1, but there's a new version for Windows 10.
• Clear all subkeys from HKLM\SOFTWARE\Microsoft\WIMMount\Mounted Images that do not correspond to currently-mounted images.

Also, rebooting can't hurt.

How Candy Crush interferes with Windows deployment

One step in the Windows imaging and deployment process is to use Sysprep to generalize the machine (immediately before capturing the image). Today, however, I received an error saying:

Sysprep was not able to validate your Windows installation. Review the log file at %WINDIR%\System32\Sysprep\Panther\setupact.log for details. After resolving the issue, use Sysprep the validate your installation again.

I hear that this error can appear when the installation was upgraded rather than installed cleanly. That wasn't my situation though. Looking at setuperr.log in the folder mentioned by the error, I saw an app name clearly mentioned: Candy Crush. Evidently, Sysprep really doesn't like it when a modern app is installed for one user but not technically provisioned. After uninstalling Candy Crush from its context menu from the Start menu's programs list, I ran Sysprep again. It failed, this time mentioning Twitter. So I removed the Twitter app, and it worked.

So that's how Candy Crush interfered with my Windows deployment today. Also, I didn't know that Candy Crush or Twitter were installed by default in Windows 10.

FMod - v2.8.4

Before releasing yesterday's changes as Abiathar v2.8.4, I decided to move the link highlighting feature out of the Linker tool. It now is triggered by the semicolon key, irrespective of what tool is active. That makes more usability sense because people are likely to be in the middle of something else when they want to trace a link; making them switch tools would be a speed bump.

Once I did that and ironed out a little bug with the Tileinfo Overlay settings dialog, I put the new version in the auto-updater.

FMod - Tileinfo and link visibility control

I made two usability improvements to Abiathar today.

Previously, to show special properties like "deadly" or "door" in the tileinfo overlay, a setting had to be enabled in editor.aconf, which requires an Abiathar restart. Changing that one setting turned on the display of several properties and there was no way to pick and choose. I added a dialog box to select which special properties are shown. To the menu of choices, I added "switchable" tiles, which have an animation offset but a zero animation time. They get an S on them when their option is enabled.

I made sure to make that dialog easy to use with only the keyboard so that its options can be changed quickly. While there is a new item on the status bar when the Tile Properties overlay is on, the dialog can also be called with Ctrl+B. Its checkboxes can be togged with the corresponding letter, which also identifies the setting in the new editor.aconf entry.

I had also realized that dealing with levels that contain huge amounts of links (switches and doors) can get difficult when the lines are crossing everywhere; it gets hard to figure out what links where. Therefore, I added a way to highlight a link. With the Linker tool active, pressing Enter when the mouse is over a link source or destination will cause any links to or from there to turn pink instead of orange. Highlighted links always render last, so they're guaranteed to be visible on top of normal links.

StackReviewAttrib - Logging and resiliency

I did some more tests of StackReviewAttrib today. I discovered that the ASP.NET pages were in need of an Access-Control-Allow-Origin header; there had been JavaScript errors about violations of Cross Origin Resource Sharing. I also added a timeout to the part that waits for the web browser simulator thread to finish so that I don't end up with a boatload of stuck threads if something fails.

To keep an eye on the parsing (so that I don't have to wait until the session's end to see whether everything worked), I also added some logging to the page-adding routine that shows the progress of each page's addition in the Visual Studio console.

It looks like everything is ready for me to actually use the tool to start a video series.

Ongoing StackReviewAttrib work

It turns out I haven't completely finished StackReviewAttrib. Today I identified and dealt with a corner case involving the Suggested Edits review queue. On the review screen for edits proposed by anonymous users, a div element that my program looks for is missing. Therefore, I added a branch to check for that case and write an appropriate summary of the content.

When I tried to do a full test of the setup, I found that my Start and Stop buttons in the Chrome extension's pop-up don't seem to be working, though they do work at least some of the time. Also, the WebBrowser control is suddenly getting certificate-related security warnings that I cannot reproduce in any normal browser.

Finished the Stack Exchange review attribution helper

Yesterday, I wrote a good chunk of StackReviewAttrib, a tool to create a references/attribution list for reviewed posts at Stack Exchange. Today I worked out the bugs and wrote the Chrome extension.

The problem with the WebBrowser control was challenging to fix. The DocumentText property doesn't seem to update as JavaScript does its work. Instead, I had to use this:

Document.GetElementsByTagName("html")(0).OuterHtml

I also added a 1.5 second wait between the DocumentCompleted event firing and the accessing of that property to make sure that all script had time to finish. I then fixed some minor glitches in the final HTML output renderer and addressed some edge cases in the review items (like an answer being deleted).

The only hiccups in the construction of the Chrome extension were Chrome's security policies. I had to wire up all event handlers in the extension's icon popup via JavaScript without having any JavaScript literals in the HTML page.

The code for both components is now on GitHub.

Building a parser for the Stack Exchange review item screen

I've decided that it would be pretty neat to make YouTube videos of myself doing reviews on Stack Exchange, but in order to comply with attribution guidelines (since I'd be showing other people's work on screen), I need some way of producing a conveniently-browsable list of the posts I viewed and their authors, along with links to the involved user profiles.

Therefore, today I spent a bit of time throwing together an ASP.NET web site to be run on localhost that will be invoked by a Chrome extension for each review item. The .NET site uses the HTML Agility Pack to scrape the page for the relevant names and links. At the end of the session, it spits out a new HTML page that satisfies the attribution requirements.

I did run into a major snag. The review item UI is rendered dynamically with JavaScript, so the things I need aren't actually in the HTML when my program downloads the page. Therefore, I'm now working on rigging up a horror that involves the WebBrowser control.