When using Git from IntelliJ or Android Studio fails with "illegal character <:>"

I recently helped some students who were trying to work with Git from Android Studio but received "illegal character" errors mentioning index 5 and the colon character. This turned out to be from a mistake in installing Git and trying to connect it to Android Studio. Where the IDE looks for Git is controlled by the "path to Git executable" setting on the Git page of the Version Control section of the settings. Pasting a Git repository URL there (in an attempt to clone it) causes this error. Instead the path box can be left blank to make Android Studio try to automatically find the Git installation.