TL;DR

Android Studio has a bug in its AVD manager (Android Virtual Device - basically a virtual machine managed by Android Studio) that causes it to incorrectly report that it doesn’t have enough memory to start an AVD. The error code returned has a misleading name, in the code it is called “AccelerationErrorCode.NOT_ENOUGH_MEMORY”¹ but the error message produced by this references some Intel-only virtualization technology called “HAXM”² which is not used by default on Linux and is not related to the source of the error.

A binary patch in bsdiff format that disables the faulty memory check is provided below for Android Studio Canary 2021.03, as well as a Nix overlay to apply the patch. I also filed an upstream bug report here.

The Story

I recently wanted to use Android Studio on NixOS to do some Android emulation. Unfortunately, when I tried to create an AVD, I got a weird error, and it wouldn’t start:

I did some googling and I found out that the error is only produced in one place, the AVDConnectionManager class, in the checkAcceleration() function³:

854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902

/**
 * Run "emulator -accel-check" to check the status for emulator acceleration on this machine.
 * Return a {@link AccelerationErrorCode}.
 */
public AccelerationErrorCode checkAcceleration() {
  if (!initIfNecessary()) {
    return AccelerationErrorCode.UNKNOWN_ERROR;
  }
  Path emulatorBinary = getEmulatorBinary();
  if (emulatorBinary == null) {
    return AccelerationErrorCode.NO_EMULATOR_INSTALLED;
  }
  if (getMemorySize() < Storage.Unit.GiB.getNumberOfBytes()) {
    // TODO: The emulator -accel-check current does not check for the available memory, do it here instead:
    return AccelerationErrorCode.NOT_ENOUGH_MEMORY;
  }
  if (!hasQEMU2Installed()) {
    return AccelerationErrorCode.TOOLS_UPDATE_REQUIRED;
  }
  GeneralCommandLine commandLine = new GeneralCommandLine();
  Path checkBinary = getEmulatorCheckBinary();
  if (checkBinary != null) {
    commandLine.setExePath(checkBinary.toString());
    commandLine.addParameter("accel");
  }
  else {
    commandLine.setExePath(emulatorBinary.toString());
    commandLine.addParameter("-accel-check");
  }
  int exitValue;
  try {
    CapturingAnsiEscapesAwareProcessHandler process = new CapturingAnsiEscapesAwareProcessHandler(commandLine);
    ProcessOutput output = process.runProcess();
    exitValue = output.getExitCode();
  }
  catch (ExecutionException e) {
    exitValue = AccelerationErrorCode.UNKNOWN_ERROR.getErrorCode();
  }
  if (exitValue != 0) {
    return AccelerationErrorCode.fromExitCode(exitValue);
  }
  if (!hasPlatformToolsForQEMU2Installed()) {
    return AccelerationErrorCode.PLATFORM_TOOLS_UPDATE_ADVISED;
  }
  if (!hasSystemImagesForQEMU2Installed()) {
    return AccelerationErrorCode.SYSTEM_IMAGE_UPDATE_ADVISED;
  }
  return AccelerationErrorCode.ALREADY_INSTALLED;
}

The failing check in question

On closer inspection, it seems that the getMemorySize() codepath depends on JVM vendor, with a fallthrough case that returns 32GB⁴:

1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195

public static long getMemorySize() {
  if (ourMemorySize < 0) {
    ourMemorySize = checkMemorySize();
  }
  return ourMemorySize;
}

private static long checkMemorySize() {
  OperatingSystemMXBean osMXBean = ManagementFactory.getOperatingSystemMXBean();
  // This is specific to JDKs derived from Oracle JDK (including OpenJDK and Apple JDK among others).
  // Other then this, there's no standard way of getting memory size
  // without adding 3rd party libraries or using native code.
  try {
    Class<?> oracleSpecificMXBean = Class.forName("com.sun.management.OperatingSystemMXBean");
    Method getPhysicalMemorySizeMethod = oracleSpecificMXBean.getMethod("getTotalPhysicalMemorySize");
    Object result = getPhysicalMemorySizeMethod.invoke(osMXBean);
    if (result instanceof Number) {
      return ((Number)result).longValue();
    }
  }
  catch (ClassNotFoundException | NoSuchMethodException e) {
    // Unsupported JDK
  }
  catch (InvocationTargetException | IllegalAccessException e) {
    IJ_LOG.error(e); // Shouldn't happen (unsupported JDK?)
  }
  // Maximum memory allocatable to emulator - 32G. Only used if non-Oracle JRE.
  return 32L * Storage.Unit.GiB.getNumberOfBytes();
}

The offending function(s)

This seems to be where things are going wrong. Presumably, it’s not returning 32GiB, since otherwise the check wouldn’t trigger, but I have no idea why checkMemory() is returning a value smaller than 1GiB. In any case, just skipping this check should be enough to get Android Studio to launch AVD’s. Sounds simple - since Android Studio’s source code is online, it should be easy enough to patch, right? Well, no, apparently despite being technically “open source”, mere mortals cannot actually build Android Studio themselves⁵:

Which leaves only unconventional patch methods…

So I decided I would just patch the .jar file containing the AVDManagerConnection class, located in plugins/android/lib/android.jar. It shouldn’t be too hard to unpack it, poke around in the .class file and remove the offending checkMemorySize() check, and repack it. I first decided to use Recaf for this, which was great at disassembly and looking at the JVM bytecode. It has a decompiler mode as well as a JVM bytecode editor and a hex editor. You can even edit the decompiled code in-place, which is pretty cool.

Unfortunately, the assembler feature was not working that day, and despite some helpful suggestions from the Recaf discord, the modified .jar file didn’t work when I tried to run the original program:

Instead, I ended up just deciding to just overwrite the code with NOP instructions using a hex editor. The encoding of the NOP instruction is only one byte, so I can just replace the failing check with a series of NOP instructions without screwing up any offsets in the file. Since there is no repacking or recomputation involved in this, it should “just work” - with some caveats, namely that the JVM is a stack-based virtual machine so I need to make sure the stack is in the right state after the JVM executes my modified code. After looking at the analyzer in Recaf’s bytecode editor to decide exactly which instructions to rewrite, I decided on just overwriting the GETSTATIC and ARETURN operations starting on line 34 of the disassembled bytecode, since the stack is empty after those two instructions - no need to worry about the stack state.

Now I need to figure out where exactly in the binary to write my NOP instructions, since Recaf doesn’t display this information in the bytecode editor. Luckily, there is another cool program mentioned in the Recaf docs, called Kaitai https://ide.kaitai.io/, that will help with this. You just upload your file, pick a file format analyzer, and it will display the kinds of objects that are contained in your file and where exactly they are located within it. For us, we’re looking for the checkAcceleration() method, which Recaf tells us in the table view is index number 44 (Recaf’s table mode indexes from 1) in the methods table and that its code section begins at offset 0xC68D in the .class file.

From here, we can just read the bytecode byte-by-byte (I just used https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions as a reference) and look for the instructions we want to replace with NOPs. We can skip ahead bit by looking for an INVOKESTATIC, since it’s used right before the section we’re looking for. GETSTATIC is three bytes and ARETURN is one byte, so we end up zeroing out a grand total of four bytes. Once this is done, it will skip returning after the faulty checkAcceleration() check and everything will Just Work™.

For actually editing the bytes, I can’t rely on Recaf, since it recomputes some sections of the binary when you use the hex editor and I want to change as little as possible to minimize possible variables in case something goes wrong. Instead, I used Bless, a handy hex editor for Linux.

Finally, after repacking the .jar file, it just works!

It would have been nice if I could have just written a patch for the affected software and recompiled it myself, but when that’s not an option, I guess it’s nice to know how you can still proceed from there.

If you are affected by this bug in Android Studio, feel free to comment on my bug report: https://issuetracker.google.com/issues/229453055

Patching Instructions

This is a binary patch for a specific version of Android Studio. If you are using a different version of Android Studio where the file being patched is different in any way at all from what is expected, then it probably won’t work.

$INSTALL_DIR is wherever you installed it to. I use NixOS so it’s /nix/store/ydpc8sjgd93lznxippyvnc9dvz9crfdd-android-studio-canary-2021.3.1.7-unwrapped/ but yours will probably be different.

To apply the patch, you need to install bsdiff from your package manager or find a bspatch.exe binary somewhere if you use Windows (https://www.romhacking.net/utilities/929/ maybe? I haven’t tested this). Then, run in a console bspatch /path/to/original/android.jar /path/to/patched/android.jar /path/to/android_studio_haxmfix.bsdiff, and then replace your original android.jar with the patched version. Don’t try to patch it in-place.

NixOS

Here is an overlay that applies the bsdiff binary patch:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

{
self: super: let
  pkgs = import <nixpkgs> {};
  patchfile = pkgs.fetchurl {
    url = "https://ftp1.ornx.net/patch/android_studio_haxmfix/android_studio_haxmfix.bsdiff";
    sha256 = "de62c3618bb9475f9698e99ea3b6de07bde120e99e6c5f8a488797fff8de4131";
  };
  android_studio_unwrapped = super.androidStudioPackages.canary.unwrapped.overrideAttrs (prev: {
        name = prev.name + "-patched";
        buildInputs = [pkgs.bsdiff];
        patchPhase = ''
        ORIG=plugins/android/lib/android.jar
        TMP=$(mktemp -u $ORIG.XXXXXX);
        bspatch $ORIG $TMP ${patchfile};
        mv $TMP $ORIG
        '';
  });
  in {
    androidStudioPackages.canary = super.androidStudioPackages.canary.overrideAttrs (prev: {
      name = prev.name + "-patched";
      startScript = builtins.replaceStrings [super.androidStudioPackages.canary.unwrapped.outPath] [android_studio_unwrapped.outPath] prev.startScript;
      unwrapped = android_studio_unwrapped;
  });
};
}