libvirt, TianoCore EFI firmware, suspend-to-mem enabled leads to blank VM console in virt-manager

I don’t know yet if it’s really a bug or not but I’ve seen some weird behavior while trying to programmatically set up a VM in EFI mode with libvirt which involves a blank VM console unless I explicitly disable suspend-to-mem for the EFI firmware.

Consider the following minimal XML code to set up a libvirt VM with the TianoCore EFI firmware (instead of legacy BIOS), deliberately not defining any storage volumes or boot possibilities since all we care about here is whether we can see something on the VM console in virt-manager or not:


<domain type="kvm">
  <name>test</name>
  <vcpu>1</vcpu>
  <memory>1048576</memory>
  <os>
    <type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x86_64-smm-ms-code.bin</loader>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-model'/>
  <pm>
    <!-- uncomment next line and suddenly the console works -->
    <!--<suspend-to-mem enabled='no'/>-->
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <console type='pty'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
    </graphics>
    <video>
      <model type='qxl'/>
    </video>
  </devices>
</domain>

Save this to testvm.xml and run it through virsh -c qemu:///system define --file testvm.xml to define a new VM from it. Then, in virt-manager, start the VM. You will see: nothing. Just a black screen, no sign of the TianoCore EFI firmware that should appear.

The blank VM console
The blank VM console

Huh? Weird. I spent quite some time to figure out why this happens while a VM defined through virt-install would show a console, I thought maybe me SPICE setup was too minimal or broken but the solution wasn’t really in an area where I would have expected it. Shut down and delete the VM, uncomment the <suspend-to-mem> line in the XML code (shown in boldface above), save it and redefine the VM.


[...]
  <pm>
    <!-- uncomment next line and suddenly the console works -->
    <suspend-to-mem enabled='no'/>
  </pm>
[...]

And voila:

The working VM console
The working VM console

More like what we expected (the error message about missing boot medium is expected).

So I know that the default of suspend-to-mem appears to be the culprit but I still don’t know why. Any clues anyone? For reference, this happens to me on openSUSE 15.2 with Linux kernel 5.3.18, libvirt 6.0.0, qemu 4.2.1, virt-manager 2.2.1, ovmf 201911.

UPDATE: so it seems the problem is the combination of Secure Boot and 64-bit architecture. My interpretation is that the OVMF (Open Virtual Machine Framework, a port of Intel’s TianoCore to qemu) module for Secure Boot requires SMM (System Management mode), however S3 (Suspend-to-RAM) also requires SMM and coexistance of the two is only supported in 32-bit versions of the firmware (see the notes in the “Build OVMF” section of this Tianocore Github repo page. I think this is the reason why tools such as virt-install always disabled S3 (and also S4, Suspend-to-Disk, by the way).