Synopsis: The article is to tell how interrupts works in x86_64 and how interrupts are delivered in my hypervisor
Keywords: vt-x, interrupt delivery
Let's first introduce how interrupts are delivered in i386 and x86_64. per my understanding, the devices' interrupt pins are connected to PIC(programmable interrupt controller, e.g. Intel 8259) in legacy computers and APIC(advanced programmable interrupt controller) in recent computers. I found my DELL workstation support both. through configuration, The PIC or APIC is able to delivery an interrupt to the processor at proper moment(the vector is not masked and the processor's interrupt flag is clear). please see Chapter 10 of Intel Mannuals for more of APIC. when an interrupt is sent to processor, how does the processor react? well, it depends on the mode and previlege level the processor is working in. the table below is given to describe the actions taken under different circumstances: +----------------------------------------------------------------------------------------+ |mode |previlege Level| actions | +----------------------------------------------------------------------------------------+ | 32bit protected | |1.perform stack switch | | mode/64bit com- | |2.push original ESP/SS | | patible mode |PL 3 |3.push original EFLAGS/EIP/CS | | | |4.push error code if there is any | | +----------------------------------------------------------------------+ | | |1.DO NOT push original SS/ESP | | |PL 0 |2.push original EFLAGS/EIP/CS | | | |3.push error code if any | +----------------------------------------------------------------------------------------+ |64-bit long mode |PL0/PL3 |1.perform stack switch for inter-PL cases | | | |2.push RSP/SS unconditionally | | | |3.push RFLAGS/RIP/CS | | | |4.push error code if any | +-----------------+----------------------------------------------------------------------+ when the steps above are done, the processor transfers to the address specified in IDT(Interrupt Description Table) which usually has the length 256. each IDT entry contains the address with which the RIP register will be loaded. in order to save to restore processor state from interrupt context. additional work must be done before going further to other routines. Reference [1] is the work done for 32 protected mode and reference [2] is for 64-bit long mode. restoring processor state is in reverse order, it means we should restore the general purpose registers and then restore the registers which are automatically pushed during interrupt occurs, but the rest of work is done by issuing the one signle instruction 'iret'. the 'iret' instruction will restore the stack and privilege level according to what's onto the stack. Next I am going to talk about how I deliver the interrupt in my hypervisor. first is that I emulate an interrupt controller, like I said before, the Intel 8259 chip is the controller that x86 computers used to have. what I did that emulation inside my hypervisor. it's quite simple to have the minimum function set emulated with PIO channel: the offset of vector and the mask of the pin. you can find the details in file vm_monitor/device_8259pic.c. note with 8259(also with APIC), the end of interrupt must be acknowledged via guest by writing the command port. this is all about interrupt controller emulation. with VT-x, hypervisor is able to inject any interrupt vector at any time regardless of the IF of RFLAGS in vmx non-root mode, that's to say, the IF is cleared in guest OS and you inject the interrupt, the guest OS is to respond to the interrupt, but this is definitely not the right behavior. what the guest OS expects is when the global interrupt gate:IF is shut down, it never receives interrupts! Fortunately, vt-x provides the interrupt window mechanism: vt-x will let you know when is proper time to inject your interrupts for the guest. the bit 2 of primary processor-based vm-execution controls determines whether the interrupt window causes a vm exit: once set, a VM exit occurs at the beginning of any instruction if RFLAGS.IF = 1 and there are no other blocking of interrupts. it's absolutely safe to inject your interrupts. you should always disable interrupt window when there is no interrupts to deliver otherwise it keeps unnecessary trapping all the time. when we are sure to deliver an interrupt, we should also query the PIC interrupt pin mask to see whether it's masked. and also add the offset to the vector. to actually inject the interrupt, we should write the vm-entry interrupt-information field in vmx root mode, see more for the format at SDM 24.8.3. as the conslusion, I would say, the correct way to inject the interrupt is to wait until interrupt window is open and for external interrupt source, we also check the PIC interrupt pin mask to ensure the interrupt delivery is legitimate. Reference: 1: https://github.com/chillancezen/ZeldaOS/blob/master/x86/interrupt_entry.s 2: https://github.com/chillancezen/ZeldaOS.x86_64/blob/master/x86_64/interrupt_service_routine.S