Deprecation and introduction of interfaces

Specifications may appear to be static codifications set in stone, but they are often evolutionary species. One of the challenges in evolving a specification includes when, if ever, to deprecate content in lieu of new additions. Sometimes technology may fall out of use or be deemed by the market not to be the most viable. Often, though, this class of information is not elided from specifications unless they are hard scientific reasons, like elision of MD5 or SHA1 from specifications 

because of pre-image attacks https://en.wikipedia.org/wiki/Preimage_attack. I recall one person telling me that about 40% of the Wifi specification was relevant; the key was knowing 'which' 40% merited attention.

This blog will talk about a few specifications, such as the UEFI, PI and Intel SDM, that weigh in at several thousands of pages. To be with the UEFI and PI specification, one area that has potential for deprecation is Itanium support. The EDKII upstream has already removed Itanium related code from the various packages. There are vestiges of Itanium in chapter 2 of the UEFI Specification https://uefi.org/specs/UEFI/2.10/02_Overview.html#intelitanium-based-platforms for the calling conventions, though. Similarly, support for the Itanium reset paths https://uefi.org/specs/PI/1.8/V4_MCA_INIT_PMI_Protocol.html# and extended SAL https://uefi.org/specs/PI/1.8/V4_Sal.html services can be found in the PI specification. The latter of which was the EDKII adaptation of calling the Itanium System Abstraction Layer (SAL) (SAL) https://redirect.cs.umbc.edu/portal/help/architecture/24535901.pdf interfaces from a UEFI environment. 

As a quick background, Itanium had a platform scoped SAL and processor scoped Processor Abstraction Layer (PAL) set of firmware layers designed to provide both boot and runtime services. The RISC-V Supervisor Binary Interface (SBI) https://github.com/riscv-non-isa/riscv-sbi-doc is sort of an amalgam of SAL and PAL since it provides both core, SOC and (potential) platform capabilities. SAL is interesting in that unlike UEFI, that goes into a virtual-only calling mode after SetVirtualAddressMap(), the SAL calls could be called in either physical or virtual mode throughout the life of the platform. This posed some challenges for writing UEFI code since position independent code (PIC) options for C compilers haven't been universally supported across all of the EDK toolchains, especially in the early days with Visual Studio, or for IA32 with its inability to read the instruction pointer address as possible in other architectures. For assembly-language Itanium code it was pretty simple to write PIC code. So the Extended SAL (ESAL) of the PI spec and EDKII provided a way to have non-fixed up and fixed up C code that would use a common data area. 

In addition to the SAL support, another fascinating aspect of Itanium was the support for floating point exceptions in the Floating-Point Software Assist (FPSWA) https://redirect.cs.umbc.edu/portal/help/architecture/24541501.pdf driver, as described in https://www.amazon.com/IA-64-Linux-Kernel-Design-Implementation/dp/0130610143 

This driver was loaded from the EFI System Partition during boot and would provide runtime support for floating point exceptions. Regrettably soft-loading critical flows from disk like the FPSWA have not been pervasive, even in the face of relatively expensive $/byte of semiconductor NOR SPI flash. This stems from the supply chain challenge where the motherboard and fixed disk media may be provisioned, secured, and recovered by alternate parts of the ecosystem, namely OEM/ODM for the board versus OEM/ODM/integrator/VAR/IT for the disk and its bootloader and OS images, respectively.  

Another area that is an interesting artifact from the past is the BIOS interface in the Intel Software Developer Manual https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3a-part-1-manual.pdf. This interface is an Int15h API to manage microcode patches.

Starting in the early 1980s, the PC/AT BIOS exposed services through 16-bit 'int' calls or software traps, which the most famous being Int13h for disk access, Int 10h for video, etc. These API's all have correlatives in UEFI, with Int13h having the equivalent in EFI_BLOCK_IO_PROTOCOL, for example. The Intel Framework Compatibility Support Model (CSM) specification https://www.intel.com/content/dam/www/public/us/en/documents/reference-guides/efi-compatibility-support-module-specification-v098.pdf provided a bridge from EFI into BIOS calls in the early days where there were few EFI native drivers, for example. But the CSM support was not introduced into the UEFI PI specification since the idea with PI commencing in 2006 would have UEFI-spec-defined APIs. In fact, Intel declared the PC/AT BIOS interfaces to be end-of-life in 2020 https://www.phoronix.com/news/Intel-Legacy-BIOS-EOL-2020 

and http://www.uefi.org/sites/default/files/resources/Brian_Richardson_Intel_Final.pdf.

That's where the Int15h API mentioned above is interesting. It can either go away as its the only BIOS API in the SDM, or it could be complemented by/replaced with a UEFI equivalent. To that end, the https://raw.githubusercontent.com/tianocore-docs/Docs/master/White_Papers/A_Tour_Beyond_BIOS_Capsule_Update_and_Recovery_in_EDK_II.pdf design reads on this capability 

The mapping of the UEFI interfaces to the Int15h included

with the specific capsule itself having the following layout

The generic capsule overview flow is described in the UEFI specification https://uefi.org/specs/UEFI/2.10/08_Services_Runtime_Services.html#update-capsule 

and other write-ups https://embeddedcomputing.com/technology/security/software-security/understanding-uefi-firmware-update-and-its-vital-role-in-keeping-computing-systems-secure and https://archive.fosdem.org/2020/schedule/event/firmware_culisfu/attachments/slides/3709/export/events/attachments/firmware_culisfu/slides/3709/FOSDEM_2020_Intel_Capsule_Update.pdf. As the code-base has been re-arranged in the open, the most recent location to find the FMP DXE Microcode Capsule support is https://github.com/tianocore/edk2-platforms/tree/master/Silicon/Intel/IntelSiliconPkg/Feature/Capsule and of course https://github.com/tianocore/edk2/tree/master/SignedCapsulePkg. 

This builds upon the generic FMP 

and capsule update flow 

widely deployed today. Maybe avoiding putting something 'post-Int15h' in the SDM is a wise move, though, considering the plurality of interfaces for firmware updates, from https://slimbootloader.github.io/security/firmware-update.html to https://uefi.org/sites/default/files/resources/OCPsummit2016_Towards%20a%20Firmware%20Update%20Standard.pdf to https://uefi.org/sites/default/files/resources/PRM_Platform_Runtime_Mechanism_1_1_release_candidate.pdf to https://uefi.org/sites/default/files/resources/Intel_MM_OS_Interface_Spec_Rev100.pdf to.... It's a veritable embarrassment of riches.

Beware Experts, Beware Disruptors

This blog entry hearkens back to a tale of expertise versus disruption. The scene is DuPont, WA in the late 1990's. Prior to Tiano (2000+) there was an effort to write a BIOS from scratch in the erstwhile workstation product group (WPG) at Intel. The code name of the effort was "Kittyhawk" (1998). Like all boot firmware, the initialization was broken up into phases. Instead of the tuple of {SEC, PEI, DXE, BDS} of UEFI PI of {BOOTBLOCK, ROMSTAGE, RAMSTAGE, PAYLOAD) of coreboot, Kittyhawk had {VM0, VM1, VM2, BOOTLOAD} phases.

Although the platform initialization of Kittyhawk was in native code protected mode C code (and cross-compiled to Itanium for the latter boot variant), this was prior to emergence of IA-32 EFI and it's OS's, which was a parallel effort in DuPont. Instead, the 32-bit version of Kittyhawk relied upon a PC/AT BIOS boot interface for MBR-based operating systems. To make this happen an engineer decomposed an existing PC/AT BIOS from it's POST (Power-On Self Test) component from it's 'runtime', or the 16-bit code that provided the Int-callable BIOS API's. The 32-bit Kittyhawk code did device discovery and initialization, and then the 32-bit code provided the device information in a data block into the generic 16-bit BIOS 'runtime.' This 16-bit code was called 'the furball' by management in anticipation of a day when it could be 'coughed up' in lieu of a native 32-bit operating system load.

This Kittyhawk effort on IA-32 was definitely deemed disruptive at the time. The DuPont Intel site was more of a rebellious effort, with the larger product teams in Oregon constituting the existing 'experts'. There were cries from Oregon that the disruptors in Washington would never boot Windows, and if they did, they'd never pass WHQL, or the suite of compliance tests for Windows. The furball booted OS's and passed compliance tests. It turned out that having a non-BIOS engineer look at the problem didn't entail the prejudices of what's possible, and the work about having a clean interface into a BIOS runtime was used in the subsequent Tiano Framework work such as the Compatibility Support Module (CSM) csm https://www.intel.com/content/dam/www/public/us/en/documents/reference-guides/efi-compatibility-support-module-specification-v098.pdf API design.

So this part of the story entailed providing some caution in listening to the experts, but....

You sometimes need to beware disruptors. Specifically, riding high upon the success of building a common C code based to support IA-32 and Itanium, with the furball providing 32-bit OS support and the Intel Boot Initiative (IBI)/EFI 0.92 sample provide 64-bit Itanium OS support, the next challenge was scaling to other aspects of the ecosystem. Part of this scaling entailed support of code provided by other business entities. The EFI work at the time had the EFI images based upon PE/COFF, so the Kittyhawk team decided to decompose the statically linked Kittyhawk image into a set of Plug In Modules (PIM's).

After the Kittyhawk features were decomposed into PIM's, I remember standing in the cubicle of one of the Kittyhawk engineers and a BIOS guru visiting from Oregon helping with the 64-bit bring-up, the topic ranged over to option ROM's. The Kittyhawk engineer said "let's just put our PIM's into option ROM containers." I was a bit surprised, since to me the problem with option ROM's wasn't carving out code to run in the container but more entailed 'how' to have the option ROM's interoperate with the system BIOS. That latter point is where standards like EFI came into play by having a 'contract' between the Independent Hardware Vendors (IHV's) that built the adapter cards and the OEM's building the system board BIOS.

So the work of the disruptors was laudable in showing that you could have shared, common code to initialize a machine, but to scale to new paradigms like OS and OpROM interfaces you really needed a broader paradigm switch. And as would be shown, it took another decade to solve this interoperability issue.

As such, sometimes you have to be a bit wary of the disruptors, although the disruptive Kittyhawk did provide many learning's beyond the furball/CSM concept, including how up to today EDKII builds its ACPI tables via extracting data tables from C code files. Alas the Kittyhawkfort was shut down as part of the shuttling of the Workstation Group, and efforts to build a native code BIOS solution fell into the hands of the EFI team as part of the emergent (1999+) Tiano effort. At this time there was the EFI sample that eventually grew into the EFI Development Kit (aka the first Tiano implementation), now referred to as EDKI versus today's EDKII. EDK leveraged a lot of the learning's of EFI to inform the Intel Framework specifications, including the phases of SEC, PEI, DXE, and BDS we know today.

This original PEI phase differed somewhat from the PEI you can find in the UEFI PI specification, though. Instead of the concept of C-based PEIM's and PEI core, the original instantiation of the PEI Core Interface Specification (PEI-CIS) was based upon processor registers. There was still the business requirement of separate binary executables and the solution of GUID's for naming interfaces was used. But instead of a PPI database, the PEIM's exposed exported interfaces through a Produced GUID List (PGL) and imported interfaces via a Consumed GUID List (CGL). During firmware construction and update the CGL and PGL were reconciled and linked together. And in order to having services that interoperate, the concept of a 'call level' was introduced. The call level detailed which registers could be used by services in a PEIM since there was no memory for a call stack with this early PEI-CIS 0.3.

At the same time we were defining PEI 0.3, there was another approach to writing pre-DRAM code without memory, namely the Intel® Applied Computing System Firmware Library V1.0 (hereafter referred to as Intel® ACSF Library).

ACSFL was described in a Intel Update article (same dev update magazine https://github.com/vincentjzimmer/Documents/blob/master/it01043.pdf)

that provided 32-bit callable libraries for boot-loaders. This effort came from the embedded products team and entailed delivering a series of libraries built as .a files. This simplified initialization code addressed the lack of a memory call stack by using the MMX registers to emulate a call stack. This differed from the PEI 0.3 model of reserving a set of registers for each call level. The ACSFL concept was smarter in that constraining PEIM's to a given call level really impacted the composition of these modules into different platform builds.

I do enjoy the quotation 'requires only 15KB of flash' when I wander over to https://github.com/IntelFsp/FSP/blob/master/KabylakeFspBinPkg/Fsp.fd with its size of 592k, or 39x size dilation of silicon initialization code over the last 20 years. This is similar to the 29x scaling in the number of transistors between the 7.5 million in a Pentium II https://en.wikipedia.org/wiki/Pentium_II and the 217 million in a Kaby Lake https://www.quora.com/How-many-transistors-are-in-i3-i5-and-i7-processors.

The same article provides the software layering. One can see some similarity of ACSF Library to the Intel Firmware Support Package (FSP). This is no accident since the Intel FSP was intended to originally target the same embedded market, although Intel FSP in 2012 had the embedded mantle being carried by the Intel of Things Group. As another point of irony, the original FSP built w/ the Consumer Electronics Firmware Development Kit (CEFDK). The CEFDK was in fact the evolution of the ACSF library, going through various instances, like FDK. The latter were used to enable phones and tablets beyond just embedded.

So ACSF Library provided some learning's, and at the same time the LinuxBIOS (prior name of coreboot) solved this issue of supporting C code without initialized DRAM via the ROMCC tool.  ROMCC was a custom compiler that used processor registers to emulate a stack so that you could write things like DRAM initialization in C code.

The initial implementations of PEI-CIS 0.3 https://www.intel.co.kr/content/dam/www/public/us/en/documents/reference-guides/efi-pei-cis-v09.pdf with just registers proved difficult to deploy, and since part of the Tiano effort entailed a requirement to use standard tools, techniques like ROMCC were deemed not tenable. As such, as the PEI CIS went from 0.3 to 0.5, we moved PEI to the concept of temporary memory, with using the processor cache as RAM (CAR) as the 'temp ram.' Regrettably we were asked to not disclose how we implemented temp RAM, and the technique and initialization code were not made open source or published (and a modern instance of this reluctance can be found in the FSP-T abstraction of FSP 2.0). The coreboot community, though, didn't have this constraint and https://www.coreboot.org/images/6/6c/LBCar.pdf https://www.coreboot.org/data/yhlu/cache_as_ram_lb_09142006.pdf provided details on how to use this technique.

As time progresses, I am amused about the various recollections. During the late 2000's someone told me 'you rewrote PEI many times' when in fact the only substantive change was going from registers in 0.3 to temporary memory in 0.5. Although not necessarily a full re-write, the EFI implementations did have a long history:

IBI Sample -> EFI Sample -> Tiano Release 1..8, Tiano Release 8.1..8.7,  EDK1117, ...

...UDK2015, UDK2016, .......

Also, some fellow travelers mentioned to me their fond recollections of EFI discussions in 1996. I smile because the first discussions of EFI (in the form of IBI) weren't until 1998. But I suspect everyone's memory gets hazy over time, with this blog post having its own degree of fog and haze.

And these efforts also remind me that changes in this space take time. A recent reminder that api support takes time, such as the discussion of the EFI Random Number protocol in Linux
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.5-EFI-RNG-x86. This is an API that was defined in 2010 by Microsoft for Windows 8 and then introduced in the UEFI standard in 2013. Or features like UEFI secure boot from 2011 UEFI 2.3.1 showing up in Debian Buster https://wiki.debian.org/SecureBoot just in mid 2019.

The other interesting perspective is https://fs.blog/2019/12/survivorship-bias/, namely you often hear about the success stories, not failures. Only for extreme cases like https://en.wikipedia.org/wiki/Normal_Accidents do you find a rich journal of failures. So in portions of this blog and the EFI1.2 discussion in http://vzimmer.blogspot.com/2013/02/anniversary-daynext-arch-ps-and-some.html I have tried to shed light upon some of the not so successful paths.

This one is for 20, or Anniversary.Next^5

As I reach my 20 year anniversary with Intel today, I reflect upon advice that resonates with me. I especially like the posting http://perspectives.mvdirona.com/2016/11/advice-to-someone-just-entering-our-industry/, including the admonition to "Play the Long Game."

20 years. And all doing firmware. Several different firmware architectures and many instances of EFI-style firmware (e.g., Release 1-Release 8.1/2/3/4/5/6, Release 9 "aka EDKII").....

Hopefully this won't encourage me to abuse logical fallacies like argument from authority, saying 'In my 20 years at Intel we.....' Instead you're only as good as the last game you've played, not your record of games.

Or having a Whiggish view of tech history. Instead it's more Kolmogorov probability that monotonically increasing (or decreasing) progress and determinism.

Speaking of history, my original badge from February 24, 1997 can be found below, with the drop-e logo and, gasp, a suit and tie.

And now

Ah, the thick head of hair that I had in the 90's. And my Harry Potter glasses. I recall visiting Shanghai and Suzhou in '01. In the latter city the locals pointed at me in those crazy glasses and a scratch on my forehead from my two year old daughter (that resembled the lightning bolt), reinforcing the Potter doppelganger experience. Pre-SARs in Shanghai, so it was still possible to eat snake, drunken shrimp, and dining colleague from the south China province whose restaurant jaunt truly lived up to the saying "... the Chinese eat anything with four legs except a table, and anything that flies that isn't an airplane..." http://simonlesser.blogspot.com/2009/06/food.html.

So my journey at Intel started in 1996 after contact from an Intel recruiter while I lived in Houston,TX. He exhorted me to join Intel, especially given the 'imminent' Merced CPU development. I interviewed in Hillsboro, OR in October 1996 and was told that I could go to Oregon for IA32 Xeon, or DuPont, WA for IA-64 Merced. Having grown up in Houston, Texas and not realizing that the Pacific Northwest even existed prior to this conversation, I naturally chose DuPont in order to be part of the 64-bit revolution.

Fast forward to February 1997. My wife and I moved to Olympia, WA. Given some of the, er, delay in Merced, I had the opportunity to pick up a Masters at the University of Washington

and at the same time work on developing getting our Itanium firmware ready. This included working on the System Abstraction Layer (SAL) http://www.intel.com/content/www/us/en/processors/itanium/itanium-system-abstraction-layer-specification.html with my BIOS hero/guru Sham D. in Hillsboro, along with Mani and Suresh in Santa Clara. The original boot flow entailed SAL-A for memory initialization, SAL-B for platform initialization and the "SAL_PROC" for the OS-visible API's to enable boot-loaders. The loader API into the firmware was a direct mapping of the PC/AT BIOS calls, with examples including instances like SAL_PROC 0x13 having a similar command set to int13h https://en.wikipedia.org/wiki/BIOS_interrupt_call.

As an arbitrary pedantic sidebar, you definitely see a pattern in firmware for 'phases' that typically include 'turn on memory,' 'turn on platform', and 'provide the boot loader environment.' Itanium had SAL-A, SAL-B, EFI. UEFI PI has SEC, PEI, DXE, BDS/TSL/UEFI API's. coreboot has bootblock, rom stage, ram stage, payload (including Seabios or UEFI or Depthcharge or ...). Power8 has hostboot, skiboot, and Petitboot (or EDKII UEFI). The workstation BIOS for IA-32 below had VM0, VM1, VM2, Furball. PC/AT BIOS has bootblock, POST, BIOS runtime. You see a pattern here?

Writing SAL_PROC code was pretty exciting. It could be invoked in virtual or physical mode. With hand-crafted Itanium assembly it was pretty reasonable to write position independent code (PIC) and use the GP register to discern where to find global data. But in moving to C, writing portable C code to abstract the SAL services was quite a feat. This is distinct from the UEFI runtime where were are callable in 1:1 mapping and then non-1:1 after the invocation of the SetVirtualAddress call by the OS kernel.

Regarding gaps with SAL_PROC as a boot firmware interface, as chronicled in page 8 of http://www.intel.com/content/dam/www/public/us/en/documents/research/2011-vol15-iss-1-intel-technology-journal.pdf, Intel created the Intel Boot Initiative (IBI) as a C-callable alternative. The original IBI specification looked a lot like ARC http://www.netbsd.org/docs/Hardware/Machines/ARC/riscspec.pdf. Ken R., a recent join to Intel from an MS (where he had a lot of DNA for ACPI), helped turn IBI into what we know as EFI 1.02, namely evolving IBI to have discoverable interfaces like protocols (think COM IUnknown::QueryInterface) and Task Priority Levels (think NT IRQLs), and of course the Camelcase coding style and use of CONTAINING_RECORD macro for information hiding of private data in our public protocol interface C structures. Many thanks to Ken.

Building out EFI was definitely evolutionary. It started from the 'top down' with EFI acting as that final phase/payload in the first instances with alternative platform initialization instances underneath. This view even informed the thema of 'booting from the top down' that informed how we sequenced the chapters in the 2006 Beyond BIOS book, for example. The initial usage of EFI was the 'sample implementation' built on top of the reference SAL code and a PC/AT BIOS invoked by the EFI 'thunk' drivers.

As we moved into the 2000's, the Intel Framework Specifications were defined in order to replace the SAL for Itanium and PC/AT BIOS for Itanium and IA-32, respectively. We internally referred to things like SAL + BIOS + EFI Sample as a "Franken-BIOS." The associated code base moved from the EFI Sample to the EFI Developer Kit, or EDKI, to distinguish it from the EDKII done in the later 2000's. This internal code-base was called 'Tiano', thus the name of community sites like http://www.tianocore.org.  Someone said the name came from the sailor with Columbus who first noticed America, but the only citation I could find publicly is the "Taino" tribe with whom Columbus engaged.

As a funny sidebar, I do recall the meeting where someone found "Tiano Island" http://pf.geoview.info/tiano,4033365 on the web. At the time it cost some number of millions of dollars. The original director of our team, numbering just a few engineers in the room, said 'let's each pool a couple percent of our stock options and buy the island.' I guess Stu had a much more significant equity position than I did, as a lowly grade 7 engineer.

In late 90's at DuPont, SAL and EFI sample were not the only code base activities. While in DuPont the erstwhile workstation group also created a clean-room replacement for the early boot flow. This started on IA32 and the OS interface was the PC/AT BIOS. For this effort we didn't have an image loader and instead just used non-1:1 GDT settings in order to run the protected mode code. For booting the protected mode code provisioned the 16-bit BIOS blob with information like the disk parameters, etc, so that the 16-bit code was just the 'runtime interface.' The 16-bit BIOS blob was called the 'furball' since we hoped to 'cough it up' once the industry had transitioned into a modern bootload erenvironment, such as EFI.

I still recall colleagues in the traditional business units yelling 'you'll never pass WHQL' with the above solution, but it did work. In fact, the work informed the subsequent interfaces and development with the Intel Framework Compatibility Support Module http://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-compatibility-support-module-specification-v096.html.

We then ported the workstation code to boot the first Itanium workstation. I left that effort and joined the full EFI effort afterward. I recall the specific event which precipitated the decision. I was chatting with Sham and the workstation BIOS lead in the latter's cube. The lead said 'Now that we have our BIOS in modular code code "Plug-In Modules" (PIM's) we can tackle the option ROM problem. I thought to myself that just refactoring code into separate entities isn't the challenge in moving from PC/AT 16-bit option ROM's into a native format, it's all about the 'interfaces, namely how would a 'new' option ROM snap into a modern firmware infrastructure. IBI (now called EFI) was on that path to a solution, whereas a chunk of 'yet another codebase with PIM's' wasn't. Thus I was off to chatting with my friend Andrew, then lunch with Mark D, and onto the EFI quest in 1999. Quite the firmware long-game.

Next we're of finishing the first EFI, going from IBI to EFI .98 to EFI 1.02.

Next we're off on a cross-divisional team to create the '20 year BIOS replacement' called Tiano and the Intel Framework Specifications are born.

Next we solve the option ROM and driver problem with EFI 1.10.  Along the way between 1.10 and UEFI 2.0 we incubate a lot of future technology with the never release 'EFI 1.20' work.

Next Andrew Fish and I ported EDK to Intel 64. And I had fun with a port to XScale back in 2001. I have always enjoyed firmware bring-up on new CPU's.

Fast forward to 2005. The EFI specification became the UEFI 2.0 specification, and many of the Intel Framework Specifications became the UEFI Platform Initialization specification. Wrote the first EFI interface and platform spec for TPM measured boot https://github.com/vincentjzimmer/Documents/blob/master/EFIS004Fall06.pdf.

Fast forward to the 2010's.  More open source. More device types. More CPU ports. Continue to evolve network booting, such as IPV6 https://tools.ietf.org/rfc/rfc5970.txt and HTTP https://github.com/vincentjzimmer/Documents/blob/master/UEFI-Recovery-Options-002-1.pdf. Good stuff. Helped deliver UEFI Secure Boot https://github.com/vincentjzimmer/Documents/blob/master/SAM4542.pdf https://github.com/vincentjzimmer/Documents/blob/master/UEFI-Networking-and-Pre-OS-Security.pdf.

In parallel, I often had side firmware engagements, including a fun tour of duty helping our the solid state disk (SSD) team on firmware.

I still believe in better living through tools, too, whether they have landed in the community http://www.uefi.org/sites/default/files/resources/2014_UEFI_Plugfest_04_Intel.pdf, almost made it https://github.com/vincentjzimmer/Documents/blob/master/Vij_KRHZRWL_13.pdf https://github.com/termite2/Termite, or are in incubation https://www.usenix.org/system/files/conference/woot15/woot15-paper-bazhaniuk.pdf.

Fast forward to 2017. Year 20. It's still a lot of fun solving crossword puzzles with hardware and firmware.

During my time at Intel I've also appreciated the wisdom of others, whether through the mentoring of direct interaction or the written word. For the latter I heartily recommend keeping the following close at hand.

So am I done this morning? Let's do a final rewind to February 1992 when I jumped into industry in Houston. First I wrote firmware for embedded systems attached to natural gas pipelines http://www.emerson.com/resource/blob/133882/bb9c3232256dfab98cc6a20a27d43c1f/document-3-9000-309-data.pdf - sensors, serial protocols with radio interfaces to SCADA host, control algorithms, I2c pluggable expansion cards, loaders in microcontroller mask ROM's, porting a lot of evil assembly to C code...  fun stuff. The flow computer/Remote Telemetry Unit (RTU) work was an instance of the Internet of Things before the IOT was invented. Then on to industrial PC BIOS and management controller firmware. Then on to hardware RAID controllers and server BIOS. And then Intel in February 1997. 5 years of excitement in Houston prior to my Intel journey.

So I guess that sums out to 25. Now I feel tired. Time to stop blogging and playing the rewinding history game. Here's looking to the next 25.

Cheers.

What is 'Compatibility'?

So why another blog just a week after the last one?   Given the aperiodic nature of my earlier blogs, this frequency seems a bit out of character.  Well, I finished reading Fowler's Passionate Programmer and he exhorts the reader to practice, practice, practice.  Not just code, but writing.  And one venue for writing is a web log.  No referees, reviewers, publisher, etc between my typing and publication.  So here you go, for better or worse.

Now back to the topic of this blog.

There is often talk of 'compatibility' in the computer industry.  From the perspective of the platform firmware, we define compatibility as 'the software you choose to run.' In this taxonomy, compatibility is not some absolute term or Platonic truth living outside the world but instead very much a contextual term.

So this is all well and good, but how does this relate to the platform firmware specifically?  In the case of the boot firmware, we think of compatibility in terms of the PC/AT BIOS and its interfaces.  These interfaces include the 16-bit int-callable interfaces published by the system board BIOS and BIOS extensions for use by booted software.  The 'booted software' can include the MBR first stage loader and subsequent system software. The BIOS industry referred to these int-callable interfaces, such as int10h for video, int13h for disk, int16h for keyboard, along with data structures like the BIOS Data Area (BDA) and Extended BIOS Data Area (EBDA) as the "BIOS Runtime."  This BIOS Runtime is distinct from the code that executes after a platform restart and the germination of the BIOS runtime interfaces.  This phase of execution has been historically lumped under the term Power-On Self Test (POST).

Back in the early 1999 when the Extensible Firmware Interface (EFI) was first being deployed, this phase of execution, or the Boot Services (BS) and subsequent Run Time (RT), were really akin to the BIOS Runtime.   This parallel is borne of the fact that the EFI specification, nor its successor the UEFI specification, dictates 'how' these environments appear.  The pre-EFI/UEFI phase is opaque to and distinct from the UEFI BS and RT, just as POST is distinct from the BIOS Runtime and its int-calls.

In the case of EFI, the Intel Platform Innovation Framework (aka "Framework") and its successor, the UEFI Platform Initialization (PI) specifications, defined a stylized POST phase.  This includes the SEC, PEI, and DXE phases of execution.  But just as EFI is distinct from its preceding phase, the PI phase does not necessarily have to create a full UEFI environment.   And it is this PI versus UEFI where compatibility comes into play.   You see, as EFI and UEFI worked to gain acceptance, the bulk of platforms in the market since 1982 supported PC/AT BIOS and 16-bit BIOS option ROMs and OS loaders.

To leverage that large class of 'legacy' BIOS infrastructure, the Compatibility Support Module (CSM) was borne.  The CSM was developed by a project that preceded the "Tiano" program but was codified into a Framework Specification http://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-compatibility-support-module-specification-v097.html.  This earlier program referred to the compatibility code as the 'fur ball' with the hope that one day a native mode firmware stack would prevail and 'cough up' the compatibility codes.

The CSM provides a way to encapsulate a BIOS runtime inside a Framework or PI-style firmware.  The CSM specification abstracts the information collected during a PI-style POST, namely SEC, PEI, and DXE, and passes that information into the CSM wrapped version of a BIOS 16-bit runtime.  We often refer to this binary in our edk2-based trees as a CSM16 binary.

Recall that UEFI provides services for blocks such as the EFI Block I/O protocol, and the BIOS supports similar capabilities via the int13h API.  To leverage an underlying 3rd party storage option ROM's 16-bit disk services, a UEFI driver can be generically written to thunk or call down into the 16-bit code.  The thunk is provided as a general utility via a API member of the CSM protocol.  There is also a reverse thunk, but it is rarely if ever used.  You see, a CSM16 cannot practically call back into native UEFI 32-bit or 64-bit code because when control is passed to a 16-bit MBR loader, the int15h E820h memory map that the legacy OS receives has the UEFI boot service pages mapped as available.  And since there is no equivalent of ExitBootServices() for a legacy OS boot, UEFI services in the form of boot services drivers cannot reliably be invoked via a reverse thunk after the int19h IPL.

EFI/UEFI calling down into PC/AT BIOS CSM16, on the other hand, can be done throughout the boot services phase since the CSM16 services are available until ExitBootServices, so a BiosBlockIo or BiosVideo driver can reliably thunk to the int13h and int10h CSM16, respectively.   The only way to have 32-bit or 64-bit native codes that are shareable between CSM16 and a DXE-based UEFI BS is to put the codes into something like a DXE SMM driver.   The downside of the latter is that 3rd party UEFI drivers cannot be loaded into SMRAM.  This only works for system board hardware, such as USB ports in the chipset.

And now even within EFI/UEFI there are compatibility concerns.  EFI1.02 to EFI1.10 introduced OpenProtocol, and implementations of the EFI1.02 HandleProtocol had to use the later reference-counted API to implement the earlier non-reference counted API.  Then UEFI2.0 in 2006 deprecated UGA and DeviceIo.  From UEFI2.0 to 2.1, then 2.2, 2.3, 2.3.1 have introduced new services.  Framework HII to UEFI HII.  A style of programming to maintain compatibility often includes reading the specification version in the system table in order to take the appropriate code path.  The same holds true for PI.  PI became with the donated Intel Framework specifications, and from Framework 0.9x to PI 1.0, 1.1, 1.2, 1.2.1, 1.2.1a, etc there have been similar concerns with SEC, PEI, DXE, SMM.

So that's my blog for today on compatibility.