Saturday, December 26, 2020

operating system vendors and firmware

The PC industry has followed an interesting arc, beginning with the hardware and firmware details of the original IBM PC http://bitsavers.org/pdf/ibm/pc/pc/6025008_PC_Technical_Reference_Aug81.pdf and the closed source Microsoft (MS) operating system MS/DOS https://github.com/microsoft/MS-DOS. The former allowed for the ability to have clones of the IBM PC, whereas the latter ensured some consistency in the platform design as compatibility with DOS, and then Windows, helped provide for the open platform that non-MS OS's like Linux today enjoy. 

Was the distinction between MS as an OS vendor (OSV) and the industry as a platform provider always so stark? In fact, the existence of MS-written firmware, as one example, has had various examples spanning the 90's up through today's news. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To begin:

In the early days of Windows NT, including 3.51 and 4.0, Microsoft wrote the ARC boot firmware for their DEC Alpha and MIPS NT platforms. ARC stands for Advanced RISC Computing https://en.wikipedia.org/wiki/Advanced_RISC_Computing and the document http://www.netbsd.org/docs/Hardware/Machines/ARC/riscspec.pdf provided guidance on both platform hardware and firmware. A couple of MS engineers wrote the first ARC firmware and the ARC specification, along with IEEE 1275 Open Firmware (OF) https://www.openfirmware.info/data/docs/of1275.pdf, were considered as a solution for 'how to boot Itanium' during the early days of the IBI/EFI specification. 1275 came up because the PowerPC port of NT booted via OF. EFI ended up looking more like ARC, see similarity between the ARC Firmware Function Vector and UEFI System Table, although OF does have some advantages, especially for security https://www.cs.cornell.edu/~kozen/Papers/acsac.pdf

Another interesting aspect of the ARC Alpha firmware was that the NT NDIS miniport drivers from NT were embedded in the firmware for purposes of pre-OS networking, such as network booting. Other than boot loaders like U-Boot which liberally leverages portions of device driver code, this OS and firmware code sharing helps solve one of the main challenges of boot firmware, namely having to create a firmware version of a device driver for the block, network, and console services in the pre-OS and another variant for the OS runtime. 

 |

 |

\/

Fast forward to the early 2000's. At that time there was a project called FlexGo    https://en.wikipedia.org/wiki/Microsoft_FlexGo which allowed for a metered, or subscription PC.  There was MS firmware integrated into the code morpher, or microcode-like firmware of the Transmeta device https://en.wikipedia.org/wiki/Transmeta https://www.extremetech.com/extreme/76602-amd-to-resell-transmetas-payasyougo-chip. For some of the standard PC's at the time, there was exploration of having the monitoring agent from MS as an additional handler in System Management Mode (SMM) of the BIOS

 |

 |

\/

In the early 2010's, the industry was moving from a TPM 1.2 to 2.0. One of the learning's from the 1.2 era was that the specification for the commands lent themselves to various interpretations. As such, the TPM 2.0 specification was written in 'literate code' where the C based implementation of the commands could be extracted. The latter C code has formed the underlying implementation   https://github.com/microsoft/ms-tpm-20-ref for all of the integrated and discrete 2.0 devices. This code was born of the 'firmware TPM' work described in the technical report https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/msr-tr-2015-84.pdf

 |

 |

\/

In the 20-teens, there emerged from Microsoft Project Mu for BIOS https://microsoft.github.io/mu/. Although this is officially referred to as a downstream fork of EDKII on tianocore.org, there are many unique aspects, especially features like DFCI in https://github.com/microsoft/mu_plus.

A more recent example includes MS writing the SMM supervisor for AMD SecureCore PC https://www.microsoft.com/security/blog/2020/11/12/system-management-mode-deep-dive-how-smm-isolation-hardens-the-platform/ variant mentioned during OSFC '20 'hallway chat.'

And for the broader industry MS wrote many SMM audit and checking tools in Mu to help prepare the OEM ecosystem for having their handlers running in the jailed context of the SecureCore PC SMm supervisor.

 |

 |

\/

In the future, MS hardware Pluton hardware and firmware https://www.microsoft.com/security/blog/2020/11/17/meet-the-microsoft-pluton-processor-the-security-chip-designed-for-the-future-of-windows-pcs/ may be even more deeply integrated.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Beyond the long run of the MS examples above you can see a similar, albeit shorter in time, arc for Google, especially Google as an OSV with respect to ChromeOS. It begins with the Google Chromebooks leveraging coreboot + vboot  https://www.coreboot.org/ where Google employees are many of the upstream maintainers. 

 |

 |

\/

More recently, the Titan/OpenTitan  https://opentitan.org/ has emerged. This is a root of trust with the CPU based upon RISC-V and the operating system based upon the Tock OS https://github.com/google/tock-on-titan written in Rust. Interesting stuff.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

And you cannot have firmware without hardware, of course. Google Titan-M and the OpenTitan are one example.

For Microsoft, MS already has its XBox360 https://en.wikipedia.org/wiki/Xbox_360_technical_specifications and XBox One https://en.wikipedia.org/wiki/Xbox_One, which were custom PowerPC "Watermoose" and AMD "Jaguar" based SOC's, respectively.

There are also always rumors in the air for MS, such as recent one on ARM https://arstechnica.com/gadgets/2020/12/microsoft-may-be-developing-its-own-in-house-arm-cpu-designs/ and earlier one on custom designs like E2 https://www.theregister.com/2018/06/18/microsoft_e2_edge_windows_10/.

For Google we already have ample public details on their Tensor Processing Unit (TPU) https://semiengineering.com/knowledge_centers/integrated-circuit/ic-types/processors/tensor-processing-unit-tpu/ and the above-listed Open Titan first party silicon, but these are not general purpose system on a chip (SOC). Of course even Google has the rumor mill swirling on occasion with stories like "Whitechapel" https://www.theverge.com/2020/4/14/21221062/google-processor-pixels-chromebooks-whitechapel-samsung-qualcomm, too.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So how does the OS impact the firmware? Well, since the firmware is closely tied to the overall platform and hardware design, OS 'requirements' documents http://download.microsoft.com/download/7/0/e/70e74967-b0fe-477a-974f-c1ed16ee31df/windows8-1-hardware-cert-requirements-system.pdf and https://source.android.com/compatibility/10/android-10-cdd can dictate some of these choices. And even for more open firmware implementations like Chromebooks you can see the coupling https://www.chromium.org/chromium-os/2014-firmware-summit.



Monday, December 14, 2020

musings about firmware cultures

In a quick journey around firmware cultures in this posting, I'll talk a bit about the recent Open Source Firmware Conference (OSFC) 2020.

The Open Source Firmware Conference (OSFC) started in 2018. It was a broadening of the coreboot conference to include other open source firmware projects. Some examples of the earlier coreboot conference include the 2016 https://www.coreboot.org/Coreboot_conference_San_Francisco_2016 where Intel talked about the ‘then recent’ UEFI Payload project https://youtu.be/I08NHJLu6Us and Intel FSP 2.0 https://youtu.be/uzfiTiP9dEM. I was invited to give the keynote of the inaugural OSFC conference in 2018 https://2018.osfc.io/uploads/talk/paper/1/OSFC_Keynote-005.pdf  There common themes such as chipsec, Slim Bootloader, Min Platform, coreboot, and Intel FSP were noted.

 

Now have FSP SDK   https://github.com/universalpayload/fspsdk/tree/qemu_fsp_x64 mentioned in that keynote, too.

Fast forward to OSFC 2020 https://cfp.osfc.io/osfc2020/schedule/ and you see many of the same sentiments being reiterated. Intel talked about extending Intel FSP for programmable service engine support https://cfp.osfc.io/osfc2020/talk/TNTFYV/, a more modern configuration mechanism https://cfp.osfc.io/osfc2020/talk/AS7EZR/, and the efforts https://cfp.osfc.io/osfc2020/talk/VUNDSC/ to have a more reusable payload  https://github.com/universalpayload. From those talks we then go to the efforts to remove dependencies upon SMM, including the Platform Resource Monitor (PRM) https://cfp.osfc.io/osfc2020/talk/MCJASB/

Beyond those talks, Jiewen Yao co-presented on the Security Protocol and Data Model (SPDM) https://cfp.osfc.io/osfc2020/talk/ECQ88N/ along with Xiaoyu Ruan. SPDM is a new standard from the DMTF for device and host firmware security that is critical for upcoming security initiative support. openspdm is a sample implementation of SPDM specification. It will be used in multiple device and host firmware implementations including UEFI EDK II and possibly other platform firmware, such as a Baseboard Management Controller (BMC) based upon OpenBmc, etc.

Beyond SPDM, Jiewen also shared the background and efforts with Virtual Firmware for Intel Trust Domain Extensions (TDX) https://cfp.osfc.io/osfc2020/talk/CRKZB8/. This effort entails open source efforts to help scale enabling for TDX and provides a venue to discuss aligning enabling with other confidential computing efforts like AMD Secure Encrypted Virtualization (SEV). The TdShim is also used as a foundation for any service Trust Domain (TD) for TDX advanced feature in an EFI-light environment, such as the virtual firmware for a container or virtual TPM services. It bridges the gap between TD startup and applications running, and it enables the customers building their own use cases on top Intel TDX.

Andy Jassy during re:invent this year also spoke about how change in large companies is often driven by outsiders since long-time employees are often reluctant to replace what they've built in the past. And in the spirit of change, Rust was a topic of a few https://osfc.io/ talks this year, including the virtual hallway track discussion.

Specifically Jiewen Yao and I presented on Enabling Rust in UEFI firmware https://cfp.osfc.io/osfc2020/talk/SLFJTN/. This is a complementary talk to those by Google on enabling Rust in oreboot https://cfp.osfc.io/osfc2020/talk/SLFJTN/ and Rome https://cfp.osfc.io/osfc2020/talk/TBSHA8/, respectively. Although these are early days, there is definitely a groundswell of interest to evolve how critical infrastructure code such as firmware is written, especially given the needs for more robust code and efficiency of the developer workflow. And as an example of that sentiment, since OSFC is a virtual conference, the closest thing to the ‘hallway track’ was commentary in the sharing tool, including the following exchange on Rust

  

Open Source Firmware Conference 2020

Bret Barkelew6 hours ago
I agree with Ron that now I've worked with Rust it's like pulling teeth when I have to go back. ;)
J. Redpath5 hours ago
a sign of something good
Vincent Zimmer5 hours ago
yes. moving from C to Rust feels like the same dynamic of moving from assembly-based firmware to C 20 years ago.
Diego Rodríguez5 hours ago
^this

In addition to that hallway track discussion above, there were other hallway discussions in the Facebook session about UEFI versus coreboot complexity.This reminded me of the culture of UEFI and coreboot, or as I sometimes think, a "windows-bios" versus a "linuxbios." By that I mean the EDK code is written in the style of Windows kernel code and coreboot is written in the style of Linux kernel code.

To begin, coreboot literally started as "LinuxBIOS", as mentioned in chapter 4 of https://www.amazon.com/gp/product/B01JC1LDTY/. EDKII history is described in the "Beyond BIOS" article https://www.intel.com/content/dam/www/public/us/en/documents/research/2011-vol15-iss-1-intel-technology-journal.pdf.  

The reality of the latter is that Ken Reneris, mentioned in http://vzimmer.blogspot.com/2018/10/ghosts-of.html, created the original IBI/EFI core. He was a OS/2 and Windows kernel/Hal veteran who joined Intel in '98 around the time of the initial IBI effort. Ken brought along the same Camel Case coding style as Windows kernel code, the Containing Record (CR) https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/8_private_context_data_structures/81_containing_record_macro macro from ntddk.h into efi.h, TPL's from Windows IRQL's, .inf's, and the build command-driven build of the original EFI sample which became the EFI Developer Kit's I and II. CR's are pretty interesting in that it allows for a 'public' interface in a structure to have some appended, implementation-specific instance 'private' information. This allows for C++ keyword functionality for public/private to be emulated in C code.

Speaking of the legacy of ex-MS Ken, the prevalent use of GUID's in EFI, along with the 'protocols', or GUID-named API's, bear not a small resemblance to COM https://en.wikipedia.org/wiki/Component_Object_Model. Think iUnknown versus HandleProtocol, etc. but forging COM's C++ infrastructure in C.

Ron, on the other hand, notes that the original LinuxBIOS was derived from Linux, thus it has stutter-case/Indian Hill https://www2.cs.arizona.edu/~mccann/cstyle.html coding standard, basic make support, and Kconfig as LinuxBIOS became coreboot. coreboot also started out as fully-open, whereas EDKII slowly released more sources of the last 20 years.

Beyond coding standards, the EDK-based implementations of UEFI always vied to cover the entire boot flow, from the tuple of {reset, silicon init, platform init, OS bootload phase} as {SEC, PEI, early DXE, late DXE}. DXE is both platform initialization and the UEFI core. coreboot, on the other hand, supports the same tuple as {bootblock, romstage, ramstage, payload}. The payload for coreboot could always have been a full kernel, as CSM-like compatibility module like Seabios, or today even a EDKII-Dxe implementation in the core of the UefiPayloadPkg. 

The richness of the OS bootload phase in coreboot is separated from the basic silicon and board initialization. With the DXE phase doing both platform initialization and OS bootload, the complexity of the latter ends up encroached on the former. The OS bootload code is highly reused and rich, per the UEFI specification, whereas board initialization is a high traffic area where board specific changes often occur and is the venue to host a lot of the bring-up and debug experiences.

Popping up from my trip down memory lane, OSFC also had debates between MS and others on the chat channel about the distinction between the general purpose platform where the OS producer may be different from the platform producer, versus a more vertical platform with the firmware and OS provisioned under the same authority.

Or the distinction between a full feature platform that supports a plurality of shrinkwrap OS's versus doing the minimum and letting the OS do the rest.

Another big distinction between the two is that EDK grew up initially closed and any of the open elements were released under a permissive BSD license. whereas coreboot grew up open with the GPL license. I was curious about the value of the GPL in firmware and received the following comment from Marc Jones many years ago  via the question 'why hoard SATA bug fixes?' EDK BSD allows for snapping the open source instance and not redistributing changes, such as bug fixes, whereas the GPL of coreboot, Linux, and u-boot obligate the party changing the GPL code to redistribute their fixes.

Luckily, there are more than a few common points of alignment between EDKII and coreboot today.  These include EDKII-based UEFI Payload Package for shrinkwrap OS support, both communities are pursuing unit tests w/ cmocka, ACPI is the predominate runtime interface, etc.

From those present-day alignments there are also some similar trends, such as the oreboot from C-based coreboot and the RUST-on-edkII investigation versus full EDKII C code

Speaking of EDKII implementations, in addition to the PDF of some of the OSFC talks, the videos of talks for OSFC 2020 can be found at https://vimeo.com/showcase/osfc-2020.

Beyond the OSFC talks, one can find the recently-published "Understanding the Trusted Boot Chain Implementation" at the following links  https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.pdfhttps://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.mobi, and https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.epub. These binaries are compiled from https://github.com/tianocore-docs/edk2-TrustedBootChain. This material is above-and-beyond the recent book https://www.amazon.com/Building-Secure-Firmware-Armoring-Foundation-dp-1484261054/dp/1484261054/ and was created in order to provide guidance to communities, such as the Trusted Computing Group and TianoCore, on consuming and producing the measurements in a more interoperable fashion.

I also just noticed another security related EDKII publication, namely an insightful analysis of PE/COFF image loading https://arxiv.org/pdf/2012.05471.pdf.  This is close to home for UEFI and EDKII. The work reminds me of the more generic studies like https://eprint.iacr.org/2019/564.pdf.As you may have guessed from my earlier blogs and writings, I'm a fan of more formality and rigor in the pursuit of future-looking designs and validation, respectively.



Friday, December 4, 2020

Resources for starting with host firmware

 u-root general slack channel question on 'Does anybody have any recommended books or learning resources for getting into firmware development? I have a background in embedded systems and systems software, but am looking to learn more about end-host / server firmware (e.g UEFI, Coreboot and friends). https://osfw.slack.com/archives/C0RAR7JRM/p1606927939015000



This reminds me of curating firmware blogs a few years back http://vzimmer.blogspot.com/2015/06/firmware-related-blogs.html,

To that end, here's a list.  

LinuxBoot book https://github.com/linuxboot/book 

LinuxBoot can be a payload for either coreboot or Slim Bootloader. Some details on the latter 2 can be found at


coreboot https://doc.coreboot.org/ 

coreboot book https://www.apress.com/gp/book/9781484200711 

and


Slim Bootloader https://slimbootloader.github.io/ 


Slim Bootloader and coreboot provide 'platform initialization' (PI) and depend upon a payload.


U-boot is interesting since it can be either a payload or the full PI implementation.


U-boot https://www.denx.de/wiki/U-Boot/Documentation 

U-boot porting example http://ptgmedia.pearsoncmg.com/images/9780134030005/samplepages/9780134030005.pdf 


The same holds for EDKII. EDKII can be a platform implementation or act as a payload for Slim Bootloader and coreboot.


EDKII training https://github.com/tianocore-docs/Training https://github.com/tianocore-training 

EDKII white papers https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-white-papers 


For the ARM ecosystem, ARM Trusted Firmware (ATF) acts as a substrate to compose different PI implementations


ARM Trusted Firmware https://trustedfirmware-a.readthedocs.io/en/latest/


Some firmware security training that covers many of the firmware frameworks can be found in below.


Firmware security training https://github.com/enascimento/firmware-security-training  https://opensecuritytraining.info/

Firmware security book https://link.springer.com/book/10.1007/978-1-4842-6106-4 


UEFI and ACPI are industry standards described at https://www.uefi.org/, and an overview of UEFI and its shell can be found in the below.


UEFI book https://www.amazon.com/Beyond-BIOS-Developing-Extensible-Interface/dp/1501514784/ (older version at https://www.microbe.cz/docs/Beyond_BIOS_Second_Edition_Digital_Edition_(15-12-10)%20.pdf

UEFI Shell book https://www.amazon.com/gp/product/B06XK19DW3/ 


In addition, there is nice collection of talks on open source firmware and UEFI, respectively, at


OSFC talks https://osfc.io/archive 

UEFI talks https://uefi.org/learning_center/presentationsandvideos 


ACPI overviews are a little lite IMHO, but there is a recently added introduction in the ACPI in chapter 1


ACPI Specification https://uefi.org/sites/default/files/resources/ACPI_6_3_May16.pdf

ACPI + UEFI Support https://www.intel.com/content/www/us/en/content-details/671067/a-tour-beyond-bios-implementing-the-acpi-platform-error-interface-with-the-uefi.html?wapkw=zimmer%20acpi 


11.30.2022 - new books

security

https://link.springer.com/book/10.1007/978-1-4842-6106-4

overall fw

https://link.springer.com/book/10.1007/978-1-4842-7939-7

https://link.springer.com/book/10.1007/978-1-4842-7974-8