operating system vendors and firmware

The PC industry has followed an interesting arc, beginning with the hardware and firmware details of the original IBM PC http://bitsavers.org/pdf/ibm/pc/pc/6025008_PC_Technical_Reference_Aug81.pdf and the closed source Microsoft (MS) operating system MS/DOS https://github.com/microsoft/MS-DOS. The former allowed for the ability to have clones of the IBM PC, whereas the latter ensured some consistency in the platform design as compatibility with DOS, and then Windows, helped provide for the open platform that non-MS OS's like Linux today enjoy. 

Was the distinction between MS as an OS vendor (OSV) and the industry as a platform provider always so stark? In fact, the existence of MS-written firmware, as one example, has had various examples spanning the 90's up through today's news. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To begin:

In the early days of Windows NT, including 3.51 and 4.0, Microsoft wrote the ARC boot firmware for their DEC Alpha and MIPS NT platforms. ARC stands for Advanced RISC Computing https://en.wikipedia.org/wiki/Advanced_RISC_Computing and the document http://www.netbsd.org/docs/Hardware/Machines/ARC/riscspec.pdf provided guidance on both platform hardware and firmware. A couple of MS engineers wrote the first ARC firmware and the ARC specification, along with IEEE 1275 Open Firmware (OF) https://www.openfirmware.info/data/docs/of1275.pdf, were considered as a solution for 'how to boot Itanium' during the early days of the IBI/EFI specification. 1275 came up because the PowerPC port of NT booted via OF. EFI ended up looking more like ARC, see similarity between the ARC Firmware Function Vector and UEFI System Table, although OF does have some advantages, especially for security https://www.cs.cornell.edu/~kozen/Papers/acsac.pdf. 

Another interesting aspect of the ARC Alpha firmware was that the NT NDIS miniport drivers from NT were embedded in the firmware for purposes of pre-OS networking, such as network booting. Other than boot loaders like U-Boot which liberally leverages portions of device driver code, this OS and firmware code sharing helps solve one of the main challenges of boot firmware, namely having to create a firmware version of a device driver for the block, network, and console services in the pre-OS and another variant for the OS runtime. 

 |

 |

\/

Fast forward to the early 2000's. At that time there was a project called FlexGo    https://en.wikipedia.org/wiki/Microsoft_FlexGo which allowed for a metered, or subscription PC.  There was MS firmware integrated into the code morpher, or microcode-like firmware of the Transmeta device https://en.wikipedia.org/wiki/Transmeta https://www.extremetech.com/extreme/76602-amd-to-resell-transmetas-payasyougo-chip. For some of the standard PC's at the time, there was exploration of having the monitoring agent from MS as an additional handler in System Management Mode (SMM) of the BIOS

 |

 |

\/

In the early 2010's, the industry was moving from a TPM 1.2 to 2.0. One of the learning's from the 1.2 era was that the specification for the commands lent themselves to various interpretations. As such, the TPM 2.0 specification was written in 'literate code' where the C based implementation of the commands could be extracted. The latter C code has formed the underlying implementation   https://github.com/microsoft/ms-tpm-20-ref for all of the integrated and discrete 2.0 devices. This code was born of the 'firmware TPM' work described in the technical report https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/msr-tr-2015-84.pdf. 

 |

 |

\/

In the 20-teens, there emerged from Microsoft Project Mu for BIOS https://microsoft.github.io/mu/. Although this is officially referred to as a downstream fork of EDKII on tianocore.org, there are many unique aspects, especially features like DFCI in https://github.com/microsoft/mu_plus.

A more recent example includes MS writing the SMM supervisor for AMD SecureCore PC https://www.microsoft.com/security/blog/2020/11/12/system-management-mode-deep-dive-how-smm-isolation-hardens-the-platform/ variant mentioned during OSFC '20 'hallway chat.'

And for the broader industry MS wrote many SMM audit and checking tools in Mu to help prepare the OEM ecosystem for having their handlers running in the jailed context of the SecureCore PC SMm supervisor.

 |

 |

\/

In the future, MS hardware Pluton hardware and firmware https://www.microsoft.com/security/blog/2020/11/17/meet-the-microsoft-pluton-processor-the-security-chip-designed-for-the-future-of-windows-pcs/ may be even more deeply integrated.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Beyond the long run of the MS examples above you can see a similar, albeit shorter in time, arc for Google, especially Google as an OSV with respect to ChromeOS. It begins with the Google Chromebooks leveraging coreboot + vboot  https://www.coreboot.org/ where Google employees are many of the upstream maintainers. 

 |

 |

\/

More recently, the Titan/OpenTitan  https://opentitan.org/ has emerged. This is a root of trust with the CPU based upon RISC-V and the operating system based upon the Tock OS https://github.com/google/tock-on-titan written in Rust. Interesting stuff.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

And you cannot have firmware without hardware, of course. Google Titan-M and the OpenTitan are one example.

For Microsoft, MS already has its XBox360 https://en.wikipedia.org/wiki/Xbox_360_technical_specifications and XBox One https://en.wikipedia.org/wiki/Xbox_One, which were custom PowerPC "Watermoose" and AMD "Jaguar" based SOC's, respectively.

There are also always rumors in the air for MS, such as recent one on ARM https://arstechnica.com/gadgets/2020/12/microsoft-may-be-developing-its-own-in-house-arm-cpu-designs/ and earlier one on custom designs like E2 https://www.theregister.com/2018/06/18/microsoft_e2_edge_windows_10/.

For Google we already have ample public details on their Tensor Processing Unit (TPU) https://semiengineering.com/knowledge_centers/integrated-circuit/ic-types/processors/tensor-processing-unit-tpu/ and the above-listed Open Titan first party silicon, but these are not general purpose system on a chip (SOC). Of course even Google has the rumor mill swirling on occasion with stories like "Whitechapel" https://www.theverge.com/2020/4/14/21221062/google-processor-pixels-chromebooks-whitechapel-samsung-qualcomm, too.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So how does the OS impact the firmware? Well, since the firmware is closely tied to the overall platform and hardware design, OS 'requirements' documents http://download.microsoft.com/download/7/0/e/70e74967-b0fe-477a-974f-c1ed16ee31df/windows8-1-hardware-cert-requirements-system.pdf and https://source.android.com/compatibility/10/android-10-cdd can dictate some of these choices. And even for more open firmware implementations like Chromebooks you can see the coupling https://www.chromium.org/chromium-os/2014-firmware-summit.

musings about firmware cultures

In a quick journey around firmware cultures in this posting, I'll talk a bit about the recent Open Source Firmware Conference (OSFC) 2020.

The Open Source Firmware Conference (OSFC) started in 2018. It was a broadening of the coreboot conference to include other open source firmware projects. Some examples of the earlier coreboot conference include the 2016 https://www.coreboot.org/Coreboot_conference_San_Francisco_2016 where Intel talked about the ‘then recent’ UEFI Payload project https://youtu.be/I08NHJLu6Us and Intel FSP 2.0 https://youtu.be/uzfiTiP9dEM. I was invited to give the keynote of the inaugural OSFC conference in 2018 https://2018.osfc.io/uploads/talk/paper/1/OSFC_Keynote-005.pdf  There common themes such as chipsec, Slim Bootloader, Min Platform, coreboot, and Intel FSP were noted.

 

Now have FSP SDK   https://github.com/universalpayload/fspsdk/tree/qemu_fsp_x64 mentioned in that keynote, too.

Fast forward to OSFC 2020 https://cfp.osfc.io/osfc2020/schedule/ and you see many of the same sentiments being reiterated. Intel talked about extending Intel FSP for programmable service engine support https://cfp.osfc.io/osfc2020/talk/TNTFYV/, a more modern configuration mechanism https://cfp.osfc.io/osfc2020/talk/AS7EZR/, and the efforts https://cfp.osfc.io/osfc2020/talk/VUNDSC/ to have a more reusable payload  https://github.com/universalpayload. From those talks we then go to the efforts to remove dependencies upon SMM, including the Platform Resource Monitor (PRM) https://cfp.osfc.io/osfc2020/talk/MCJASB/. 

Beyond those talks, Jiewen Yao co-presented on the Security Protocol and Data Model (SPDM) https://cfp.osfc.io/osfc2020/talk/ECQ88N/ along with Xiaoyu Ruan. SPDM is a new standard from the DMTF for device and host firmware security that is critical for upcoming security initiative support. openspdm is a sample implementation of SPDM specification. It will be used in multiple device and host firmware implementations including UEFI EDK II and possibly other platform firmware, such as a Baseboard Management Controller (BMC) based upon OpenBmc, etc.

Beyond SPDM, Jiewen also shared the background and efforts with Virtual Firmware for Intel Trust Domain Extensions (TDX) https://cfp.osfc.io/osfc2020/talk/CRKZB8/. This effort entails open source efforts to help scale enabling for TDX and provides a venue to discuss aligning enabling with other confidential computing efforts like AMD Secure Encrypted Virtualization (SEV). The TdShim is also used as a foundation for any service Trust Domain (TD) for TDX advanced feature in an EFI-light environment, such as the virtual firmware for a container or virtual TPM services. It bridges the gap between TD startup and applications running, and it enables the customers building their own use cases on top Intel TDX.

Andy Jassy during re:invent this year also spoke about how change in large companies is often driven by outsiders since long-time employees are often reluctant to replace what they've built in the past. And in the spirit of change, Rust was a topic of a few https://osfc.io/ talks this year, including the virtual hallway track discussion.

Specifically Jiewen Yao and I presented on Enabling Rust in UEFI firmware https://cfp.osfc.io/osfc2020/talk/SLFJTN/. This is a complementary talk to those by Google on enabling Rust in oreboot https://cfp.osfc.io/osfc2020/talk/SLFJTN/ and Rome https://cfp.osfc.io/osfc2020/talk/TBSHA8/, respectively. Although these are early days, there is definitely a groundswell of interest to evolve how critical infrastructure code such as firmware is written, especially given the needs for more robust code and efficiency of the developer workflow. And as an example of that sentiment, since OSFC is a virtual conference, the closest thing to the ‘hallway track’ was commentary in the sharing tool, including the following exchange on Rust

  

Open Source Firmware Conference 2020

Bret Barkelew6 hours ago

I agree with Ron that now I've worked with Rust it's like pulling teeth when I have to go back. ;)

.

J. Redpath5 hours ago

a sign of something good

.

Vincent Zimmer5 hours ago

yes. moving from C to Rust feels like the same dynamic of moving from assembly-based firmware to C 20 years ago.

.

Diego Rodríguez5 hours ago

^this

In addition to that hallway track discussion above, there were other hallway discussions in the Facebook session about UEFI versus coreboot complexity.This reminded me of the culture of UEFI and coreboot, or as I sometimes think, a "windows-bios" versus a "linuxbios." By that I mean the EDK code is written in the style of Windows kernel code and coreboot is written in the style of Linux kernel code.

To begin, coreboot literally started as "LinuxBIOS", as mentioned in chapter 4 of https://www.amazon.com/gp/product/B01JC1LDTY/. EDKII history is described in the "Beyond BIOS" article https://www.intel.com/content/dam/www/public/us/en/documents/research/2011-vol15-iss-1-intel-technology-journal.pdf.  

The reality of the latter is that Ken Reneris, mentioned in http://vzimmer.blogspot.com/2018/10/ghosts-of.html, created the original IBI/EFI core. He was a OS/2 and Windows kernel/Hal veteran who joined Intel in '98 around the time of the initial IBI effort. Ken brought along the same Camel Case coding style as Windows kernel code, the Containing Record (CR) https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-guide/8_private_context_data_structures/81_containing_record_macro macro from ntddk.h into efi.h, TPL's from Windows IRQL's, .inf's, and the build command-driven build of the original EFI sample which became the EFI Developer Kit's I and II. CR's are pretty interesting in that it allows for a 'public' interface in a structure to have some appended, implementation-specific instance 'private' information. This allows for C++ keyword functionality for public/private to be emulated in C code.

Speaking of the legacy of ex-MS Ken, the prevalent use of GUID's in EFI, along with the 'protocols', or GUID-named API's, bear not a small resemblance to COM https://en.wikipedia.org/wiki/Component_Object_Model. Think iUnknown versus HandleProtocol, etc. but forging COM's C++ infrastructure in C.

Ron, on the other hand, notes that the original LinuxBIOS was derived from Linux, thus it has stutter-case/Indian Hill https://www2.cs.arizona.edu/~mccann/cstyle.html coding standard, basic make support, and Kconfig as LinuxBIOS became coreboot. coreboot also started out as fully-open, whereas EDKII slowly released more sources of the last 20 years.

Beyond coding standards, the EDK-based implementations of UEFI always vied to cover the entire boot flow, from the tuple of {reset, silicon init, platform init, OS bootload phase} as {SEC, PEI, early DXE, late DXE}. DXE is both platform initialization and the UEFI core. coreboot, on the other hand, supports the same tuple as {bootblock, romstage, ramstage, payload}. The payload for coreboot could always have been a full kernel, as CSM-like compatibility module like Seabios, or today even a EDKII-Dxe implementation in the core of the UefiPayloadPkg. 

The richness of the OS bootload phase in coreboot is separated from the basic silicon and board initialization. With the DXE phase doing both platform initialization and OS bootload, the complexity of the latter ends up encroached on the former. The OS bootload code is highly reused and rich, per the UEFI specification, whereas board initialization is a high traffic area where board specific changes often occur and is the venue to host a lot of the bring-up and debug experiences.

Popping up from my trip down memory lane, OSFC also had debates between MS and others on the chat channel about the distinction between the general purpose platform where the OS producer may be different from the platform producer, versus a more vertical platform with the firmware and OS provisioned under the same authority.

Or the distinction between a full feature platform that supports a plurality of shrinkwrap OS's versus doing the minimum and letting the OS do the rest.

Another big distinction between the two is that EDK grew up initially closed and any of the open elements were released under a permissive BSD license. whereas coreboot grew up open with the GPL license. I was curious about the value of the GPL in firmware and received the following comment from Marc Jones many years ago  via the question 'why hoard SATA bug fixes?' EDK BSD allows for snapping the open source instance and not redistributing changes, such as bug fixes, whereas the GPL of coreboot, Linux, and u-boot obligate the party changing the GPL code to redistribute their fixes.

Luckily, there are more than a few common points of alignment between EDKII and coreboot today.  These include EDKII-based UEFI Payload Package for shrinkwrap OS support, both communities are pursuing unit tests w/ cmocka, ACPI is the predominate runtime interface, etc.

From those present-day alignments there are also some similar trends, such as the oreboot from C-based coreboot and the RUST-on-edkII investigation versus full EDKII C code

Speaking of EDKII implementations, in addition to the PDF of some of the OSFC talks, the videos of talks for OSFC 2020 can be found at https://vimeo.com/showcase/osfc-2020.

Beyond the OSFC talks, one can find the recently-published "Understanding the Trusted Boot Chain Implementation" at the following links  https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/, https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.pdf, https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.mobi, and https://tianocore-docs.github.io/edk2-TrustedBootChain/release-1.00/edk2-TrustedBootChain-release-1.00.epub. These binaries are compiled from https://github.com/tianocore-docs/edk2-TrustedBootChain. This material is above-and-beyond the recent book https://www.amazon.com/Building-Secure-Firmware-Armoring-Foundation-dp-1484261054/dp/1484261054/ and was created in order to provide guidance to communities, such as the Trusted Computing Group and TianoCore, on consuming and producing the measurements in a more interoperable fashion.

I also just noticed another security related EDKII publication, namely an insightful analysis of PE/COFF image loading https://arxiv.org/pdf/2012.05471.pdf.  This is close to home for UEFI and EDKII. The work reminds me of the more generic studies like https://eprint.iacr.org/2019/564.pdf.As you may have guessed from my earlier blogs and writings, I'm a fan of more formality and rigor in the pursuit of future-looking designs and validation, respectively.

Resources for starting with host firmware

 u-root general slack channel question on 'Does anybody have any recommended books or learning resources for getting into firmware development? I have a background in embedded systems and systems software, but am looking to learn more about end-host / server firmware (e.g UEFI, Coreboot and friends). ' https://osfw.slack.com/archives/C0RAR7JRM/p1606927939015000

This reminds me of curating firmware blogs a few years back http://vzimmer.blogspot.com/2015/06/firmware-related-blogs.html,

To that end, here's a list.  

LinuxBoot book https://github.com/linuxboot/book 

LinuxBoot can be a payload for either coreboot or Slim Bootloader. Some details on the latter 2 can be found at

coreboot https://doc.coreboot.org/ 

coreboot book https://www.apress.com/gp/book/9781484200711 

and

Slim Bootloader https://slimbootloader.github.io/ 

Slim Bootloader and coreboot provide 'platform initialization' (PI) and depend upon a payload.

U-boot is interesting since it can be either a payload or the full PI implementation.

U-boot https://www.denx.de/wiki/U-Boot/Documentation 

U-boot porting example http://ptgmedia.pearsoncmg.com/images/9780134030005/samplepages/9780134030005.pdf 

The same holds for EDKII. EDKII can be a platform implementation or act as a payload for Slim Bootloader and coreboot.

EDKII training https://github.com/tianocore-docs/Training https://github.com/tianocore-training 

EDKII white papers https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-white-papers 

For the ARM ecosystem, ARM Trusted Firmware (ATF) acts as a substrate to compose different PI implementations

ARM Trusted Firmware https://trustedfirmware-a.readthedocs.io/en/latest/

Some firmware security training that covers many of the firmware frameworks can be found in below.

Firmware security training https://github.com/enascimento/firmware-security-training  https://opensecuritytraining.info/

Firmware security book https://link.springer.com/book/10.1007/978-1-4842-6106-4 

UEFI and ACPI are industry standards described at https://www.uefi.org/, and an overview of UEFI and its shell can be found in the below.

UEFI book https://www.amazon.com/Beyond-BIOS-Developing-Extensible-Interface/dp/1501514784/ (older version at https://www.microbe.cz/docs/Beyond_BIOS_Second_Edition_Digital_Edition_(15-12-10)%20.pdf) 

UEFI Shell book https://www.amazon.com/gp/product/B06XK19DW3/ 

In addition, there is nice collection of talks on open source firmware and UEFI, respectively, at

OSFC talks https://osfc.io/archive 

UEFI talks https://uefi.org/learning_center/presentationsandvideos 

ACPI overviews are a little lite IMHO, but there is a recently added introduction in the ACPI in chapter 1

ACPI Specification https://uefi.org/sites/default/files/resources/ACPI_6_3_May16.pdf

ACPI + UEFI Support https://www.intel.com/content/www/us/en/content-details/671067/a-tour-beyond-bios-implementing-the-acpi-platform-error-interface-with-the-uefi.html?wapkw=zimmer%20acpi 

11.30.2022 - new books

security

https://link.springer.com/book/10.1007/978-1-4842-6106-4

overall fw

https://link.springer.com/book/10.1007/978-1-4842-7939-7

https://link.springer.com/book/10.1007/978-1-4842-7974-8

Different boot and testing schemes

In the news, interesting to see the details of the M1-based Apple devices. As a firmware person, I was curious about how the machines boot. 

They mention the omission of bootcamp https://support.apple.com/boot-camp https://discussions.apple.com/thread/252021896 or the support of 3rd party OS's https://www.zdnet.com/article/top-apple-exec-native-windows-10-on-m1-macs-is-a-choice-microsoft-needs-to-make/.

Does that mean the boot paradigm of the Macs has changed from UEFI http://refit.sourceforge.net/info/boot_process.html? Apple was never fully UEFI compliant and often eschewed more complex capabilities like SMM-protected Authenticated Variables and other rich runtime services, although they did support ACPI.

The answer was formally revealed in the video https://developer.apple.com/videos/play/wwdc2020/10686 at 16:22

which mentions that the boot process is leveraged from the iPhone/iPad, namely iBoot https://en.wikipedia.org/wiki/IBoot. 

The native iBoot launch was confirmed by Apple UEFI engineer https://twitter.com/NikolajSchlej/status/1275951574200709120, along with using device tree https://elinux.org/Device_Tree_What_It_Is in lieu of ACPI https://twitter.com/NikolajSchlej/status/1275951827754774528. Flattened Device Tree predates ACPI and was part of how the original PowerPC MACs booted, with OpenFirmware Forth-based FCode and FDT as the equivalent of UEFI/ACPI for the programmatic and declarative runtime tables interfaces, respectively. Open Firmware was circa 1994 https://en.wikipedia.org/wiki/Open_Firmware whereas ACPI was 1996 https://en.wikipedia.org/wiki/Advanced_Configuration_and_Power_Interface#:~:text=First%20released%20in%20December%201996,Play%20BIOS%20(PnP)%20Specification.&text=ACPI%20defines%20a%20hardware%20abstraction,UEFI)%20and%20the%20operating%20systems., respectively. EFI, the precursor the UEFI, is the youngest of all three, dating its earliest appearance in 1998 https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface.

As Apple showed in slide 14, the T2-based Mac's already support the bridgeOS kernel going from iBoot to UEFI. This sort of layering of UEFI implementations on top of a non-PI based platform code is pretty common, with the https://github.com/tianocore/edk2/tree/master/UefiPayloadPkg providing an EDK2-based implementation of UEFI for coreboot https://doc.coreboot.org/ and slim bootloader https://slimbootloader.github.io/index.html. We discussed this in figure 6-17 of https://link.springer.com/chapter/10.1007/978-1-4842-0070-4_6 and we are trying to generalize the use of payloads in https://github.com/universalpayload. We compare a T2-based secure boot chain to other implementations in chapter 4 of https://www.amazon.com/Building-Secure-Firmware-Armoring-Foundation/dp/1484261054, too.

I suspect the disk layout is still based upon the GUIDed Partition Table (GPT), which was an invention of EFI. You can support GPT even without performing a non-UEFI OS boot, as showed by Chromebooks https://www.chromium.org/chromium-os/chromiumos-design-docs/disk-format, for example. To support a non-UEFI OS you simple can omit the EFI System Partition (ESP) and have your boot loader load a kernel from some arbitrary GUID-defined partition type.

We also show how the concept of a phase-based boot and clean interfaces to payloads enables more radical firmware stacks, such as slide 24 of https://cfp.osfc.io/media/osfc2020/submissions/SLFJTN/resources/OSFC2020_Rust_EFI_Yao_Zimmer_NDK4Dme.pdf. 

 

Another interesting example of stacking a Rust-based UEFI implementation on an alternate platform initialization layer can be found in https://github.com/retrage/rust-hypervisor-firmware/tree/coreboot-support, too, that I saw on the open source firmware's oreboot slack channel. 

Speaking of another A-prefixed company that's not Apple, Amazon has been doing some pretty interesting things in firmware. Former Intel colleague Mark Tuttle's work in mentioned in the write-up https://www.freertos.org/2020/02/ensuring-the-memory-safety-of-freertos-part-1.html about using the C Bounded Model Checker (CBMC) https://www.cprover.org/cbmc/ to ensure memory safety of FreeRTOS. This is a nice balance of providing better assurance with an existing type-weak language like C versus the forklift upgrade to something like Rust. We discuss CBMC a bit in chapter 21 of https://www.amazon.com/Building-Secure-Firmware-Armoring-Foundation/dp/1484261054, including an example of usage in listing 21-2. 

This work in the rebar-embossed book is nothing like the fully operationalized example of the real-time OS listed above, though. Beyond this specific work, I appreciate how Amazon endeavors to make formal methods more developer friendly, as described in the work by Nathan, Mark, and their colleagues in https://assets.amazon.science/d0/de/cbec0b4547e3ae7ff077f8aa978f/code-level-model-checking-in-thesoftware-development-workflow.pdf. This type of analysis is much more relevant to my day-job than the former treatise on FM they wrote regarding TLA https://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-services-uses-formal-methods/fulltext, although some of Hillel Wayne's https://www.hillelwayne.com/post/why-dont-people-use-formal-methods/ work may help build a bridge for me. More examples of the good stuff published by their engineers/'builders' that I also referenced in my last blog posting.

And formal verification of system software is popping up in other places, such as the mention "Formal verification of the code running at EL2" in https://linuxplumbersconf.org/event/7/contributions/780/attachments/514/925/LPC2020_-_Protected_KVM_.pdf. I haven't dug into this too much, but I wonder if it is aligned with Google Project Oak, including the Hafnium hypervisor https://hafnium.googlesource.com/hafnium/+/HEAD/docs/Architecture.md and its Coq based verification https://github.com/project-oak/hafnium-verification/tree/hfo2/coq-verification? Or some other KVM-focused verification? 

As a parting note, I'd like to thank the work of Tim Lewis https://uefi.blogspot.com/ as the technical reviewer of https://link.springer.com/book/10.1007/978-1-4842-6106-4 and Leendert Van Doorn https://www.paramecium.org:4443/~leendert/ for writing the preface. I have appreciated the opportunity to work with these two technical leaders in various roles since the 2000's.

Silicon valley, innovation, and logos

As we near the end of SeptemberOctober I realize that I haven't updated this blog in a bit. Sometimes my entries are inspired by recent events or material I bump into, such as https://spectrum.ieee.org/tech-history/cyberspace/todays-internet-still-relies-on-an-arpanetera-protocol-the-request-for-comments. This format is refreshingly simple when I compare the effort in curating the https://www.rfc-editor.org/rfc/rfc5970.txt versus some of the more painful processes in other venues.

Speaking of ipv6, I remember visiting Facebook in 2011 or 2012 to help with some UEFI ipv6 network boot issues. Since we didn't retrofit IPV6 boot to legacy PXE, datacenter folks going all ipv6 https://www.datacenterknowledge.com/archives/2010/06/10/facebook-deploys-ipv6 UEFI was the only game in town for deployment. I still recall the rough, open interior of the Facebook campus https://www.theregister.com/2011/02/08/facebook_in_menlo_park/, but the best memory was when I was leaving. My host walked me through a courtyard and a recall passing a building with a windowed corner office where I saw Zuckerberg leaning back in a chair in his T-shirt with a phalanx of suited men standing in front of his desk. It was as if a king of yore was holding court with his many knights and vassals. Fascinating stuff.

People may decry the "Fall of silicon valley" https://www.robrhinehart.com/the-fall-of-silicon-valley/, but from my first trip there in 1997, to that 2012 trip, to the training session http://www.chromium.org/chromium-os/2014-firmware-summit/2014%20Chrome%20OS%20Firmware%20Summit-%20Overview.pdf at the CHM https://computerhistory.org/, to my last trip to Intel HQ last year....

including catching a glimpse of Grove http://vzimmer.blogspot.com/2014/02/anniversary-day-next-next.html 

Speaking of Grove and his collected wisdom

such as "disagree and commit," there has always been a battle to enforce this, including folks who might "disagree and de-commit." Regrettably, for '20 a new one has crept in. namely, "agree and ignore. But as always, a culture is only as strong as the folks who continually fight to uphold it. 

Regarding values I now find myself often using the triple of 'business first, team second, and career third' to explain to engineers the priority they should use in their career. Namely, focus on solving business problems first, even if the issues are outside of your organization. Next ensure that you have a healthy team and support your organization. Finally, worry about your career. Mistakes often happen when engineers put their career's as a first priority. In the latter case it may yield a small local victory but in the end is corrosive to both the business and the team.

In addition, the I found the value of 'always leaving' to be of interest from the Google culture https://www.oreilly.com/library/view/software-engineering-at/9781492082781/. It doesn't mean that you should always be switching jobs but instead it argues that employees always ensure that they provide sufficient documentation and collateral such that they can 'leave' a role for a more important mission and easily have someone take their place. Someone once remarked "I guess that I have job security since my code is so complicated." That's not demonstrating 'always leaving' but instead a more pernicious 'always nesting.' And in fact the reply to that person was that such a type of code will lead to disruption because long term the business cannot sand for that state of affairs.

Beyond Grove's book and recent musings in the Google text, some of the insights from Amazon are quite interesting. These include the role of PE's in their organization https://deloitte.wsj.com/cio/2020/09/16/at-aws-engineers-drive-architecture-shape-products/ to the Builders' Library https://aws.amazon.com/builders-library/?cards-body.sort-by=item.additionalFields.customSort&cards-body.sort-order=asc. The latter is interesting in that it solves many problems - a scalable way for PE's to mentor, and by providing the material publicly, demonstrates the expertise and competence of their technical leads to customers along with their internal population.

Beyond 'ignoring' and other things, I do like the quote "Innovation takes something that people use and improves upon it. "from the above-cited "Fall of Silicon Valley" article. I should reprise my thoughts http://vzimmer.blogspot.com/2013/12/invention-and-innovation.html on this topic. 

On the subject of improvements, we are also trying to enhance the workflow across many different firmware technologies, such as slimboot, coreboot, and PI-based implementations like EDKII - the span of which we discuss a bit in chapter 1 of https://www.springer.com/us/book/9781484261057, too. This work includes an effort to allow for interoperability between these environments via a more standardized 'payload' We have a draft specification at https://universalpayload.github.io/documentation/spec/spec.html and various implementations of payloads, including Linux https://github.com/universalpayload/linuxpayload and EDKII https://github.com/universalpayload/edk2, alongside various bootloaders like slimboot https://github.com/universalpayload/slimbootloader and coreboot https://github.com/universalpayload/coreboot to invoke the payload. The latter will grow to EDKII Min Platform, Oreboot, and the former to U-Boot and potentially Skiboot or Hostboot.

In addition to curating works on the universal payload above, the earlier-promised 'FSP SDK' mentioned in slide 20 of https://2018.osfc.io/uploads/talk/paper/1/OSFC_Keynote-005.pdf is being developed in https://github.com/universalpayload/fspsdk.

Regarding additional improvements, we continue to explore Rust for firmware, including the talk https://www.youtube.com/watch?v=dCu0-frSURE and presentation https://uefi.org/sites/default/files/resources/Enabling%20RUST%20for%20UEFI%20Firmware_8.19.2020.pdf. We elaborate on these points in chapter 20 of the upcoming https://www.springer.com/us/book/9781484261057, too. 

In addition to improving code through language based security, the boothole vulnerability led to some interesting exchanges on Twitter https://twitter.com/vincentzimmer/status/1290377140223934465, including a curation of the various defense-in-depth activities underway for EDKII https://github.com/jyao1/SecurityEx/blob/master/Summary.md. This work is something that the tianocore infosec https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues can help drive, along with tightening up the CVE allocation process https://cve.mitre.org/news/archives/2020/news.html#September182020_TianoCore_Added_as_CVE_Numbering_Authority_CNA. Writing CVE's appears to be as much art as science.

The NSA also weighed in on UEFI Secure boot usage in https://www.nsa.gov/news-features/press-room/Article/2347822/nsa-releases-cybersecurity-technical-report-on-uefi-secure-boot-customization/, including https://trustedcomputinggroup.org/wp-content/uploads/TCG_EFI_Platform_1_22_Final_-v15.pdf. The latter has an interesting list of contributors who have moved on - Lee and Wooten and Springfield retired, Shiva at HPE, Bill at Amazon, Monty at GE....

I guess that I've been on this project too long. I even saw a reference in the TPM dev miniconference to figure from https://people.eecs.berkeley.edu/~kubitron/courses/cs194-24-S14/hand-outs/SF09_EFIS001_UEFI_PI_TCG_White_Paper.pdf. This was before Brogan became master of Mu https://github.com/microsoft/mu_tiano_platforms and when I had a lot more hair.

Ironically I even find people who sometimes try to explain to me the intent behind some thing I created 20 years ago. Recently it was some nuance of the PEI infrastructure. I didn't have the heart to tell them that I invented the item in question, wrote the initial implementation and the specification on the topic, too. Or mention why the Terse Executable (TE) image has the 'VZ' signature. I just smiled and nodded by head. Ah, youth.

As a final thought, the Intel logo change inspired a trip down memory lane http://vzimmer.blogspot.com/2014/01/.

The new logo https://images.anandtech.com/doci/16063/20200902172826_575px.jpg

is pretty compelling.

And on that I think I'll close on this, a happy note.

Happy & safe continued quarantine to those in geo's with restrictions.