Showing posts with label Framework. Show all posts
Showing posts with label Framework. Show all posts

Monday, January 8, 2024

"Advances in Platform Firmware 'Beyond BIOS'" - 20 years later.....

 A decade has passed since the posting https://vzimmer.blogspot.com/2014/01/advances-in-platform-firmware-beyond.html describing the journey of Tiano https://www.researchgate.net/publication/377810413_TechnologyIntel_Magazine_-_Advances_in_Platform_Firmware_Beyond_BIOS_and_Across_all_Intel_R_Silicon, which at the time was Intel Framework based specifications and the EDK implementation. As such, this month makes 20 years since the article landed on the internet.


The write-up was curiously available in several languages:

    English - https://github.com/vincentjzimmer/Documents/blob/master/it01043_eng.pdf

    Portuguese - https://github.com/vincentjzimmer/Documents/blob/master/it01043_pg.pdf

    Japanese - https://github.com/vincentjzimmer/Documents/blob/master/it01043_j.pdf

    Russian - https://github.com/vincentjzimmer/Documents/blob/master/it01043_ru.pdf

    Spanish - https://github.com/vincentjzimmer/Documents/blob/master/it01043_sp.pdf

    Chinese - https://github.com/vincentjzimmer/Documents/blob/master/it01043_cn.pdf 

Today the EDK https://sourceforge.net/projects/edkit/ has become https://github.com/tianocore/edk2 and the Framework https://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-specifications-general-technology.html corpus is UEFI PI https://uefi.org/specs/PI/1.8/index.html. The OS-visible interface has gone from EFI 1.10 https://www.intel.com/content/dam/doc/product-specification/efi-v1-10-specification.pdf to UEFI https://uefi.org/specs/UEFI/2.10/, too. 

Given the amount of change over this interval, I always have a tough time refreshing the graphic 


Many interesting twists and turns on this journey that started in the late 90's and still continues as we step into the year 2024.

Tuesday, December 6, 2022

Homebrew computer club

One of the nice things about commutes back to the office includes catching up on podcasts and audio books. For the latter I've been listening to Walter Isaacson's book https://www.amazon.com/Steve-Jobs-Walter-Isaacson/dp/1451648537 on Steve Jobs. I liked the mention of folks like Wozniak and their valley friends, including early Apple employee Allen Baum. There was also mention of the Homebrew computer club https://en.wikipedia.org/wiki/Homebrew_Computer_Club which to me represents the spirit of low-level development I've enjoyed with firmware development these last decades.

Although Allen Baum has popped up recently on RISC-V circles https://riscvglobalforum2020.sched.com/speaker/allen.baum, the longer arc of his career is journaled in https://archive.computerhistory.org/resources/access/text/2018/06/102717165-05-01-acc.pdf. I remember Baum from early 2000's Intel chipset work and his thoughts on the Intel tenure.



Quite the storied career. 

One detail of the book also mentions includes the extensibility of the Apple, such as the slots on the Apple II that were subsequently eschewed by Jobs with the Macintosh. There are seems to be a trend of closed versus open going through cycles. The spirit of silicon valley and the ability to touch and modify hardware is often challenged by concerns about security, ease-of-use, and IP protection.

Speaking of the openness, I like how my co-author https://github.com/reinauer from https://link.springer.com/book/10.1007/978-1-4842-0070-4 still exercises this spirit both on the job with Chromebooks and its developer mode https://sites.google.com/a/chromium.org/dev/chromium-os and off the job with his Amiga group https://amiga.technology/ efforts.

In addition to having the host-firmware updatable, the platform also has an open source embedded controller (EC). Recently the EC moved from a bespoke implementation for Chromebooks to using Zephyr, as described at https://chromium.googlesource.com/chromiumos/platform/ec/+/HEAD/docs/zephyr/README.md, in OSFC talk https://cfp.osfc.io/osfc2020/talk/SXYUQD/ and book https://link.springer.com/book/10.1007/978-1-4842-7974-8



And in the spirit of hardware extensibility (i.e., Apple II slots for the 2020's) and more end-user modifiable firmware, there is the Framework Chromebook edition https://frame.work/laptop-chromebook-12-gen-intel. This feels like a fusion of the spirit of the club given the extensibility of the hardware modules and firmware.

It also looks like 'some' of that spirit follows with the Pixel phone w/ alternate OS's like https://grapheneos.org and its Pixel phone virtualization https://blog.esper.io/android-dessert-bites-5-virtualization-in-android-13-351789/. It's almost as if the traditional 'OS' is really a VM with art like this and the recent Mac's.

Regrettably my hobby budget for tech is challenged by the last two products mentioned, so I have limited my recent investments to RISC-V hacking on the D1 and VisionFive SBC and the Pinephone https://www.pine64.org/pinephone/ for more open telephony. More extreme splurges are https://www.bunniestudios.com/blog/?p=5921 which fuses Rust-based firmware and soft RISC-V RV64 AP core and RISC-V EC realized with an FPGA with open source tooling. 

PS
I have been wondering whether this blog 'written in the cracks' (i.e., over a few spare moments on the weekend or lunch-hour) is good or bad. With the advent of chatGPT https://chat.openai.com/chat I figured perhaps tech could take the reins for me, viz.,



Open source firmware is a type of software that is developed and maintained by a community of volunteers. Unlike proprietary firmware, which is owned and controlled by a single company, open source firmware is freely available for anyone to use, modify, and distribute.

One of the main benefits of open source firmware is that it allows users to have complete control over their devices. With proprietary firmware, users are often limited in what they can do with their device and are at the mercy of the company that created the firmware. With open source firmware, users can customize and modify their device to suit their needs, without being restricted by the limitations of proprietary firmware.

Another benefit of open source firmware is that it promotes collaboration and innovation within the tech community. Because the source code is freely available, anyone can contribute to the development of the firmware, and can suggest improvements or new features. This allows for a faster pace of innovation, as more people are able to contribute their expertise and knowledge to the project.

One of the most popular examples of open source firmware is the Linux operating system, which is used on millions of computers, servers, and other devices around the world. Linux is developed and maintained by a global community of volunteers, and is freely available for anyone to use and modify.

Another example of open source firmware is the firmware that powers many home routers and other networking devices. This firmware, known as OpenWrt, is widely used by tech enthusiasts and hobbyists who want to customize and improve the capabilities of their home networking devices.

In addition to the benefits of open source firmware, there are also many active communities of people who are interested in creating and using their own computer hardware. These communities, known as "homebrew computer clubs," are made up of individuals who are passionate about building their own computers, either from scratch or by using kits and other pre-made components.

Homebrew computer clubs often focus on using open source firmware and software, as it allows them to have complete control over their creations and to customize them to their specific needs. Additionally, these clubs provide a great way for like-minded individuals to connect, collaborate, and learn from one another.

In conclusion, open source firmware and homebrew computer clubs are important parts of the tech community. Open source firmware provides users with the freedom and flexibility to customize and control their devices, while homebrew computer clubs promote collaboration and innovation within the community. Together, they help to drive the advancement of technology and the development of new and exciting technologies. 

At first I thought I had broken the AI but instead it just needed some think-time. Quite the time to be in tech these days.

PPS

apropos of last sentiment, https://twitter.com/vincentzimmer/status/1598935527293870081 now becomes https://twitter.com/MarcoFigueroa/status/1600171380175011844

PPS+

 

 
 

The soul of the computer lies within its UEFI, a complex and intricate system that dictates the very essence of its being. It is the driving force behind the machine's every action, the mastermind that orchestrates its inner workings. But just as in the great works of Dostoyevsky, the UEFI is also a source of great turmoil and conflict. It is a battleground for competing interests and competing ideas, and the decisions made within its walls can have profound consequences for the computer and all who use it.

At the heart of the UEFI is the BIOS, a set of instructions that determine how the computer starts up and communicates with its various components. But the UEFI is much more than just the BIOS. It is a sophisticated system that controls the flow of information within the computer, and it is constantly being updated and refined by the engineers who design it.

The UEFI is a powerful tool, capable of performing complex tasks with ease. But it is also a delicate instrument, and it must be handled with care. One wrong move, and the entire system can be thrown into chaos. In the hands of a skilled user, the UEFI can be a force for good, enabling the computer to operate at its full potential. But in the hands of a malicious actor, it can be a weapon of destruction, capable of wreaking havoc on the computer and all of its data.

In the end, the UEFI is a complex and mysterious system, one that is both essential to the functioning of the computer and fraught with danger. Like the characters in the great works of Dostoyevsky, it is a source of both light and darkness, and it must be approached with caution and care.

Saturday, November 30, 2019

Beware Experts, Beware Disruptors

This blog entry hearkens back to a tale of expertise versus disruption. The scene is DuPont, WA in the late 1990's. Prior to Tiano (2000+) there was an effort to write a BIOS from scratch in the erstwhile workstation product group (WPG) at Intel. The code name of the effort was "Kittyhawk" (1998). Like all boot firmware, the initialization was broken up into phases. Instead of the tuple of {SEC, PEI, DXE, BDS} of UEFI PI of {BOOTBLOCK, ROMSTAGE, RAMSTAGE, PAYLOAD) of coreboot, Kittyhawk had {VM0, VM1, VM2, BOOTLOAD} phases.

Although the platform initialization of Kittyhawk was in native code protected mode C code (and cross-compiled to Itanium for the latter boot variant), this was prior to emergence of IA-32 EFI and it's OS's, which was a parallel effort in DuPont. Instead, the 32-bit version of Kittyhawk relied upon a PC/AT BIOS boot interface for MBR-based operating systems. To make this happen an engineer decomposed an existing PC/AT BIOS from it's POST (Power-On Self Test) component from it's 'runtime', or the 16-bit code that provided the Int-callable BIOS API's. The 32-bit Kittyhawk code did device discovery and initialization, and then the 32-bit code provided the device information in a data block into the generic 16-bit BIOS 'runtime.' This 16-bit code was called 'the furball' by management in anticipation of a day when it could be 'coughed up' in lieu of a native 32-bit operating system load.

This Kittyhawk effort on IA-32 was definitely deemed disruptive at the time. The DuPont Intel site was more of a rebellious effort, with the larger product teams in Oregon constituting the existing 'experts'. There were cries from Oregon that the disruptors in Washington would never boot Windows, and if they did, they'd never pass WHQL, or the suite of compliance tests for Windows. The furball booted OS's and passed compliance tests. It turned out that having a non-BIOS engineer look at the problem didn't entail the prejudices of what's possible, and the work about having a clean interface into a BIOS runtime was used in the subsequent Tiano Framework work such as the Compatibility Support Module (CSM) csm https://www.intel.com/content/dam/www/public/us/en/documents/reference-guides/efi-compatibility-support-module-specification-v098.pdf API design.

So this part of the story entailed providing some caution in listening to the experts, but....

You sometimes need to beware disruptors. Specifically, riding high upon the success of building a common C code based to support IA-32 and Itanium, with the furball providing 32-bit OS support and the Intel Boot Initiative (IBI)/EFI 0.92 sample provide 64-bit Itanium OS support, the next challenge was scaling to other aspects of the ecosystem. Part of this scaling entailed support of code provided by other business entities. The EFI work at the time had the EFI images based upon PE/COFF, so the Kittyhawk team decided to decompose the statically linked Kittyhawk image into a set of Plug In Modules (PIM's).

After the Kittyhawk features were decomposed into PIM's, I remember standing in the cubicle of one of the Kittyhawk engineers and a BIOS guru visiting from Oregon helping with the 64-bit bring-up, the topic ranged over to option ROM's. The Kittyhawk engineer said "let's just put our PIM's into option ROM containers." I was a bit surprised, since to me the problem with option ROM's wasn't carving out code to run in the container but more entailed 'how' to have the option ROM's interoperate with the system BIOS. That latter point is where standards like EFI came into play by having a 'contract' between the Independent Hardware Vendors (IHV's) that built the adapter cards and the OEM's building the system board BIOS.

So the work of the disruptors was laudable in showing that you could have shared, common code to initialize a machine, but to scale to new paradigms like OS and OpROM interfaces you really needed a broader paradigm switch. And as would be shown, it took another decade to solve this interoperability issue.

As such, sometimes you have to be a bit wary of the disruptors, although the disruptive Kittyhawk did provide many learning's beyond the furball/CSM concept, including how up to today EDKII builds its ACPI tables via extracting data tables from C code files. Alas the Kittyhawkfort was shut down as part of the shuttling of the Workstation Group, and efforts to build a native code BIOS solution fell into the hands of the EFI team as part of the emergent (1999+) Tiano effort. At this time there was the EFI sample that eventually grew into the EFI Development Kit (aka the first Tiano implementation), now referred to as EDKI versus today's EDKII. EDK leveraged a lot of the learning's of EFI to inform the Intel Framework specifications, including the phases of SEC, PEI, DXE, and BDS we know today.

This original PEI phase differed somewhat from the PEI you can find in the UEFI PI specification, though. Instead of the concept of C-based PEIM's and PEI core, the original instantiation of the PEI Core Interface Specification (PEI-CIS) was based upon processor registers. There was still the business requirement of separate binary executables and the solution of GUID's for naming interfaces was used. But instead of a PPI database, the PEIM's exposed exported interfaces through a Produced GUID List (PGL) and imported interfaces via a Consumed GUID List (CGL). During firmware construction and update the CGL and PGL were reconciled and linked together. And in order to having services that interoperate, the concept of a 'call level' was introduced. The call level detailed which registers could be used by services in a PEIM since there was no memory for a call stack with this early PEI-CIS 0.3.

At the same time we were defining PEI 0.3, there was another approach to writing pre-DRAM code without memory, namely the Intel® Applied Computing System Firmware Library V1.0 (hereafter referred to as Intel® ACSF Library).
ACSFL was described in a Intel Update article (same dev update magazine https://github.com/vincentjzimmer/Documents/blob/master/it01043.pdf)
that provided 32-bit callable libraries for boot-loaders. This effort came from the embedded products team and entailed delivering a series of libraries built as .a files. This simplified initialization code addressed the lack of a memory call stack by using the MMX registers to emulate a call stack. This differed from the PEI 0.3 model of reserving a set of registers for each call level. The ACSFL concept was smarter in that constraining PEIM's to a given call level really impacted the composition of these modules into different platform builds.

I do enjoy the quotation 'requires only 15KB of flash' when I wander over to https://github.com/IntelFsp/FSP/blob/master/KabylakeFspBinPkg/Fsp.fd with its size of 592k, or 39x size dilation of silicon initialization code over the last 20 years. This is similar to the 29x scaling in the number of transistors between the 7.5 million in a Pentium II https://en.wikipedia.org/wiki/Pentium_II and the 217 million in a Kaby Lake https://www.quora.com/How-many-transistors-are-in-i3-i5-and-i7-processors.
The same article provides the software layering. One can see some similarity of ACSF Library to the Intel Firmware Support Package (FSP). This is no accident since the Intel FSP was intended to originally target the same embedded market, although Intel FSP in 2012 had the embedded mantle being carried by the Intel of Things Group. As another point of irony, the original FSP built w/ the Consumer Electronics Firmware Development Kit (CEFDK). The CEFDK was in fact the evolution of the ACSF library, going through various instances, like FDK. The latter were used to enable phones and tablets beyond just embedded.

So ACSF Library provided some learning's, and at the same time the LinuxBIOS (prior name of coreboot) solved this issue of supporting C code without initialized DRAM via the ROMCC tool.  ROMCC was a custom compiler that used processor registers to emulate a stack so that you could write things like DRAM initialization in C code.

The initial implementations of PEI-CIS 0.3 https://www.intel.co.kr/content/dam/www/public/us/en/documents/reference-guides/efi-pei-cis-v09.pdf with just registers proved difficult to deploy, and since part of the Tiano effort entailed a requirement to use standard tools, techniques like ROMCC were deemed not tenable. As such, as the PEI CIS went from 0.3 to 0.5, we moved PEI to the concept of temporary memory, with using the processor cache as RAM (CAR) as the 'temp ram.' Regrettably we were asked to not disclose how we implemented temp RAM, and the technique and initialization code were not made open source or published (and a modern instance of this reluctance can be found in the FSP-T abstraction of FSP 2.0). The coreboot community, though, didn't have this constraint and https://www.coreboot.org/images/6/6c/LBCar.pdf https://www.coreboot.org/data/yhlu/cache_as_ram_lb_09142006.pdf provided details on how to use this technique.

As time progresses, I am amused about the various recollections. During the late 2000's someone told me 'you rewrote PEI many times' when in fact the only substantive change was going from registers in 0.3 to temporary memory in 0.5. Although not necessarily a full re-write, the EFI implementations did have a long history:

IBI Sample -> EFI Sample -> Tiano Release 1..8, Tiano Release 8.1..8.7,  EDK1117, ...

...UDK2015, UDK2016, .......

Also, some fellow travelers mentioned to me their fond recollections of EFI discussions in 1996. I smile because the first discussions of EFI (in the form of IBI) weren't until 1998. But I suspect everyone's memory gets hazy over time, with this blog post having its own degree of fog and haze.

And these efforts also remind me that changes in this space take time. A recent reminder that api support takes time, such as the discussion of the EFI Random Number protocol in Linux
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.5-EFI-RNG-x86. This is an API that was defined in 2010 by Microsoft for Windows 8 and then introduced in the UEFI standard in 2013. Or features like UEFI secure boot from 2011 UEFI 2.3.1 showing up in Debian Buster https://wiki.debian.org/SecureBoot just in mid 2019.

The other interesting perspective is https://fs.blog/2019/12/survivorship-bias/, namely you often hear about the success stories, not failures. Only for extreme cases like https://en.wikipedia.org/wiki/Normal_Accidents do you find a rich journal of failures. So in portions of this blog and the EFI1.2 discussion in http://vzimmer.blogspot.com/2013/02/anniversary-daynext-arch-ps-and-some.html I have tried to shed light upon some of the not so successful paths.



Wednesday, August 9, 2017

Accessing UEFI UpdateCapsule from the operating system runtime

"Accessing UEFI from the operating system runtime" http://vzimmer.blogspot.com/2012/12/accessing-uefi-form-operating-system.html represents my most frequently accessed blog posting. In fact I scrawled this quick posting in response to an engineer having recently sent me a mail referencing the above posting and decrying lack of information on access to the UpdateCapsule interface from the various OS's.

To begin, let's start with the API exposed by the UEFI firmware is defined as followed:
The capsule in memory follows:



From my perspective as a 'builder' of firmware I often focus on the underlying constituent elements, but that's a smaller audience than the consumers of the firmware. At the time of the posting, the UEFI Variable interface was the more important interface in order to access both UEFI specification defined variables, namely those {GUID, Unicode String} named pairs codified in the UEFI specification, and vendor-defined variable GUID's and Names.

In the five years that have followed that posting, there's another important extensible run time interface that has been exposed to the operating system run time, namely the UpdateCapsule interface. The Capsule infrastructure began as part of the Intel Framework corpus https://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-capsule-specification.html, but was eventually donated into the UEFI Forum in a similar specification arc as HII. Recall that much of the Intel Framework specifications, such as PEI and DXE, became pillars of the UEFI Platform Initialization (PI) specifications, but when an interface needs interoperability between the pre-OS ISV's and OS runtimes, that is purveiw of the UEFI (and ACPI) specifications. Microsoft complemented this Framework-era capsule infrastructure with the ESRT, or a list of updatable elements in the platform defined by a list of GUID's.

Although the UpdateCapsule API can be used to convey any information from the run into the pre-OS, including crash-dump, management information, etc, the 'firmware update' usage is the most important from a business perspective.

And regarding the API, having a definition of the interface and the data enveloping mechanism are necessary but not sufficient. You also need producers of the update interface on system boards and infrastructure software to invoke the interface. To that end, the EDKII community has published a rich set of infrastructure code to provide the interface https://github.com/tianocore/tianocore.github.io/wiki/Capsule-Based-Firmware-Update-and-Firmware-Recovery with a detailed code explication in https://github.com/tianocore-docs/Docs/raw/master/White_Papers/A_Tour_Beyond_BIOS_Capsule_Update_and_Recovery_in_EDK_II.pdf. On the operating system side, there is infrastructure to support invoking the interface for both Linux https://lists.gt.net/linux/kernel/2149809 and Microsoft Windows https://docs.microsoft.com/en-us/windows-hardware/drivers/bringup/system-and-device-firmware-updates-via-a-firmware-driver-package.

The Linux kernel exposes the capsule loader interface via sysfs in a similar fashion to how the UEFI variable interfaces are exposed. The Windows implementation, though, doesn't expose the direct interface but instead conjoins issuing capsules on top of the infrastructure for installing drivers. This is where the distinction between capsules as a mechanism to pass a GUID-named data payload with a scatter-gather list in memory back to firmware compares to usage of this interface to pass payloads that are a firmware update. On the latter point of updates, the Linux community has build out the fwupd service http://fwupd.org/ to facilitate pushing out updates in a similar fashion to Windows Update http://www.uefi.org/sites/default/files/resources/2014_UEFI_Plugfest_07_Microsoft.pdfhttps://blueprints.launchpad.net/ubuntu/+spec/foundations-w-uefi-capsule-update provides an interesting view into steps involved in plumbing a Linux distribution for this end-to-end use case, too.

You can think of the UpdateCapsule invocation as a syscall back to the firmware. This is different than UEFI Variables where the expectation that the 'set' call persists immediately without and intervening platform restart. Instead, by having the UpdateCapsule take effect (typically) across a restart, the update of the underlying firmware can occur in the early boot of the firmware Trusted Computing Base (TCB) prior to running third party code. Or a capsule can just be passed through, such as the case of the OS runtime sending its panic screen to be displayed across a restart to its UEFI OS loader.

Philosophical postlude -
The difference between UpdateCaspule versus the Get/Set Variable interface is that the latter has been available in the EFI (and then UEFI) OS's since 1999. Update Capsule, and the corresponding ESRT, have only appeared more recently. If I had a chance to invoke George Cox's http://vzimmer.blogspot.com/2015/06/guids-revisions-interrupts.html "I could do it better the 2nd time" penchant of engineering, I would have argued that art such as UEFI Authenticated Variables would have been better built as signed UEFI Capsules versus UEFI Variables since authentication-at-reset in the PI phase (BIOS TCB) is much easier to build than an authentication agent in the firmware that is isolated from the OS or hypervisor run time, as needed by the UEFI Authenticated Variables.
Sigh. Hindsight is 20/20.

Tuesday, December 13, 2016

Provisioning, Porting and Types

I'd like to begin this posting with a review of work presented years ago. Specifically, my friend Harry H provided me copies of my first three Intel Developer Forum (IDF) presentations from 2003  https://github.com/vincentjzimmer/Documents/blob/master/Non-IA%20Silicon%20Support%20with%20the%20Intel%20Platform%20Inovation%20Framework%20for%20EFI%20-%202003.pdf https://github.com/vincentjzimmer/Documents/blob/master/EFI_Specification_Evolution_Final_04%20-%202003.pdf and 2004 https://github.com/vincentjzimmer/Documents/blob/master/EFIS001_100_2004.pdf, respectively.

The first presentation was jointly delivered https://github.com/vincentjzimmer/Documents/blob/master/Non-IA%20Silicon%20Support%20with%20the%20Intel%20Platform%20Inovation%20Framework%20for%20EFI%20-%202003.pdf with Bob H and Michael K. This work informed some of the information in chapter 7 of the UEFI Book. The basics of the architecture haven't changed, with the notable exception of advocating the use of Intel(R) FSP https://github.com/tianocore-docs/Docs/raw/master/White_Papers/A_Tour_Beyond_BIOS_Open_Source_IA_Firmware_Platform_Design_Guide_in_EFI_Developer_Kit_II.pdf and associated marriage with open source platform code on http://www.tianocore.org.

That same 2003 event featured a solo talk on security features https://github.com/vincentjzimmer/Documents/blob/master/EFI_Specification_Evolution_Final_04%20-%202003.pdf. This talk included an introduction of the modular network stack which we internally ear-marked for a never-released "EFI 1.2" specification. These API's on slide 15 ended up appearing in the UEFI 2.0 specification circa 2006 http://www.uefi.org/sites/default/files/resources/UEFI_Specification_2_and_Errata_Sept16_08.pdf and later in the open source https://github.com/tianocore/edk2/tree/master/NetworkPkg. Beyond these API's, though, other elements like the EAP-Teanie method were never realized in the market. The best documentation of the latter appeared in a paper on using UEFI in the Cloud on pp. 4-5  https://github.com/vincentjzimmer/Documents/blob/master/SAM6560.pdf. Custom EAP methods violate the design precept of UEFI, namely leveraging well-known art, including authentication methods. Thus the recent focus on TLS for HTTP-S and EAP-TLS for our various network use-cases.

From 2004, other items that landed on the cutting run floor included the EFI_SECURITY_SUPPORT_PROTOCOL on page 20 of the presentation. In the ensuing ten years I attempted to standardize an interface like this in the standards body, but we have ended up instead with a library class https://github.com/tianocore/edk2/tree/master/CryptoPkg which can be layered directly on a static library such as OpenSSL or a private protocol. Finally, the pp 23-24 "COB's" of the EFI_CONFIGURATION_OBJECT_PROTOCOL never appeared in the open source or the standards, but the configuration aspects of slide 30 finally appeared in the UEFI standard from the original OEM-scoped HII Framework standard http://www.intel.com/content/dam/www/public/us/en/documents/reference-guides/efi-human-interface-infrastructure-specification-v091.pdf.

So much for IDF in 2003. In 2004 we presented on EFI Security extensions again in https://github.com/vincentjzimmer/Documents/blob/master/EFIS001_100_2004.pdf. In this talk we elaborated on the 2003 talk with more details, including slide 17 for PE/COFF EFI image integrity. Since that talk we have evolved a UEFI image integrity model in https://github.com/vincentjzimmer/Documents/blob/master/SAM4542.pdf (2007),
https://github.com/vincentjzimmer/Documents/blob/master/UEFI-Networking-and-Pre-OS-Security.pdf (2011), and https://github.com/vincentjzimmer/Documents/blob/master/A_Tour_Beyond_BIOS_into_UEFI_Secure_Boot_White_Paper.pdf (2012).  Same story on the rest of the vision, though. The vision of smart objects like COB's was not realized in the market.

A common theme on the IDF 2003 and 2004 presentation was the topic of provisioning, though. With the UEFI 2.6 specification, x-UEFI, and HTTP boot, the vision has been realized in a different figuration in 2016 https://github.com/vincentjzimmer/Documents/blob/master/UEFI%20Plugfest%202015%20-%20UEFI%20and%20the%20Cloud%20-%20003%20Bulusu%20-%20Zimmer%20.pdf https://github.com/vincentjzimmer/Documents/blob/master/UEFI_Plugfest_2015_Challenges_in_the_Cloud_Whitepaper_0.pdf. As such, I can claim the ball has been moved down the field in the last decade.

Another topic that perhaps hasn't progressed as much in the last decade entails language-based security (LBS). Back in the early 2000's I investigated how technology like http://www.cs.cornell.edu/talc/ might be applied to the EFI firmware https://patents.google.com/patent/US6978018B2/en, including a failed port to https://cyclone.thelanguage.org/. That's why I am excited to see efforts like https://firmwaresecurity.com/2015/09/04/rust-and-uefi/, including the write up http://ceur-ws.org/Vol-1746/paper-15.pdf. These efforts show promise to complement other security efforts in this space. I also enjoyed the reference in the latter paper to the EFI memory map white paper, UEFI book, and Cloud presentation. To me this affirms the value of openness, from publications down to the source code, in evolving an ecosystem.

Another interesting intersection of Rust and system software is http://www.tockos.org/ which a colleague at a nearby company mentioned to me over lunch. I need to dig into this work in the future.

Speaking of ecosystem evolution, I also want to cite our write up on the recently upstreaming capsule design https://github.com/tianocore-docs/Docs/raw/master/White_Papers/A_Tour_Beyond_BIOS_Capsule_Update_and_Recovery_in_EDK_II.pdf. If this paper work were to have pre-dated the Rust paper perhaps it would have been the capsule reference in lieu of the "BZ15" one? This latter white paper reads on 'platform recovery,' such as described in the UEFI PI specification. As a complementary overview of operating system (OS) recovery there is https://github.com/vincentjzimmer/Documents/blob/master/UEFI-Recovery-Options-002-1.pdf which explicates some of the technology in the UEFI specification and touched upon by me at the last UEFI plug fest.

Now off from working in an urban area

to a more suburban office

I'm not sure which is spookier.

Well, enough of looking in the rear-view mirror today. Let's instead face the wind screen and continue forward down the road.



Monday, June 6, 2016

Shields and Networks

In this blog I opine about shields, API's, and networks.

To begin with shields, the clip 'UEFI on Agents of S.H.I.E.L.D.'  https://www.youtube.com/watch?v=9lc95nXKWMM and associated transcript dialog http://transcripts.foreverdreaming.org/viewtopic.php?f=140&t=27259 included "This is called a Unified Extensible Firmware Interface."

What a fascinating occurrence in the popular culture of 2016. It reminds me of the introduction of the "Unified" term to the 'Extensible Interface' when our circa 1998 EFI 1.02 specification was sent to the standards body www.uefi.org, and shortly afterward in the book
http://www.amazon.com/Beyond-Bios-Implementing-Extensible-Interface/dp/0974364908/

I wasn't the first choice for the book. The opportunity was offered to several others and only landed upon my doorstep after the editor had read the 2004 white paper UPDATE. I was motivated the pursue this effort in order to declare that UEFI was being implemented by "Intel's Framework", or our code base infrastructure that became EDKII on tianocore.org. This first implementation was also based upon the Intel Framework specifications http://www.intel.com/content/www/us/en/architecture-and-technology/unified-extensible-firmware-interface/efi-specifications-general-technology.html that subsequently became the UEFI Platform Initialization (PI) Specifications.

While at Intel, Richard Wirt https://www.crunchbase.com/person/richard-wirt-m-s-ph-d#/entity was our Vice President and told me that there is both 'real and perceived' leadership. I believe we demonstrated 'real' leadership by delivering the EFI 1.0 specification into UEFI 2.0, the Framework specifications into PI 1.0, and the Framework code into the EFI Developer Kit (and later the EFI Developer Kit II - EDKII). But the book helped highlight that our 'real' leadership also had attendant 'perceived' leadership in the market.

Another good sound bite from management in those days was "if you cannot describe your job in a sentence or two, you don't really know what you're doing."

We updated the book five years later
http://www.amazon.com/Beyond-BIOS-Developing-Extensible-Interface/dp/1934053295/


The part of the latter book that most pleased me was sneaking the work 'eschew' past the editor. From page 285

'Although the development and design team eschewed use of proper names in code or the resultant binaries, the "VZ" and "Vincent Zimmer" association appeared harmless, especially given the interoperability advantages.'

Regarding UEFI ,a nice thing about UEFI and PI include locking down interfaces for purposes of interoperability.  The Intel Firmware Support Package (FSP) builds upon this API codification, with the following quote from R. Minnich in the introduction to
http://www.amazon.com/Embedded-Firmware-Solutions-Development-Practices/dp/1484200713/
So API's are one thing, but the system also has interfaces on the network. This is where networking and wire protocols come into play. I'm happy to see the boot-from-HTTP that we codified in UEFI 2.5 and expanded upon with RAM disk scenarios in UEFI 2.6 continuing its build out. We struggled a bit thinking about how to evolve the UDP/TFTP-based PXE to the HTTP-based use cases. One of the features of the UDP-based PXE was the multi-cast variant of TFTP. We explored evolving PXE to be more scalable with streaming, big block, and reducing the number of ACK's. Dave Thaler told me that with those efforts in hand we were 'inventing' TCP, thus the move to the best-known implementation of TCP today, namely the application protocol HTTP, built upon TCP.

And for use-cases where you need the type of scalability found in multicast TFTP if PXE, we have DNS and HTTP for the UEFI HTTP boot. And the entire web model, from content delivery networks (CDN's) to load balancers, optimize the scaling of HTTP. Given that 'the world' is working on that particular scaling problem, the meagre firmware network use case should ride that wave of industry R&D practices. This is in the spirit of EFI wherein we tended to avoid reinventing known art, like file systems (FAT12/16/32), image formats (PE/COFF), and image integrity (Authenticode).

Speaking of scaling HTTP in these first 10000 days of the web, I have to hearken back to my days at the University of Washington 1998 when I was pursuing my masters. I took the computer performance class from John Zahorjan https://homes.cs.washington.edu/~zahorjan/homepage/.

At UW, this project entailed working on evaluating Round Robin DNS policies using web data from Brian Pinkerton's https://en.wikipedia.org/wiki/WebCrawler MetaCrawler. My partner on this proejct was an engineer from the erstwhile Teledesic effort, and we built a discrete event simulator fed by web traffic to assess response latencies based upon DNS server load balancing techniques.



I now appreciate the blog and web posting of information, such as on github, since the code and documentation from this project are lost in the sands of time. If I were to do this project today, I'd definitely share the results on the web. At the time I was also interested in heavy tailed traffic characterization, including work by Mark Crovella http://www.cs.bu.edu/fac/crovella/list.html.

Now we are in 2016 and we are booting from web servers in a standards based fashion.  Cool.

Regarding networking use-cases, it is a continuous challenge deciding  how much functionality to put into the pre-OS versus just booting a deployment operating system, like Linux or Microsoft Windows Pre-installation Environment (PE). I'd say that for diskless workstations, P-blades (processor + memory only compute nodes without local disk), diskless clients, recovering a a failed main OS on disk, etc having integrated networking makes sense. But for sophisticated deployments that need a multi-processor, multi-threaded, interrupt-driven, high performance and feature rich environment, a Linux or Windows PE makes the most sense.

These questions of balanced design weigh upon me as I look after the UEFI networking and security subteams in the UEFI Forum. I worry that I sometimes starve the former given the amount of issues and work in the latter.

And now for some final thoughts on networking and security. At ToorCamp 2012 http://vzimmer.blogspot.com/2012/08/one-conference-down-one-to-go.html I still recall Jacob Appelbaum suggested a removable TPM that the platform owner could destroy when encountered by law enforcement, and then Cryptocat author Nadim Kobeissi mentioned porting to UEFI in order to have a safer environment to do network communications.

The other convention-time suggestion from this era occurred during a trek to the Ubuntu Developer Summit in Orlando. My colleague Harry and I bumped into Mark Shuttleworth in the expo area after the presentations. After discussing UEFI with Mark for a few moments he asked us both about just putting Linux in the hardware for booting Linux from disk.

I'm not sure if these 2012 suggestions will ever come to pass in the platform, but it's always to follow the arc of technology in this industry.

And speaking of following arcs, I need to follow the arc of 'getting back to work.'

Cheers

PS
And don't think about the meat clown




Saturday, December 20, 2014

"So Long, and Thanks for All the Fish"

Yesterday my last day in DuPont after nearly 18 years, 15 of those in the same cube, here at this WA Intel site. Off to work at the Seattle office in early '15. So long, DuPont, and thanks for the memories. And all of the Fish, although in this case 'Andrew Fish.' Although Fish doesn't work at Intel any longer, the inventor of the "Intel Boot Initiative" (IBI), which became EFI/UEFI, still lives in the South Puget Sound. We broke bread for the final time at our favorite local restaurant.


This is the "R&R" mentioned in the acknowledgements section of the 2nd edition of Beyond BIOS http://www.amazon.com/Beyond-BIOS-Developing-Extensible-Interface/dp/1934053295/, "Also many of this team will recall over ten years of "design discussions" at R&R." I captured Fish on the left and Mark Doran http://newsroom.intel.com/community/intel_newsroom/bios?n=Mark%20S.%20Doran&f=searchAll on the right eating our final Teriyaki bowls. As always, good conversations on firmware and technology. And the picture is also epic in that the duo often introduces itself as "Andrew Fish who invented EFI, and Mark Doran who delivered it to the industry."  Mike Kinney & I held down the other side of the table. I've learned quite a bit working with all three of these colleagues over these many years.

During my final days at Compaq in late 1996 I talked to Intel about joining, and my options were "Join Portland for Xeon, or DuPont for Merced." I was inspired by the thought of joining the 64-bit revolution, so there was no doubt in my mind as to which locale to choose.  Also, having grown up in Texas, the Pacific Northwest was a large unknown, so the distinction between OR and WA didn't even factor into my decision.

And my whole tenure at Intel in DuPont since February 1997 has been boot firmware.  I began with a PC/AT BIOS underneath the Merced System Abstraction Layer (SAL), with the original OS loader interface of SAL_PROC (0x13), as a 64-bit mapping of BIOS 13h.  Same for video mapping of SAL_PROC (0x10), keyboard of SAL_PROC (0x13), etc. The OS community wasn't amused.

As the OS interface evolved to C-based IBI, then EFI, the underlying boot infrastructure changed. I moved from driving some of the SAL-on-BIOS for server (aka "FrankenBIOS"), to the short-lived Workstation Product Division (WPD) where we evolved a C-based boot solution. The latter pioneered art such as the Furball and interposer, which were the predecessor of the Intel Framework Compatibility Support Module (CSM) and CSM16 binary.  We called the former 'furball' because it was something we imagined a native 32-bit (IA32) and 64-bit (Itanium) firmware solution would eventually 'cough up', with a less than pleasant imagery derived from the world of cats.

In 1999 we had evolved the workstation firmware to binary modules called "Plug-In Modules", or PIM's. I recall a fateful meeting with the lead on that effort and my colleague visiting from OR. The conversation went something like 'And we can also put the PIM's into option ROMs, too.' At that point I realized the trajectory was ill-fated since the problem with open platforms and option ROM's isn't factoring code to run in that storage container so much as 'how' to interoperate with the system firmware, namely what API's to expose, where to define them, and how to evolve this art. And to that end, I realized that EFI would win as it was the only man standing to solve this problem.

Specifically, EFI nailed down the interface methodology to the OS loader and option ROM community where the broad interoperability was needed.  Moving the underlying initialization firmware to a new methodology, such as what became Intel Framework and then UEFI PI was valuable, but not the necessary first step.  The first step was to lay the foundation for broad interop in order to have time for the top-level OS and IHV ecosystem to start moving their dependent infrastructure.

At that point in my career when I had that realization, I was lucky enough to have a friend and Intel colleague like Andrew Fish who suggested I have lunch with Mark Doran.  Afterward I on-boarded to the EFI team in 1999 and the rest has been a great 15 run on helping evolve EFI into UEFI, EFI-on-BIOS into Framework and then UEFI PI, and the open source offering from "EFI Sample" to EDK to today's EDK II.  Since then a lot of code and specifications have ensued



with hopefully more to come in 2015+.  Not sure if there is another 15 years into the run.  Although one co-developer of Tiano, the code-name for the underlying Framework (and now PI work) quipped a few years ago "We said Tiano was the '20 year' BIOS replacement solution, so when did you start counting?"

Speaking of 2015+, I had better get back to work if I want to get a head start on the next way of the firmware revolution started in South Puget Sound, WA.

Cheers