Thursday, February 24, 2022

25 or Anniversary.Next^10 and early '22 rumblings

In the spirit of work anniversary posts, here's the successor to http://vzimmer.blogspot.com/2021/02/24-or-anniversarynext9-and-128-bits.html.  This post signals the 25th year of working at Intel and 30 years in the industry. 

Since I am so sporadic in blogging, I'll begin with some touches on recent work in the project Universal Scalable Firmware (USF) https://github.com/universalscalablefirmware. I described some of this work at https://talks.osfc.io/osfc2021/talk/HYZL3U/ and it was earlier picked up at  https://www.phoronix.com/scan.php?page=news_item&px=Intel-USF-Firmware, too. The effort builds upon earlier work in FSP https://link.springer.com/book/10.1007/978-1-4842-0070-4 https://www.youtube.com/watch?v=uzfiTiP9dEM  https://www.phoronix.com/scan.php?page=news_item&px=Intel-Better-FSP-License, including adding more traceability https://uefi.org/sites/default/files/resources/Traceable%20Firmware%20Bill%20of%20Materials%20-%2020211207%20-%20007.pdf. It also builds upon earlier OSFC-described work in https://cfp.osfc.io/media/osfc2020/submissions/SLFJTN/resources/OSFC2020_Rust_EFI_Yao_Zimmer_NDK4Dme.pdf and https://2018.osfc.io/uploads/talk/paper/1/OSFC_Keynote-005.pdf.

As always, the hallway track of the last OSFC was the most interesting. Regrettably with COVID this hallway track was a group chat session. I recall some interesting feedback, including 'binaries foster proprietary software development,' 'big companies realize they need to be open but the strategy often fails in middle management,' and 'big companies eventually do open source but some of the original drivers within the company get tired and leave.' As always, it's valuable to process all input and realize the lens through which it is delivered.

Beyond the business side, I appreciated the reference to Roscoe's recent OSDI talk https://people.inf.ethz.ch/troscoe/pubs/2021-07-16-OSDIKeyNote-Handout.pdf cited by one of the OSFC attendees in that 'hallway track' chat session. This talk really resonates with me and others working in this space at many levels. It includes how platform orthodoxy to 'just run a Linux OS' dictates platform architecture and the rich set of programmable compute elements and firmware in a modern platform. The latter has not been so visible historically to the broader research community but it has been the vocation of many of us for decades. I welcome the increased focus of academics in this domain since the ratio of security research attack talks to defense and academic research is deafening. 

The above talks and research show that a career in the firmware space never ends. It's a journey with many fascinating stops and definitely the need to always learn and evolve. I still recall my first BIOS job in the 90's prior to Intel where one of the senior engineers said 'if you know MS DOS and how to debug a PC/AT BIOS you'll always have a job.' Hmmm.

Fast forward to February 24, 1997 with the bushy-haired, suit-wearing engineer who showed up in DuPont, WA http://vzimmer.blogspot.com/2017/02/this-one-is-for-20-or-anniversarynext5.html.  


to




As I look at the things I've worked on, I am reminded of the quote regarding an ethos of 'missionary versus mercenary' when applied to your career. I know that it is something of a luxury to entertain that dialectic since extreme poverty would most likely preclude making that choice. In fact, I recall my father, who grew up as an orphan, reminding me 'money may not buy happiness, but lack of money can surely buy unhappiness.' As such, I believe there is a balance, and once the basic survival needs are requited, deciding which side of the fence you live on is key. Today's frothy job market and extreme compensation packages brings this dynamic up in stark contrast. But in general an interesting thought about career motivations. I fall on the side of missionary. 

I do have to say that this is one of the most exciting times to be in the technology industry. From my early career visiting FTP sites on dialup modems with a 286 to today having multi-core, multi-gigabyte machines and the world-wide-web with its online videos, books, papers, and source repos like Github. 

I became excited about Intel in the mid-90's reading about the company in magazines like EETimes and working on Intel technology, like bringing up a BIOS on a 430HX platform. When I was recruited to Intel to lead the 64-bit Merced/Itanium BIOS work, I was part of a 31% growth wave 1996 - 1997 https://www.chiphistory.org/chc_upload/content/pdf/19/1506018723/1506018723.pdf

Along the way many colleagues have left for other companies, retired or passed away. In addition, my role afforded me the opportunity to work with many parties outside of Intel, whether it be a direct customer, community and government agency. From each I have learned various lessons. These include the perils of arrogance, especially when people confuse correlation versus causation (i.e., something that happened versus their making it happen). 

I do recall one director who always seemed interested in the technology. I asked him why he took the management track. He recounted a tale of working as a compiler engineer at Burroughs early in his career and devoting himself to the project. One day an executive came in and cancelled the overall project. At this point the then-junior-software-engineer thought "I want to be the guy on that side of the table." He the then hung up his coding spurs and pursued an MBA. I admired this manage a lot and recall driving to Oregon for his retirement event. After decades at Intel and many projects he had set up a table with a placard in front of his various projects with a quote about his most memory impact. For UEFI I thought the card would say 'changed the ecosystem' but instead it said 'told the team to stop selling.' Less learned - check.

Another memory from that director included transitioning one of our internal product efforts from legacy BIOS to EDK. There was a heated debate in a forum, including a VP telling my boss 'our customers are not asking for this," with my boss replying loudly 'how can you lead on a strategy if you wait for the customer." This is a variant of https://www.businessinsider.com/steve-jobs-quote-misunderstood-katie-dill-2019-4 

Some people say give the customers what they want, but that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, 'If I'd ask customers what they wanted, they would've told me a faster horse.' People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page.

that I've seen quoted with various degrees of verity.

I've been on this project long enough now, namely joining the EFI team in 1999 and having it re-org'd around me multiply, that I encounter people explaining to me why something I created was done. That's a good thing in my opinion since it shows people are thinking deeply and passionate about the subject.

Speaking of done, I have enjoyed the opportunity to work various endeavors after I was recruited out of Compaq in 1996 and ended up joining the company in February 1997. The path included creating the first 64-bit Itanium BIOS and powering on first workstation. The delay in Merced, and not yet having children, afforded me the opportunity to drive to the University of Washington in Seattle and take courses for my Masters of Science in Computer Science. Toward the end both the birth of my daughter and power-on's made this a tough road to follow. The experience with Itanium afforded my first glimpse into moving 'beyond BIOS', whether it was 64-bit SAL and the PC/AT BIOS atop it for late POST, moving to the workstation effort with the clean-room Kittyhawk code base written in C.

After that I moved to the EFI effort in 1999, helping deliver content from the EFI 1.02, EFI 1.10, EFI 1.20 (never shipped but a subset donated to UEFI 2.0), UEFI 2.0 through UEFI 2.9. EFI was always the higher level pre-OS OS or boot load environment, but other that the plat driver directory, it was decoupled from the underlying platform initialization. To that end, the Kittyhawk effort may have donated some interesting concepts, but the "Tiano" program with with the "Intel Platform Innovation Framework for the Extensible Firmware Interface" (aka. 'the Framework') provided a set of building blocks, including PEI, HOB's, DXE, Data HOB, CSM, SMBUS, HII, SMM, etc.

The Framework specifications informed the creation of the Tiano codebase, which consumed the original EFI sample to become the EFI Developer Kit (EDK). EDK led the charge for the early to late 2000's and was then replaced by the EFI Developer Kit II (EDKII). The latter introduced more modularity via Packaging, PCD's, base libraries, etc.

The EDK and EDKII code bases were posted to sourceforge originally and the latter is now actively maintained on Github.  Most of the 10+ Framework specifications (sans data hub and CSM) were donated to the UEFI Forum, along with Framework-related patents, to become the UEFI Platform Initialization (PI) specification. The PI packaging specification was borne of the EDKII work and also donated, as was the UEFI Shell. Ironically the EFI Shell was conjured up in a few days to support the Itanium power-on, but it took on a life of its own thereafter.

Speaking of patents, Intel has afforded me the opportunity to work at various layers of the stack and get exposed to a plurality of problem statements. And it is the problem statements that are key to ideation in my opinion. One good problem can yield a wealth of patents. And the patents of course have to meet at least the criteria of novelty and value to my employer. The latter reminds me of the quote that 'if you are working on something that the business doesn't care it it's called a 'hobby.'' No time or budget for hobby-patents. 

So for projects, patents, papers, books, standards .... the most memory aspect has always been the collaborators. Technology comes and goes, but people and relationships are the most important. From working on the original Itanium BIOS and simulation and then hardware, to Tiano first booting on the 815 (writing the C-based PEI core and platform PEIM's), to porting EFI to Intel's XScale in 2001, to porting EDK to x64 via both the 2-headed BIOS and 64-bit DXE, to ...

And on standards, from the Framework specifications (PEI, SMM - and getting an IAA for Tiano in 2004), PI 1.0-1.7, .... Collaborating on NIST 800-147 (and getting an IAA for scaling signed updates in 2012), NIST 800-193 (and today's drive for platform resiliency), Open Compute Project https://cdrdv2.intel.com/v1/dl/getContent/671185, network booting (clean-room network stack, IPV6 certified stack, RFC 5970 https://www.rfc-editor.org/rfc/rfc5970.txt, HTTP & WiFi boot), UEFI secure boot & signed updates, EFI TCG measured boot https://cdrdv2.intel.com/v1/dl/getContent/671466......  I have tried to journal some of these things at https://sites.google.com/site/vincentzimmer/cv over time versus citing the same sources here.

I apologize in advance if the reader has gotten this far. This is the 14th year of posting to this blog site and I suspect this entry may win the prize for being the most incoherent. I guess the 24x7 Work From Home (WFH), or is it 'Living At Work' (LAW), lifestyle is bearing down on me, as is the backlog of time off I've yet to use. At Intel we accrue an 8-week sabbatical every 7 years, or a 4 week one every 4 years. I took my first sabbatical in 2004 with my wife and small daughters, including an Alaska cruise, etc. Fast forward to 2011 and I fell off a ladder 2 days into that sabbatical, leading to 2 surgeries and rehab during those 2 months (although I recall 1-hand typing out my articles for https://www.intel.com/content/dam/www/public/us/en/documents/research/2011-vol15-iss-1-intel-technology-journal.pdf during those final weeks of that sabbatical).. So not so much fun in '11. Fast forward to 2018 eligibility - I continued to delay during the 3 year window, which was then extended to 4 year window, which is now extended to August '22 for expiry. Given the latter extension I am now eligible for both my 8 week and a 4 week. Add that to 4 weeks of potential vacation in '22, and I'm sitting on a potential of 4 mos. time-away.  And in tech when is there an amicable time window to jump off this bullet train for a breather.....

Going forward, I see huge opportunities in the firmware space. These include some of the technology aspects of USF that allow for evolving the different layers of the stack at different rates. This includes safer languages and more rigor in testing/validation. https://twitter.com/veorq/status/1496177874264641536?s=20&t=SQKLjCN9DIykwSgU0pK6_Q reminds me of the ever present need. I am intrigued by the work described in https://www.amazon.science/blog/a-gentle-introduction-to-automated-reasoning. As an ironic twist of fate, the mention at the bottom of the page

A fun deep track:

Some algorithms found in the automated theorem provers we use today date as far back as 1959, when Hao Wang used automated reasoning to prove the theorems from Principia Mathematica.


seems invisible on the internet, at least the PDF. Luckily my pack-rat nature for books afforded me an ex-Lib edition of IBM Tech Journal from 1960




I regret that my fellow-travelers in the journey for more formal BIOS evaluation https://www.usenix.org/system/files/conference/woot15/woot15-paper-bazhaniuk.pdf are now retired, off to Eclypsium, or in the case of one of my heroes Mark Tuttle, off to the above listed Amazon Reasoning Group (ARG). I feel like we only scratched the surface in this space. Or maybe just a Quixotic question for a computing science Archimedes "Give me a lever long enough and a fulcrum on which to place it, and I shall move the world."



fin