Which software gets the 'least respect?'

During one of this week's sunny afternoons it was good to see the two Mike's at the Intel Ridgepointe office RPT1-3. Sightings of other humans on cubicle-land is rare in this post-COVID WFH generation. Our gathering in the corner reminded us of a ritual we started in 2000, namely getting coffee in the corner on the 4th floor of the DuPont Intel site DP2-4. The decoder ring is 'site | building #-floor #'. 

We cannot really reprise the event at DuPont given recent history of the site mentioned in http://vzimmer.blogspot.com/2018/07/the-march-of-time.html. 

For the above picture, I'm on the left, Mike Kinney is in the middle, and Mike Rothman in on the right.

Kinney way back machine of IDF 2003

Rothman way back machine of BB 2006.

Back in the early 2000's when we had a coffee corner in DuPont, Mark Doran was a fixture, too. Sadly Mark's commute to the new office is pretty challenging given the eastside of Seattle traffic, so we grafted him into the above image to reproduce the quartet as a virtual attendee.

Here's a way back for Mark

https://redirect.cs.umbc.edu/portal/help/architecture/idfefi.pdf

https://masters.donntu.ru/2020/fknt/yakubov/library/article9.pdf 

It doesn't look like we have many prezos or papers w/ all 4 of our names, but there are a few items like

These couple of drop-e's reminded me of my favorite paper w/ the drop-e that'll be 20-years old next January.

So speaking of firmware and history, rewind to 2012 when I presented at ToorCamp on UEFI, as mentioned in http://vzimmer.blogspot.com/2012/08/one-conference-down-one-to-go.html.

On one of the earlier slides in the deck I joked about how firmware gets 'no respect' in the slide below.

Why no respect? Well, hardware teams assume firmware is 'software' and the software teams look at the firmware folks as members of the 'hardware' clan. Rodney Dangerfield provided the ideal quotation https://www.youtube.com/watch?v=ZCVR_ajL_Eo https://www.amazon.com/I-Dont-Get-No-Respect/dp/0843101938 that inspired the slide.

This is one of my favorite quotes about firmware, along with the original IA64 platform manager in DuPont who provided me https://link.springer.com/book/10.1007/978-1-4842-0070-4 in the late 1990's.

Hale & I were definitely in the 'firmware is software' camp in the 2010 prezo associated with the paper http://masters.donntu.ru/2020/fknt/yakubov/library/article6.pdf "Firmware is software so can suffer well known software ills."

So why am I talking about Rodney Dangerfield in 2023? I suspect the Youtube and TikTok generation won't recognize the allusion. The reason comes from a recent event.  Namely, when I visited  https://www.cs.washington.edu/events/colloquia/details?id=3305   https://www.youtube.com/watch?v=uL4H1ct_-dI talk this week

I couldn't help but smile at the oration during the 2:00 minute mark. 

Pat Hanrahan mentioned that when he won his Turing Award https://amturing.acm.org/award_winners/hanrahan_4652251.cfm, his congratulatory message from UW Ed Lazowska (the attendee sitting immediately in front of Pat H) read something like "Congratulations on your award for computer graphics which is the 'Rodney Dangerfield' of computer science." If Pat Hanrahan's work is the "Rodney Dangerfield" of comp sci, then I guess BIOS/firmware and my Rodney allusion from '12 is misplaced. Maybe firmware isn't even part of the comp sci software corpus :) ?

Good stuff.

What I did like about Pat H's talk included the fact that he focused on a problem no one else was looking at. It reminded me of the quote from Yan LeCun that the biggest impact in AI will not be in one of the explored domains but instead in an unvisited path. That's why I like reading old math or technology books and papers. They may not demonstrate the cutting-edge change upon the frontier of knowledge, but perhaps there are some alternate views or approaches buried that can be applied in a new context?

Some folks call predicting things 'seeing around corners.' The challenge is when you see around the corner people push back that the effort isn't 'relevant,' but if you wait until the enterprise rounds the corner you are invariably 'late' because driving change in low-level infrastructure 'takes time.'

Ghosts and podcasts

When the topic of banned function like memcpy https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/memcpy-wmemcpy?view=msvc-170 come up I have to smile since sometimes 'compatibility' (aka 'the software you choose to run) argues that this is still inherent service for software. This includes 'copymem', the UEFI variant of memcpy, as a fundamental system service in UEFI https://uefi.org/specs/UEFI/2.10/07_Services_Boot_Services.html#efi-boot-services-copymem and encouraged in https://tianocore-docs.github.io/edk2-UefiDriverWritersGuide/draft/4_general_driver_design_guidelines/44_optimization_techniques/443_copymem_and_setmem_operations.html#443-copymem-and-setmem-operations. 

So if existing software still uses these services, what to do? One option is to provide isolation to contain the damage, viz,  https://uefi.org/sites/default/files/resources/Enabling%20RUST%20for%20UEFI%20Firmware_8.19.2020.pdf

with EBC or ring's https://cdrdv2.intel.com/v1/dl/getContent/671411

or virtualization http://masters.donntu.ru/2020/fknt/yakubov/library/article5.pdf https://dblp.uni-trier.de/rec/conf/csreaSAM/Zimmer08.html

Moving forward, though, leveraging modern languages and types to enforce safety https://stanford-cs242.github.io/f18/lectures/07-1-sergio.html still feels like the right trend. Speaking of Rust in firmware https://cfp.osfc.io/media/osfc2020/submissions/SLFJTN/resources/OSFC2020_Rust_EFI_Yao_Zimmer_NDK4Dme.pdf, the GSOC produced first-class UEFI support in Rust in the form of the UEFI crate https://crates.io/crates/uefi for applications. As noted in the above prezo, the retrofit is touch for a large codebase, but a smaller unit of execution like a UEFI application, namely OS loaders, makes sense, especially in light of incidents like https://vulcan.io/blog/boothole-vulnerability-cve-2020-10713/.

And it looks like beyond all-Rust efforts like https://github.com/oreboot/oreboot, coreboot is now pursuing adding Rust support https://mail.coreboot.org/hyperkitty/list/coreboot@coreboot.org/thread/3QRO6A5BCFWCGHE7ZUCFGR7VL7AJ2XKQ/. The mention of payloads first makes sense since this is a stand-alone entity and doesn't entail the challenges of mixing non-borrow-checker aware C code with Rust. 

This Rust-as-a-payload aligns with https://universalscalablefirmware.github.io/documentation/ POL thinking, too

I cannot recall if I mentioned in the past how even chatting w/ the Google Chrome folks from Kirkland WA and VMWare Research guy from their Bellevue office during UW lunch event how they eschew the mixing and instead are targeting Rust for more stand-alone execution regimes like plug-ins for the browser and a service for the virtualization environment, respectively. 

As noted in the Stanford lecture, types can help but there has been a complaint that subsets of type-safe languages like Ada Spark are the true gold standard. That's where work on 'Ghost types' is so interesting with the Rust community. I first heard about this from Parno and followed-up on some of this work in https://arxiv.org/abs/2303.05491 https://arxiv.org/pdf/2303.05491.pdf https://github.com/verus-lang/verus and discussions with the community in https://docs.rs/ghost/latest/ghost/ https://github.com/rust-lang-nursery/wg-verification/issues/14. Ghost types will allow for no-harm source augmentation to have more verification-ready language conditions, as opposed to going to some intermediate or non-standard verification-friendly language like Dafny or F*. The important part of adding software assurance is to avoid taxing the developer as much as possible. 

As always, keep watching the Rust space for firmware.

Beyond languages, I recently received

shirt in the mail. 

That same day I was listening to the podcast on PTAB https://www.aurorapatents.com/blog/patent-wars-innovators-revolutionaries-and-the-race-to-reform on commute back from the office. My tidy cube is shown below with my friendly panopticon image nearby.

A few interesting fragments caught my attention, such as hedge funds bringing cases to PTAB. This reminded me of Trammell Hudson and his Two Sigma hedge fund https://www.twosigma.com/ and some of the rationale for research like Thunderstrike https://trmm.net/Thunderstrike2_details/ agsint folks like Apple when we chatted in person years back.

Other discussions about recusal for conflict of interest made me smile. I hearken back in the mid 90's when in my Compaq Computer Corporation new employee orientation down in Houston how I was chided not to transact stocks in 'customer, competitor, supplier,...' of Compaq. At the time this pretty much covered most companies on the stock exchanges. This early career lesson has stuck with me in that for the past 30 years working in tech I have been afraid to buy any shares outside of my employer and outside of my employer's stock program.

The other interesting podcast I caught was https://www.lawfareblog.com/lawfare-podcast-rob-joyce-nsa-director-cybersecurity. Hearing the director of the NSA casually opine on large language models and post-quantum cryptography is such an amazing change from my recollection of the NSA growing up. In my early years it was 'no such agency' typified by reading books like https://www.amazon.com/Puzzle-Palace-National-Intelligence-Organization/dp/0140067485. Quite the change.

Well, so much for banned functions, Rust, and podcasts. On to the next weekend to-do's.

Upgraded view

My last cubicle with a windows view was the south Seattle Union Station office mentioned in http://vzimmer.blogspot.com/2016/02/firmware-configuration-or-is-feature.html. Today I'm happy to be camped in a 3rd-floor cubicle in the Bellevue Washington Intel Ridgepointe office with the following view snapped earlier today

I need to be a bit wary on the sojourn across the street, though, as the https://www.geekwire.com/2023/suspect-in-alleged-stabbing-of-fellow-microsoft-employee-pleads-not-guilty-to-attempted-murder/ event was a block away.

Luckily if you go the other direction you'll find an oasis of food.

And after hours the neighbors across the floor here are a bit noisy, especially the duck.

Speaking of noisy, I noticed that https://www.scmagazine.com/podcast-episode/bts-6-vincent-zimmer went live today. I recorded this interview from a conference room on the 2nd floor of Ridgepointe. 

 

Interesting times.

Continued march of time

This week I hosted another retirement lunch for an Intel colleague Harry (middle on left-hand-size of the picture below) from the Ridgepointe office, namely the cubicle next to the 2017 denizen described in the http://vzimmer.blogspot.com/2017/10/the-march-of-time.html posting. I caught a picture of a few of the attendees at the nearby buffet, 

luckily for all omitting myself from the photo. A couple of years ago there was talk of manic recruiting, folks reveling in their 'fun-employment' between jobs, quiet quitting, etc.  Now the word 'layoff' permeates the air, with added 'quiet firing,' and folks taking the opportunity to retire. For retirement and the retiree Harry, he was on my interview team back in October 1996 and I still recall his vivid description of underwater hockey and his forays therein. Since then our paths have crossed often through BIOS, Itanium, EFI, UEFI, EDKII..... 

I still remember my first job in the early 90's. I was off-site working with a contractor when at the main campus the layoff message was delivered. When I returned to the office the next day folks had ravaged my cubicle, taking my office supplies, chair, etc. assuming I was part of the RIF. Ah, the cycle of tech.

At the lunch, the fortune cookie I selected seemed appropriate, too, for the eastside tech scene.

Grinding at tech is definitely the mood of the last few decades. Disruptions often make one more thoughtful, though. I am reminded of the quote from Elon Musk that the biggest mistake engineers make is working on the wrong problem, or the image of "Lancelot in Retirement" from Lex Fridman #345. 

Speaking of Leahy and the recent progress in RISC-V I was again reminded of the 2016 coreboot conference https://www.youtube.com/playlist?list=PLiWdJ1SEk1_AfMNC6nD_BvUVCIsHq6f0u I mentioned in http://vzimmer.blogspot.com/2016/11/conferences-forums-and-writings.html. Lots of the folks who presented have switched companies, landing at places such as Rivos, Meta, Apple, and Google. But I still giggle when I recall Ron M. mentioning that he had Andrew Waterman https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-1.html speak to the Google executives just to see the latter's expressions when Andrew would drop the F-bomb. Classic tech.

Speaking of Google in PNW and the future of tech, https://www.cs.washington.edu/events/colloquia/details?id=3264 provided some inspiration on the set of open problems to solve

and also offered an oppty to drop in on the new UW CSE building

In the firmware domain there are still interesting problems to solve, including how to better share source code across execution phases. Host firmware is strange, such as PI-based code w/ sec, pei pre mem, pei post mem, dxe pre BdsEntry, dxe post bds (the uefi phase), uefi runtime, smm. And then there are the other platform components like embedded controller (ec), bmc, soc uc's, not to mention other host firmware like slim bootloader, coreboot, u-boot, ....

And to evolve something you need to ship. It was far-sighted on part of Google to use Fuchsia on a class of Nest devices. I recall a lunch w/ MS engineer years about about the Singularity/Midori  OS fail. He claimed the focus was wrong, namely not having a memory safe driver model but instead the lack of focus on the application model. Maybe if it shipped (or provided user-land usages or deployed on MS 1st party devices?

And around Seattle I enjoyed reading Welsh's article https://cacm.acm.org/magazines/2023/1/267976-the-end-of-programming/fulltext. This work reminded me of item I studied during my late 90's UW masters on Simonyi's intentional programming https://en.wikipedia.org/wiki/Intentional_programming.

I have tried to move some of my public-facing posting from https://twitter.com/vincentzimmer to https://mas.to/@vincentzimmer but I continue to find myself replying on the blue bird. From re-tweeting https://twitter.com/acmeducation/status/1636806692519256064 

to answering questions on UEFI firmware, such as making a distinction between interfaces and implementation https://twitter.com/vincentzimmer/status/1633880381610164224

along with not being an apologist for either, with critique of the former https://twitter.com/vincentzimmer/status/1633890390041587712

And finally I reinforced Matthew Garret on his defense of some of the UEFI security features https://twitter.com/mjg59/status/1637661444270587906

with a shout-out to the corresponding embedded computing article https://embeddedcomputing.com/technology/security/software-security/understanding-uefi-firmware-update-and-its-vital-role-in-keeping-computing-systems-secure.

I also dropped into the Dasharo user group and had a chance to talk with Andrew Cooper. He brought up his concern about running UEFI runtime with hardware control-flow enforcement from December 2021 https://osfw.slack.com/archives/CV510QZ0D/p1639105740058800

I let him know that we had implemented this in the UEFI 2.10 specification as part of the UEFI Memory Attributes Table https://uefi.org/specs/UEFI/2.10/04_EFI_System_Table.html#efi-memory-attributes-table. 

The problem statement is described in https://bugzilla.tianocore.org/show_bug.cgi?id=3726

This is another example of using the community to help drive a security feature. The other notable example includes the exchange with Kees Cook on Twitter https://twitter.com/kees_cook/status/1290095780984786952 that led to https://bugzilla.tianocore.org/show_bug.cgi?id=3519 which is now required in https://techcommunity.microsoft.com/t5/hardware-dev-center/new-uefi-ca-memory-mitigation-requirements-for-signing/ba-p/3608714, namely the EFI_MEMORY_ATTRIBUTE_PROTOCOL.

Speaking of the UEFI spec, it has been interesting working across different standards bodies, including IETF https://www.rfc-editor.org/rfc/rfc5970.txt to the various UEFI https://uefi.org/specs/UEFI/2.10/ and UEFI Platform Initialization https://uefi.org/specs/PI/1.8/index.html specifications. Each has a different way to share IPR, from IETF's public https://datatracker.ietf.org/ipr/ to the UEFI forum's internal sharing. I still recall working w/ the Intel team to draft 

                                

for the PI SMM items. This cites the same SMM patent mentioned in https://www.intel.com/content/dam/develop/public/us/en/documents/a-tour-beyond-bios-launching-standalone-smm-drivers-in-pei-using-the-efi-developer-kit-ii.pdf, viz.,

This work was prior to the standards efforts. And the above document also demonstrated the type of behavior taken post-standards, namely a more public, code-first approach to new work. That document describes what has become the 'stand-alone MM' https://uefi.org/specs/PI/1.8/V4_Overview.html#initializing-mm-standalone-mode-in-sec-phase work in PI 1.8, for example.

Finally, I was hoping to reproduce the stacking image from http://vzimmer.blogspot.com/2021/01/books-and-computers.html for the last couple of Apress books, but I didn't have the energy to hunt all of the books down that have been published since 2006. Instead I opted for the 'COVID Triad', namely the 3 books that appeared between October 2020 and October 2022, viz.

https://link.springer.com/book/10.1007/978-1-4842-7939-7 https://link.springer.com/book/10.1007/978-1-4842-7974-8 https://link.springer.com/book/10.1007/978-1-4842-6106-4

That's about 2000pp that appeared on the market in the span of 2 years. Yikes.

A final curious note that also relates to this blog series is 

Someone had posted a link to my BlueHat stream of consciousness blog http://vzimmer.blogspot.com/2023/02/blue-hat-2023-and-uefi-secure-boot.html. I'm glad no one picked up on this since the HN https://news.ycombinator.com/ discussions can get pretty rough.

OK. Enough for Sunday blogging and back to Sunday work.

PS
I was saddened to hear the news of Gordon Moore's passing https://www.moore.org/article-detail?newsUrlName=in-memoriam-gordon-moore-1929-2023. Beyond Moore's Law https://hasler.ece.gatech.edu/Published_papers/Technology_overview/gordon_moore_1965_article.pdf, he provided guidance and wisdom to the industry throughout the years https://www.youtube.com/watch?v=gtcLzokagAw. My favorite image of him has to be the portrait composed of keycaps 

along with the painting

from Intel HQ alongside Robert Noyce.

26 or Anniversary.Next^11 and Wisdom of the "Age"s

True to form, as today is my work anniversary, let's follow-up on the tradition of posting a blog. Recall last year's entry http://vzimmer.blogspot.com/2022/02/25-or-anniversarynext10-and-early-22.html. When I am posed with the challenge of posting on a given day, sometimes my mind alights upon various data points observed during the earlier hours. The first thing I noticed today was https://twitter.com/osfw_foundation/status/1629217731395350529 which pointed to the longer blog posting https://blog.osfw.foundation/breaking-the-boundary-a-way-to-create-your-own-fsp-binary/.

This is penned by Subrata and describes some of the techniques to minimize the size of the Intel FSP. I appreciate the review of this work at https://www.phoronix.com/news/Google-Intel-More-FSP-Flexible, too. 

This workflow aligns with the 'path to openness' arc mentioned in postings like http://vzimmer.blogspot.com/2020/12/musings-about-firmware-cultures.html. And as always it's great having the opportunity collaborate with Subrata, such as the late '22 Apress books.

Speaking of Apress co-authors, today on the Twitter-stream I also saw https://twitter.com/UEFIForum/status/1628107294134108183

which describes an upcoming talk (now 'past' https://www.youtube.com/watch?v=BI9DMAOZR1I https://uefi.org/sites/default/files/resources/USF_Security_Webinar_Final.pdf) with Jiewen Yao, my Apress '20 https://link.springer.com/book/10.1007/978-1-4842-6106-4 co-author and long-time collaborator. 

Since these two data points are about sharing knowledge, and today's post entails a milestone on my career, I was reminded of some wisdom I received from an engineering luminary early in my career. Specifically, the posting https://twitter.com/siliconinsid/status/1628917395501813762 mentioned Robert Pease https://en.wikipedia.org/wiki/Bob_Pease.

Speaking of the wikipedia bio, I can say that Rob used 'RAP' in his personal, correspondence.

" Although his name was listed as "Robert A. Pease" in formal documents, he preferred to be called "Bob Pease" or to use his initials "RAP" in his magazine columns."

When he was he was 54 when he shared some thoughts with a 24 year old engineer.

I picked up my undergraduate degree in electrical engineering and my first job was doing embedded systems for real-time data acquisition. I was a big fan of Electronic Design magazine and an avid reader of Pease. When my job expanded from the real-time control firmware into working on the analogy-side of the sensor subsystem, I reached out to RAP with some questions.

I dug up the correspondence, with the following envelope

And my letter to Rob. I had not yet realized that writing simply is the best way to convey your thoughts. I cringe at some of the constructions below, but it definitely has my literary 'fingerprint.'

In response to this mail, I was amazed that I received a 3-page response from Pease

(1 of 3)

(2 of 3)

(3 of 3)

To me Pease was my first encounter of the silicon valley culture wherein engineers revel in helping each other and sharing knowledge. Fast forward to a few years ago and Intel. The spirit of helpfulness and wisdom were something I also observed in folks like Jim Keller https://en.wikipedia.org/wiki/Jim_Keller_(engineer). I was reminded of Keller when reading/watching the interview https://morethanmoore.substack.com/p/interview-with-jim-keller-tenstorrent https://www.youtube.com/watch?app=desktop&v=fOUB73dZXEo. This was pure Jim. And his wisdom about 'the big ball of mud' https://blog.codinghorror.com/the-big-ball-of-mud-and-other-architectural-disasters/ https://www.cin.ufpe.br/~sugarloafplop/mud.pdf and the need for modularity and re-writes echoes some of the Intel-time wisdom he relayed. At the time KKeller was probably 60, and I was much closer in age to him.

I recall a specific encounter in a conference room at the Robert Noyce building with a nearby view similar to the images from http://vzimmer.blogspot.com/2020/10/silicon-valley-innovation-and-logos.html. Jim would come into the discussions aggressive and strident, but if you had your data and could defend your position, he'd enjoin the discussion deeply.

To me Pease and Keller were similar souls, having traveled through myriad engineering projects and collected learnings that each would happily share.

Although I lack both the wisdom and charisma of the two, I try in my own small way and encourage others to be similar mentors and purveyors of their learnings in the engineering community.

PS
Speaking of my undergrad and Cornell, I couldn't help but think of Scott Galloway and his quotation " Hermes-ification of their institutions" https://www.businessinsider.com/scott-galloway-what-america-gets-right-and-wrong-college-education-2021-4 after seeing the following statistic, viz.

Coming from Texas I didn't know much about the Ivy League, but arriving in NY I learned that Cornell was the more accessible of the Ivy's, as composed to Harvard and Yale.  

Yikes.

Beyond some of the headline-grabbing quotes of Galloway, though, I would recommend reading his book https://www.amazon.com/Algebra-Happiness-Pursuit-Success-Meaning/dp/0593084195. I've heard said is that the mark of a good book is one you'd read twice. I have to admit the Algebra of Happiness is one that I've at least iterated twice.

PPS

Speaking of authors whose work I enjoy and the spirit of engineering information sharing from the valley, https://www.righto.com/2023/02/8086-interrupt.html has a fascinating write-up on 8086 interrupt support. Ken's work is more generally described in his various talks, such as https://podcasts.apple.com/us/podcast/ken-shirriff/id1488187473?i=1000506639971 and https://www.youtube.com/watch?v=TKi1xX7KKOI.This reminded me of my first file & issued patent (now expired) https://patents.google.com/patent/US5940587A/en, which used some trickery of the segments and the interrupt table to save space in the BIOS.  Good times.

PPS+

I just saw the following image 

 I had similar struggles with Java in 1997 in building the web crawler I had to build at UW for Weld's AI course mentioned in http://vzimmer.blogspot.com/2021/01/memories-from-uw-and-cornell.html. Although Larry and I had similar Java challenges in the late 90's, it appears that our respective careers took different arcs.