Inspire, develop, and guide a winning organization.
Create visible workflows to achieve well-architected software.
Understand and use meaningful data to measure success.
Integrate and automate quality, security, and compliance into daily work.
Understand the unique values and behaviors of a successful organization.
Explore our extensive library of experience reports.
An on-demand learning experience from the people who brought you The Phoenix Project, Team Topologies, Accelerate, and more.
Learn how making work visible, value stream management, and flow metrics can affect change in your organization.
Clarify team interactions for fast flow using simple sense-making approaches and tools.
Multiple award-winning CTO, researcher, and bestselling author Gene Kim hosts enterprise technology and business leaders.
In the first part of this two-part episode of The Idealcast, Gene Kim speaks with Dr. Ron Westrum, Emeritus Professor of Sociology at Eastern Michigan University.
In the first episode of Season 2 of The Idealcast, Gene Kim speaks with Admiral John Richardson, who served as Chief of Naval Operations for four years.
Weekly discussion around “Deming’s Journey to Profound Knowledge” with author John Willis.
VIRTUAL — Helping leaders succeed and organizations thrive (formerly DevOps Enterprise Summit).
Venue: Fontainebleau — Helping leaders succeed and organizations thrive (formerly DevOps Enterprise Summit).
DevOps best practices, case studies, organizational change, ways of working, and the latest thinking affecting business and technology leadership.
Is slowify a real word?
Could right fit help talent discover more meaning and satisfaction at work and help companies find lost productivity?
The values and philosophies that frame the processes, procedures, and practices of DevOps.
This post presents the four key metrics to measure software delivery performance.
September 11, 2023
For the last couple of years, I’ve been showing a word cloud at DevOps Enterprise Summit because it’s a nice and dense way to present a bunch of company names, but I knew we could do so much better.
What I’ve wanted to do for literally 5+ years is to show every company by industry vertical because one of the questions people often ask me is, “Which banks have presented?” or “Which companies that look like us have presented?” Often, I could recite many companies from memory, but it’s been frustrating that, until now, I haven’t been able to give an authoritative answer. The challenge has been categorizing hundreds of companies, which seemed like a lot of effort and was something I always put off until “next year.”
This summer, I was eager to find a problem that I could try solving with OpenAI’s API, so I decided to take a stab at using it to assemble this information.
But this post is to discuss some of the surprising results, present the data and analysis, and, of course, the list of industry verticals and companies from 1,179 talks (as of September 2023) in the DevOps Enterprise Summit Video Library.
(Also, make sure to come see all the new programming at DevOps Enterprise Summit Las Vegas this October 3-5!)
Here are some interpretations and big surprises I found in analyzing this data.
This is not surprising, given how technology-intensive the banking sector is. For instance, JP Morgan Chase disclosed that it will spend $12 billion on technology in 2023.
522. Banking: Credit Intermediation and Related (Count: 52)
ABN AMRO, ASB, Akbank, American Express, BBVA, BUPA, Banco de Crédito del Perú, Bancolombia, Bank of New Zealand, Barclays, Capital One, DBS Bank, Discover Financial Services, Fannie Mae, Ford Motor Credit, HSBC, ING, JP Morgan Chase Asset Management, KeyBank, Kiwibank Ltd., Large Bank, Leeds Building Society, Lloyds Banking Group, NatWest Group, National Bank of Canada, Nationwide Building Society, Nedbank, Nordea, PNC Bank, PayPal, Paysafe, Promontory Interfinancial Network, Reserve Bank of India (RBI), Royal Bank of Canada (RBC), SWIFT, Standard Bank, Standard Chartered Bank, SurePay, TIAA, The Clearing House, The Royal Bank of Scotland plc, The Western Union Company, Tide, Traive, Truist, U.S. Bank, US Bank, USAA, Wells Fargo & Co., Westpac, Worldpay
523. Banking: Securities, Commodity Contracts and Related (Count: 14)
Battery Ventures, Credit Suisse, Deutsche Bank, Fidelity Investments, H&R Block, Morgan Stanley, Nomura, Northwestern Mutual, Principal Financial Group, SIX Group AG, Saxo Bank, The Citadel, UBS, Vanguard Group
This one blew my mind.
It is defined as “administration, management, and oversight of public programs by Federal, State, and local governments.” I included any defense organization in this category.
Why is this a surprise? I always hear from government people that, “We’re way behind the commercial sector.” (And according to analysts, government agencies typically spend only 1-3% of their budget on technology, which puts them in the “middle of the pack” in terms of technology spend.)
But look at all the amazing case studies coming out of the public sector across the US, UK, and Europe!
92. Public Administration (Count: 43)
18F, A Radical Enterprise, Alignment Healthcare, Centers for Medicare and Medicaid Services, DISA, DWP Digital, Defense Unicorns, Department for Work and Pensions UK, Digital Health and Care Wales, DoD Platform One, FINRA, Financial Conduct Authority, Forrester Research, Government Digital Service, Government of Canada, HM Revenue and Customs, Joint Warfare Analysis Center, Kessel Run, Ministry of Justice, NAV, NOAA, NSA, National Association of Insurance Commissioners, Ordnance Survey, PeopleNotTech, Rebellion Consulting, SOMOS Inc., Swedish Migration Agency, The (Delicate) Art of Bureaucracy, U.S. Navy, UK Government Digital Service, US Air Force, US Air Force – Hill AFB 309 Software Engineering Group, US Air Force Platform 1, US Army Reserve, US Citizenship and Immigration Services, US Marine Corps Forces Cyberspace Command, US Navy, US Navy (Ret), UWV, United Kingdom Hydrographic Office, United States Patent and Trademark Office, Welsh Government
I combined the companies with NAICS codes 31, 32, and 33—they were all labeled as “Manufacturing” but spanned everything from food and apparel to computers and transportation. This is also surprising because I often hear, “We’re not a software company because we’re in manufacturing, which is the opposite of software.”
(According to analysts, manufacturing spends about the same as the public sector on technology as a percentage of budget.)
And yet, this is the third most common industry vertical.
31. Manufacturing: Food, Textiles, Apparel (Count: 6)
Adidas, Coats PLC, Columbia Sportswear, Nike, Procter & Gamble, Unilever
32. Manufacturing: Chemicals, Plastics, Rubber (Count: 4)
Continental Tires, Johnson & Johnson, Luminex Corporation, Sasol Limited
33. Manufacturing: Machinery, Computers, Transportation (Count: 29)
AGCO, Airbus, BAE Systems, BMW Group, Broadcom, Cisco, Daimler Truck North America, Ericsson, Fiat Chrysler Automobiles, Grundfos, Honeywell, Huawei, IBM, Intel, Jaguar Land Rover, John Deere, Lockheed Martin, Mattel, Motorola Solutions, NXP Semiconductors, Northrop Grumman, Ossur, Philips, Ponsse Plc, Raytheon Technologies Corporation, Siemens, Toyota, Varian, Waters Corporation
Healthcare has one of the highest technology spends as a function of revenue/budget, almost as high as banking and insurance. We have had only five talks from healthcare organizations. We should seek more of these. (If you know of anyone doing great work in this area, please let me know!)
62. Health Care and Social Assistance (Count: 5)
Babylon Health, Devote Consulting, Kaiser Permanente, Northwest Acute Care Specialists, Stanford Medicine
The graph below shows a histogram of the number of companies by industry vertical. I used the NAICS industry codes—this is the North American Industry Classification System published by the US Census. It is the successor to the SIC industry code, which was invented in the 1930s. NAICS is a six-digit hierarchical coding system that specifies industry verticals. The 2-digit form defines broad industry categories, and each additional digit adds even more nuance.
For example, the 2-digit NAICS code is “52. Finance and Insurance,” which is very, very broad. The 3-digit code is more specific: “522. Credit Intermediation and Related Activities.” This is already so specific that I didn’t even know this was banking-related, but it is one of the two main banking categories. (The other is “523. Securities, Commodity Contracts, and Other Financial Investments and Related Activities.”)
The above graph shows the 2-digit NAICS codes of presenting organizations.
You’ll notice that there are several exceptions: Some 2-digit codes were so large that I split them up by their 3-digit NAICS codes. These were 22: Manufacturing, 51: Information, 52: Finance and Insurance.
In order of frequency, the industry verticals are as follows:
Listed below are all the NAICS industry codes, along with the companies in that industry.
Find out whether your organization has presented and which of your competitors have presented too!
11. Agriculture, Forestry, Fishing and Hunting (Count: 1)
Syngenta
21. Mining, Quarrying, and Oil and Gas Extraction (Count: 3)
Schlumberger, Shell, Suncor
22. Utilities (Count: 1)
Duke Energy
42. Wholesale Trade (Count: 10)
Amadeus, Jet.com and Walmart Labs, Macy’s, Marks & Spencer, News UK, Sainsbury’s, Target, The Cambridge Satchel Company, Walmart Inc., Wayfair LLC
44. Retailers: Vehicles, Food and Beverage (Count: 3)
Auto Trader, Grofers, bol.com
45. Retailers: General Merchandise, Health, Clothing, Sporting (Count: 10)
H&M, IKEA Group, Moda Operandi, Nordstrom, Optum, Qurate Retail Group (QRG), Stitch Fix, TJX, Zulily, eBay
48. Transportation and Warehousing (Count: 10)
American Airlines, Delta Air Lines, Dutch Railways, FedEx, Hermes Germany GmbH, J.B. Hunt Transport Services, Inc, Lufthansa Systems, Maersk, Virgin Atlantic, easyJet
513. Publishing Industries (Count: 9)
AXELOS, Cox Automotive, EssentiallySports, Experian, Hearst, IT Revolution, S&P Global Inc., Thomson Reuters, Wiley
516. Broadcasting and Content Providers (Count: 3)
ITV, Netflix, STARZPLAY
517. Telecommunications (Count: 19)
BT, British Telecom, Charter Communications, Comcast, Deutsche Telekom IT, Lebara, Orange, Sky, Spirent, Swisscom, T-Mobile, Telus, Verizon, Verizon Business (formerly Verizon Enterprise Solutions), Verizon Communications, Virgin Media, Visible (Part of Verizon), Vodafone Group, Vodafone UK Limited
518-cloud. Computing Infrastructure Providers, Data Processing (Cloud) (Count: 11)
AWS, Akamai Technologies, Amazon Web Services, Equinix Metal, Google, Microsoft, Oracle, Salesforce, ServiceNow, Slack, Twilio
518-consumer. Computing Infrastructure Providers, Data Processing (Consumer) (Count: 5)
Adobe, BlackBerry, Findmypast, Intuit, SurveyMonkey
518-industry. Computing Infrastructure Providers, Data Processing (Industry) (Count: 20)
ADP, AdvancedMD, BD Biosciences, Blackboard, CDK Global, CSG, GE Digital, Ingenico, Jack Henry, Kronos, MediaMath, Mitchell International, Neustar, SAP SE, SPS Commerce, UKG, Visma, Visma Group, WirelessCar, Workiva
518-tech-lg. Computing Infrastructure Providers, Data Processing (Industry) (Count: 18)
Atlassian, BMC, CA Technologies, Datadog, Deloitte, Deloitte New Zealand, GitHub, GitLab, HCL Software, Hewlett Packard Enterprise, Micro Focus, New Relic, Nutanix, PagerDuty, Red Hat Inc., Splunk, Synopsys, VMware
519. Web Search Portals, Libraries, Archives, and Other Information Services (Count: 2)
MSN, Yahoo!
524. Insurance Carriers and Related Activities (Count: 22)
Admiral Group PLC, Aegon, Aflac, Allianz Deutchland AG, Allstate Insurance, Aviva, Blue Shield of California, CUNA Mutual Group, CareFirst, HCSC, Hiscox, John Hancock, LV=, Lincoln Financial Group, MetLife, Nationwide, Nationwide Insurance, State Farm, Swiss Re, Travelers, UnitedHealth Group, Zurich Insurance
541-tech-lg. Professional, Scientific, and Technical Services (Technology) (Count: 13)
Accenture, Accenture, NL, DXC Technology, Equal Experts, Ernst & Young, KPMG, KPMG Switzerland, NTT Data, North Highland, PricewaterhouseCoopers, TATA Consultancy Services, The MITRE Corporation, ThoughtWorks
56. Administrative and Support and Waste Management and Remediation Services (Count: 9)
Booking.com, John Lewis Partnership, Orbitz, PeopleScout, a TrueBlue Company, SPAR Business Services GMbH, Seek, TUI, Thomas Cook, Travelopia
61. Educational Services (Count: 20)
BYJU’S, CMU, Cornerstone, DevOps Institute, Eastern Michigan University, GamingWorks, Griffith University, Honorary Professor IIPP, UCL, UK, MIT, Minnesota State Digital Curricula Initiative, Pearson, Pepperdine University, The Ohio State University, UCLA, University College London, University of California, Berkeley, University of New Hampshire, Utah State University, Virginia Tech
71. Arts, Entertainment, and Recreation (Count: 9)
Bally’s Interactive, Betway Group, IGT, Sky Betting and Gaming, Songkick, The Walt Disney Company, Ticketmaster, William Hill, Win Technologies
72. Accommodation and Food Services (Count: 3)
Airbnb, Hyatt Hotels Corporation, Starbucks
81. Other Services (except Public Administration) (Count: 2)
Open Source Security Foundation, TSIA
I’ll post more about how I did this in a future blog post, but for now, here’s a quick outline.
I used the OpenAI API to make a first stab at generating a 2-digit NAICS code. Here is the prompt I ended up using for the nearly 700 companies.
Note the use of “multiple shot” prompting to get it to detect when organizations are public administration or military.
“You are the world’s best business analyst. For the list of organizations, with data provided in JSON format, study the data and description. Generate a new 300 word description of what the company does, and then determine what the 2 digit NAICS code should be, and then provide a reason. Software companies should be 51. If it is a government agency, the code should be 92. Similarly, any military organization or an agency that supports the military should also be 92. The NAICS code 92 represents the sector of public administration, which encompasses various government agencies and entities involved in the administration and governance of public affairs. This sector includes activities related to legislative, executive, and judicial functions, public finance and taxation, public safety and justice, public administration and management, and social programs and community services. The Employee Insurance Agency (UWV) fits within the scope of public administration as it is a government agency responsible for implementing employee insurance programs and providing labor-market and data services. In addition, another organization should have naics code 92 is the following organization, DISA (The Defense Information Systems Agency), because they support the US military and Department of Defense. DISA has the key function to support and maintain the information technology needs of the Department of Defense (DoD), to US national security. Similarly, Kessel Run should also be naics code 92, because Kessel Run is a division of the United States Air Force that provides software solutions to the military Please provide your answer in one JSON list, containing an array of answers. Each list element must have the following fields: – organization-name: the name of the company – description: the 300 word description – naics-code: 2 digit NAICS code answer (must be two digits long) – naics-code-desc: description of 2 digit NAICS code (less than 25 words) – reason: your reason, limited to 150 words Use only reliable sources.”
“You are the world’s best business analyst. For the list of organizations, with data provided in JSON format, study the data and description. Generate a new 300 word description of what the company does, and then determine what the 2 digit NAICS code should be, and then provide a reason.
Software companies should be 51.
If it is a government agency, the code should be 92. Similarly, any military organization or an agency that supports the military should also be 92.
The NAICS code 92 represents the sector of public administration, which encompasses various government agencies and entities involved in the administration and governance of public affairs. This sector includes activities related to legislative, executive, and judicial functions, public finance and taxation, public safety and justice, public administration and management, and social programs and community services.
The Employee Insurance Agency (UWV) fits within the scope of public administration as it is a government agency responsible for implementing employee insurance programs and providing labor-market and data services.
In addition, another organization should have naics code 92 is the following organization, DISA (The Defense Information Systems Agency), because they support the US military and Department of Defense. DISA has the key function to support and maintain the information technology needs of the Department of Defense (DoD), to US national security.
Similarly, Kessel Run should also be naics code 92, because Kessel Run is a division of the United States Air Force that provides software solutions to the military
Please provide your answer in one JSON list, containing an array of answers.
Each list element must have the following fields:
– organization-name: the name of the company
– description: the 300 word description
– naics-code: 2 digit NAICS code answer (must be two digits long)
– naics-code-desc: description of 2 digit NAICS code (less than 25 words)
– reason: your reason, limited to 150 words
Use only reliable sources.”
Shown below is the histogram of the 2-digit NAICS codes without breaking out the large categories, such as “51: Information.” (The problem here is that “51: Information” and “52: Finance and Insurance” dominate the graph. We need to break them up so we can better see the other categories. See next step.)
I then used the OpenAI API again to have it try to categorize the company using the 3-digit NAICS code.
(This is nice, but it is too specific and results in too many categories. Let’s use the 2-digit NAICS codes in most cases and selectively use the 3-digit codes for the high-frequency industries.)
Also in this step was breaking out the “518: Computing Infrastructure” categories into “Cloud Providers,” “SaaS for Consumers,” “SaaS for Industry,” and “SaaS for Technologists.”
That is the graph I showed at the top of this blog post.
Gene Kim is a Wall Street Journal bestselling author, researcher, and multiple award-winning CTO. He has been studying high-performing technology organizations since 1999 and was the founder and CTO of Tripwire for 13 years. He is the author of six books, The Unicorn Project (2019), and co-author of the Shingo Publication Award winning Accelerate (2018), The DevOps Handbook (2016), and The Phoenix Project (2013). Since 2014, he has been the founder and organizer of DevOps Enterprise Summit, studying the technology transformations of large, complex organizations.
In their upcoming book, Unbundling the Enterprise: APIs, Optionality, and the Science of Happy…
Welcome to the final installment of IT Revolution’s series based on the book Investments…
As a business leader, you know that artificial intelligence (AI) is no longer just…
Welcome to the twelfth installment of IT Revolution’s series based on the book Investments…
