“Technology that will become the worldwide standard for machine intelligence compute”

Founded: 2016, Bristol, UK

Category: Artificial Intelligence, Hardware, Machine Learning, Semiconductor

Primary office: Bristol, UK

Core technical team: Bristol, UK

Status: Private

Employees:101-250* *TechCrunch says 450 as of July 2020

 

Amount raised:USD$460 million (5 rounds – February 2020)

OVERVIEW

  • Graphcore’s strategy to rapidly scale was based on redesigning the infrastructure that supports the processor-intensive artificial intelligence (AI) and machine learning (ML) workloads, attracting a R&D heavy team to produce commercial-ready systems that outperform incumbent systems (who have invested in advancing existing technological approaches), and then working with strategic partners with large existing customer bases to provide channels to market
  • Graphcore sells hardware designed to support the next generation of AI applications. They provide servers and server blades powered by their proprietary silicon chips, hardware, and software to enterprises, cloud service providers and academic institutions.
  • Distinct competences: The company is distinct in its approach to designing hardware and supporting software that is built specifically for the workload required by AI processing tasks. They excelled at bringing to market a new and integrated approach to silicon chips, the hardware platforms that house the chips, and the software that optimizes the use of the chips for AI training and perception workloads
  • Origin and evolution of offers and distinct competences
    • Graphcore’s understanding of machine intelligence hardware requirements comes from its close partnerships with the AI community – across academia, institutions and corporations.
    • The company’s founders Nigel Toon (CEO) and Simon Knowles (CTO and architect of IPU design) were partners in their previous venture, Icera, that was acquired by Nvidia in 2011
    • Demand for AI specific processing power is increasing but previous designs were limited in due to power laws, so the need for a reconceptualization of the chips that power the machines that do parallel processing of vast amounts of data was required
    • Nigel Toon, CEO and co-founder of Graphcore, said: “Machine intelligence marks the start of a new age of computing which needs a radically different type of processor and software tools. This new, fast growing market creates the opportunity for Graphcore to build a major global technology company that can help innovators in AI achieve important breakthroughs.”
  • Creator of next generation AI processors (called Intelligence Processing Units – IPUs) and software

PERFORMANCE METRICS

  • Company valuation: USD$1.95 billion (2020)
  • Estimated Revenue: USD$10 million (2020)

ACHIEVEMENTS

  • Over 100 organizations using Graphcore’s hardware in some shape or form (June 2020) resulting in revenue of around USD$10 million (claimed to be rapidly increasing)
  • Graphcore unveils new GC200 chip (built using 7nm technology) and the expandable M2000 IPU Machine that runs on them (July 2020)
  • Microsoft machine learning scientist Sujeeth Bharadwaj gave a demonstration that showed the Graphcore chip could do in 30 minutes what it would take five hours to do on a conventional chip from Nvidia (May 2020)
  • 2019 highlights
    • Moved from development to a full commercial business with volume production products shipping
    • First full general availability release of the Poplar® software platform, a complete SDK (software development kit) for developers to run AI models on IPU products quickly and easily
    • Microsoft Azure IPU-Cloud open for customers (launched November 2019)
    • Production launch of Dell Technologies DSS8440 IPU Server for enterprise datacenter customers in November 2019
    • Launch of IPU-Bare Metal Cloud service in partnership with Cirrascale
    • First public customers announced including Microsoft, Citadel Securities, Carmot Capital and Qwant
    • Starts Fiscal 2020 with over $300m in cash reserves

Sells

Sell to organizations that wish to increase their machine learning (ML) compute power by offering servers, PCIe server blades, and cloud servers that all use their proprietary silicon chips, hardware system design (specifically how they handle memory), and their Poplar software (designed in parallel with the silicon and platform)

  • Silicon chips :The Colossus MK1 IPU and the Colossus MK2 IPU
  • Servers:
    • IPU Server4: Graphcore 740 IPU Server (a 2U rack-mounted chassis, with two Graphcore C2 PCIe IPU Cards, each of which has two Graphcore Colossus MK1 GC2 IPUs served by 2x Intel® Xeon Host CPU);
    • IPU Server16: For enterprise datacenter customers (8 Graphcore C2 PCIe cards connected with high-speed IPU-Links™ in an industry standard OEM system made by Dell (The Dell DSS8440 IPU Server))
    • IPU-M2000: Stackable (up to 8) modular blade for server racks that each have 4x Colossus MK2 GC200 IPUs, Quad Core SoC, high efficiency cooling system, 2.8 Tbps fabric, and 2x DDR4 DIMMs
    • IPU-POD64: Server rack hardware designed to tightly stack 16 IPU-M2000s. Can connect 1000 together.
  • Cloud: Microsoft Azure (IPU preview for customers focused on developing new breakthroughs in ML); Cirrascale IPU cloud (bare-metal cloud service and Dell IPU servers to buy)
  • Poplar software: Has graph libraries, graph compiler (creates execution model), and graph engine (handles execution) along with APIs between the software and the hardware, and SDKs between the software and existing ML frameworks *It is unclear whether the software has a cost or comes with hardware.
  • Open source: code for PopLibs libraries, TensorFlow for IPU and PopART

Channels

  • Partnerships
    • Dell – strategic investor as of 2016, co-produced IPU server DSS8440 for sale
    • Cirrascale – launched IPU Bare-Metal Cloud
    • TensorFlow (developed by Google Brain) – co-created API for Poplar software for deep compatibility
  • Free product or product sold at a discount to attract customers
    • Open source libraries developers can use
  • Third party digital platforms
    • Microsoft Azure cloud
    • Cirrascale cloud
  • Communities
    • GitHub
    • Developer community
    • AI/ML conferences and events around the world
  • Social media
    • YouTube – lots of videos of c-suite explaining tech; interns working at company; explainer vids

Competencies

  • Technology
    • Able to design cutting edge hardware engineered to be one full solution – from chip to memory to platform to drivers to software. The tech is purpose built for AI/ML applications
    • Figured out how memory can be accessed much quicker than going off of the chip to a computer’s main memory, which is still the approach of Nvidia’s latest GPUs (main competitor)
  • Marketing
    • Great at explaining the tech
  • Ecosystem
    • Able to play with others and bring together the actors that specialize in different layers of the AI/ML ecosystem to ensure that the hardware to software to deployment is all integrated
    • Able to secure and coordinate high-quality chip manufacturer TSMC (Taiwan) to manufacture the silicon chips using nanotechnology and the other suppliers, high-tech components required for their machines

Distinct AI Features

Type

  • Neural networks (e.g., natural language processing, probabilistic modelling, computer vision, recommenders and more)

  AI use

  • Advance the field
  • They have focused on AI applications, so they redesigned the hardware, how it handles memory exchanges, and software that optimizes the hardware for the purpose of powering AI (where GPUs are still used in multiple contexts)
  • Supports new levels of scalability for hardware that will enable advances in applications by those wanting to push boundaries
  • Support service providers
    • Strategic partners benefit from new hardware capabilities that they can offer to their clients (e.g., cloud)
  • Enable resource integration between service providers and beneficiaries
    • Poplar software SDK enables existing ML frameworks that developers are using to work seamlessly with new chips
  • Support beneficiaries’ well-being (e.g., end-customers)
    • Expands capacity for developers to push boundaries/science
    • Algorithmically designed pastel colouring of plastic shell on hardware is visually appealing and unique for each unit

AI useRate of return on customer’s investment to make AI work

Immediate:

  • Less cost for equal compute power
  • Software and hardware designed in parallel makes solution more seamless

Long term:

  • Promise of significant leap in processing power for AI and ML compute which will enable new breakthrough innovation to emerge
  • Significant scalability and lower cost (10x-20x lower) means more companies will have access to ML compute power, including on-premise and on the cloud

Databases

  • PopLivs Graph Libraries – a complete set of libraries, available as open source code, that support common machine learning primitives and building blocks
    • Over 50 optimized functions for common machine learning models
    • More than 750 high performance compute elements
    • Simple C++ graph building API
    • Implement any application
    • Full control flow support

  • Uses external benchmark databases for testing such as:
    • BERT (training for natural language processing)
    • EfficientNet (inference for computer vision)
    • Deep Voice (training for text-to-speech (TTS))
    • ResNeXt-101 (inference for computer vision)
    • ResNeXt-50 (training for computer vision)
    • MCMC Probabilistic Model (training for probabilistic modelling)
    • Variational Autoencoder (VAE) model (combines MCMC and Variational interence (VI)) (training for probabilistic modelling)
    • Netflix public dataset (training for recommenders)

Quantum Computing

  • Believes it will be a 10 to 15-year window before quantum or molecular computing comes along, a trajectory that might pose a lot of challenges for smaller startups trying to build in that area against large companies such as IBM

Resources

Assets

  • Industry reputation of success and expertise of two co-founders who previously exited to Nvidia (now main competitor)
  • Significant talent in software, silicon, and hardware engineering
  • Integration with:
    • TensorFlow (Google)
    • PyTorch (Facebook)
    • PaddlePaddle (Baidu)
    • ONNX (open neural network exchange)
  • Connects to existing standard 3rd party infrastructure
    • Docker (containers)
    • Kubernetes (orchestration)
    • Microsoft Hyper-V (virtualization and security)
  • Full developer environment that provides documentation, self-serve tutorials, videos and open-source code
  • Analysis tools that help developers gain a deep understanding of how applications are performing and utilizing the IPU
  • Partnerships with enterprises that provide access to customers/users

 

Processes

  • Product development (hardware, software, silicon)
  • Coordination of multiple technical teams to produce a precise and highly integrated full solution (hardware and software)
  • API creation for many different types of technology
  • Co-creation capabilities with external partners
  • Fundraising

Priorities

  • Attract the very best software, silicon and hardware engineers
  • R&D investment – Major investment in 2020 to more than double headcount in R&D
    • Major expansion of engineering centers in Bristol, UK HQ and Oslo, Norway and of Palo Alto, USA sales and support office
    • Opening of Beijing, China sales and support center; Cambridge, UK engineering center; and Hsinchu, Taiwan operations facility
  • Scale up production
  • Build a community of developers around Poplar software platform
  • Drive extended product roadmap
  • Establish support teams closer to customers