Qualcomm Unveils New AI Chips for Smartphones and PCs

Qualcomm, the US chip giant, made an exciting announcement on Tuesday, unveiling two new chips specifically designed to power artificial intelligence (AI) software on smartphones and personal computers. These chips are capable of running large language models, which have been making waves in the tech industry, without requiring an internet connection.

The interest in AI applications has skyrocketed since the release of OpenAI's Stable Diffusion image generator and ChatGPT chatbot late last year. These "generative artificial intelligence" applications demand significant processing power and have primarily relied on powerful yet power-hungry Nvidia graphics processors.

Qualcomm's new chip offerings consist of the X Elite chip for personal computers and laptops, as well as the Snapdragon series 8 Gen 3 chip for high-end Android phones.

The processing speed of the Snapdragon chip in handling AI models could spark a new feature competition among high-end Android devices, including brands like Asus, Sony, and Apple's iPhone, which introduces new AI features each year.

According to a Qualcomm executive, the latest Snapdragon chip performs AI tasks much faster than its predecessor, reducing the time needed to generate an image from 15 seconds to less than one second.

Alex Katouzian, senior vice president of Qualcomm’s mobile business, stated that while people currently prioritized CPU speed, memory capacity, and camera quality when buying a phone, in the next two or three years, they would increasingly consider the artificial intelligence features offered.

The AI boom has significantly impacted Nvidia's stock price but has largely bypassed Qualcomm. Despite Qualcomm's large shipment of smartphone chips, the company has included an artificial intelligence section called NPU since 2018.

Qualcomm's NPU has been utilized to enhance photos and other features. Now, Qualcomm claims that its smartphone chip can handle larger AI models used in generative AI, some of which contain up to 10 billion parameters. However, this capacity still falls short of the largest AI models, such as OpenAI's GPT3, which encompasses approximately 175 billion parameters.

Qualcomm executives assert that if the chip possesses sufficient speed and memory, these types of AI models can run directly on the device. They argue that running large language models locally makes more sense than relying on cloud-based processing due to its improved speed and privacy. Qualcomm also states that its chip can run a version of Meta's Llama 2 model and hopes that its customers, smartphone manufacturers, can develop their own models as well. Additionally, Qualcomm is actively developing its own AI models.

Qualcomm showcased a device running the free Stable Diffusion AI model, which generates images based on text prompts. It also demonstrated the ability to use AI to expand or fill in parts of a photo.

In the previous year, Qualcomm's second-generation chip successfully processed the same model but took 15 seconds to generate an image of a cat sitting on a beach. This year, Qualcomm's new chip can accomplish the task in half a second. Katouzian believes that this significant improvement will greatly enhance the response speed of AI applications like personal assistants.

Qualcomm suggests that future applications, such as personal voice assistants, can utilize the device's onboard AI model for simple queries, while more complex questions can be sent to powerful cloud-based computers. This is why Qualcomm collaborates closely with Microsoft to optimize its chips for AI software.

Katouzian explained that the more these devices were utilized for running AI functions, the less they relied on Microsoft Azure, which typically handled expensive inference functions. He said that now, these tasks could be offloaded. He also added that in hybrid scenarios, shifting cloud loads to edge client devices would bring huge benefits.

Furthermore, Qualcomm states that its top smartphone chip this year, the Snapdragon series 8 Gen 3, will debut in "high-end" Android devices priced above $500 from brands like Asus and Sony in early next year. Eventually, the features of high-end chips will trickle down to other devices.

Qualcomm's latest addition, the X Elite chip for personal computers, is based on the Arm architecture, directly competing with Intel's x86 chips for laptops and desktops.

The X Elite chip incorporates technology from Nuvia, a company acquired by Apple, and currently embroiled in a legal dispute with Arm. Laptops powered by this chip are expected to hit the market in the middle of next year. Qualcomm claims that its chip, utilizing the Oryon core, outperforms Apple's M2 Max chip in terms of performance while consuming less power.