AI processing is a growing industry that uses specialized hardware to perform complex tasks. AI systems require huge amounts of data and often use GPUs, which are expensive. Fortunately, there are a few ways to save power and reduce the cost of these systems. AI applications and environments can be used for a variety of purposes, from improving human interactions to helping banks and government agencies better handle spikes in claims.
Smartphones have already been taking advantage of AI for a while, but they have typically offloaded this task to the cloud. However, newer generation chips will allow AI processing to take place onboard the device, lowering power consumption and extending battery life. Here are three ways that this technology is changing the way smartphones are designed.
PIM technology integrates AI-oriented processing capabilities into high-bandwidth memory (HBM), reducing data transfer and enabling memory-bound workloads to be offloaded. Recent tests show a 2.5x performance increase and 60 percent reduction in energy consumption using this technique. This new technology is an ideal solution for companies that want to reduce their energy costs and increase their AI functionality.
AI inference engines work by multiplying information by a large array of numbers and finding the correlations between good and bad information. There are two stages in this process: the initial training process and the actual processing. Once the algorithms are trained, they produce the expected output. The final step is the mapping of the generated model to ambient data.
Another challenge is choosing the right format for representing weights, which are factors learned in training data. These weights influence a system’s prediction. For example, weights are what enable GPT-3 to generate a full paragraph from a sentence-long prompt, and DALL-E 2 to create photorealistic portraits from a caption.
GPUs are also an important part of AI processing. GPUs were originally designed to boost graphics processing but have since evolved into a vital part of AI infrastructure. RISC-V is a free and open-source instruction set architecture that’s perfect for AI processing. They also provide low-power and low-latency performance.
AI processors are also starting to make their way into consumer devices. Some of the major manufacturers have given their chips cool names, such as the Bionic Chip and Neuronal Engine. The newest Huawei phone, for example, has a NPU for AI processing. These processors can process large amounts of data, with incredible accuracy.
While the AI processor is an important part of a smartphone, it is not a stand-alone feature. In fact, most third-party AI-enhanced apps are still in development, and most of them are proprietary apps from phone manufacturers. These companies have the budget, expertise, and access to data to develop these applications.
The Exynos 9820 processor from Samsung offers powerful NPU-based AI capabilities and performance. Its NPU can process 26 trillion operations per second. Additionally, the NPU’s lightweight AI algorithms are four times faster than existing solutions and eight times lighter.
