Chips for AI: TOPS isn’t the only factor influencing performance
Manufacturers are moving fast to release improved chips to support AI applications, with ever-greater trillions of operations per second (TOPS) coming to market. However, impressive TOPS metrics are not the only factor influencing performance and end-user experience in the new world of AI.
This is according to Sudesh Pillay, executive head of iStore Business, and Lance Van Der Molen, senior solutions architect at iStore Business, who note that the mainstreaming of AI for business will have a noticeable impact on end user device performance.
Van Der Molen says: “New AI and generative AI tools are being built into the business productivity tools people use every day. Many of these are in the cloud, but we will see more and more of them residing on the end user devices. If those devices don’t have the processing power to handle gen AI, users will not enjoy an optimal gen AI experience, and their machines may become slower, hotter, and have reduced battery life as a result.”
To enable more efficient local processing of AI-enabled applications, chips with neural processing units capable of trillions of operations per second (TOPS) are coming to market. With the race on to release double-digit TOPS chips, Apple recently announced its M4 chip for the new iPad Pro built for AI and featuring its fastest Neural Engine ever, capable of up to 38 TOPS.
Pillay says: “Apple Mac has been shipping chips with neural engines since 2017, so this is part of an ongoing evolution. It should be noted though that how you measure TOPS is very subjective. It’s not just about TOPS – it’s about how efficiently the chipset talks to the wider operating system. 40 TOPS could dramatically reduce battery availability unless the entire environment was built to support that performance. Apple’s advantage is that the chip, hardware and software are all designed to work together for optimal performance, battery life and user experience.”
Van Der Molen adds: “The productivity tools that make your life easier will be the game changer, with more applications moving to run locally on knowledge workers’ devices. Older machines won’t be able to keep up if you need 40 TOPS just to run a scaled down version of an AI chat. The chip race has picked up again with the advent of ML and AI, and as chips keep pace with AI innovations, company device refreshes must keep pace with chip evolution.”
AI embedded in endpoint security will also add to the demands on end user devices, they note.
Pillay says: “Enterprises are embracing AI and have to constantly update their security, and they are doing this faster than they update the endpoints. But if you’re running the latest software on five-year-old machines, you will slow down your devices and hamper productivity. In organisations amortising laptops over five years, we’re finding knowledge workers complaining they now have to wait ten minutes for their laptops to come out of hibernation. IT knows this, but many CEOs don’t – because they have the latest devices.”
He says knowledge workers will become progressively slower as more AI tools and security are rolled out. “Organisations need to issue the knowledge workers with advanced, modern devices built to optimise AI.”
Apple Silicon has been designed to make Mac ideal platforms for AI applications, with dedicated neural engines and CPU and GPU architecture designed to boost performance, using less power.
Van Der Molen says: “Apple designs the chips, hardware and software – it’s a huge plus in terms of what you get out of the environment. For example, if the new M4 chip offers the same performance as the M2 at 40% less power, it means it can do new things without draining the battery. The new thermal designs are more efficient from a heat rating perspective and allow for thinner devices.”
If you would like to learn more about iStore Business, our products and solutions, fill out the below form and we will get in touch to guide you on the best options for your company's needs.
Click here to contact us now
iStore Business is the leading Apple Business reseller in South Africa supporting over 60% of South Africa's top 100 companies. We unlock technical complexity and ease the integration of Apple products into Enterprises.
Best for users I Best for productivity | Best for the bottom line