t’s common nowadays to see people discussing what they enjoy doing with their chatbot. Try it. I was discussing some of my favorite literary works with my Snapchat AI bot (myai), and my bot recognized that I accidentally combined lines from two different authors. Other people like using their AI bot to play games like tic-tac-toe, hangman, or picture games. AI continues to evolve to the point that it can benefit almost any application: new artwork, new rap songs (by C3PO), new stories (the fury Pi creature), robotics, security, land, sea, the latest AI flown and battle exercise fighter jet [1], surgery, stroke research, new medication, therapy, customer support, and to be honest, practically anything at all touched by technology. On the surface, one would think that AI is real-time, and presently that is often the case. Conversely, various mobile or workstation image processing applications take some time during data exchange for process and retrieval in the cloud. That is especially true in the learning then trial stages.
Artificial general intelligence (AGI) is being developed with the goal to perform just as well or better than humans in general cognitive tasks, as opposed to narrow AI intended for specific tasks. Large language models (LLMs) are, in many perspectives, an emerging part of AGI [2]. As the complexity of AI tasks evolve, hardware and software architecture continue to evolve in order to approach or support real-time AI.
With cloud AI, training and inference are handled in the cloud. Cloud AI has drawbacks due to increased amounts of data exchange, latency in data transfer, and security issues from network attacks [3].
Edge AI entails bringing computational resources geographically closer to the end user. This is analogous to a CPU with respect to its different levels of cache memory. Edge AI therefore performs the processing and implementation of learning algorithms locally on the hardware. Although that reduces associated latency, the network bandwidth delay product still remains as AI tasks become more complex, because greater amounts and types of data take more time to access and transfer. Even in the case of optimized AI models, new situations arise, requiring the update of training data and model exchange with the cloud AI, in order to update AI inferencing. At the same time, the edge device needs to process network packets and pass the data through various AI system component hardware and software layers. This matter makes the edge AI system components become complicated and highly specialized, and detracts from the effort of a truly real-time AI system.
Various integrated circuit AI chipsets and platforms have emerged [4], each with their own unique approach to efficiently process AI algorithms with neural networks, learning, and inference. Such AI chips accelerate their applications, reducing computation time and hence power consumption. Some approaches utilize resources in tens to hundreds of processor cores and dozens of gigabytes of internal memory, made possible through the latest scale integration. Other chipsets are similar in computational scaling for tensor manipulation in deep learning, but do so through increased processor speed by orders of magnitude compared to competing parts, and through use of instruction sets optimized for tensor manipulation. There are also solutions that perform in-memory computing, which reduces latency to promote efficient execution of LLMs.
Inevitably, there are design trade-offs when considering new chipsets. Some key trade-offs are the cost, adaptation to or integrating with present equipment, and planning around estimated product lifetime so as to deter rapid obsolescence. During economic booms, the component market suffers from part shortages. The chip market has always been cyclical, leading to availability issues.
SeaFire offers another option for AI platform solutions. It is consistent with modern devices operating at high speed yet maintaining power efficiency. It differs from other platforms in that it is a new type of offload engine technology coupled with artificial intelligence. At the same time, SeaFire technology provides a modular, scalable AI that is portable across edge hosts and a multitude of FPGA parts. Such a model is less restricted during chip component shortages, given that the underlying IP. may be ported to different vendor families or different vendors altogether.
The SeaFire engine offloads packet processing and makes data available at line rate, with zero operating system latency. This technology uses a new type of intelligent learning, similar to neural networks but modeled to be more conducive to a network environment, where AI operations are also performed at line rate. When the SeaFire engine is coupled with the edge device over PCI express, AGI is capable of real-time. Another advantage of the SeaFire engine is that it contains portable FPGA core I.P., usable as-is or expandable by the chip or systems developer for other AI functions and algorithms. SeaFire edge AI technology has run on networks in multiples of 10Gbps, and exceeds 100 Gbps.
Chipset and platform vendors each have a unique way of providing AI solutions. Newer systems implement technology that accelerates AI while increasing power efficiency. At the same time, there is a push for improving AGI performance through edge AI. As network speeds increase, along with faster deployed equipment, both cloud and edge AI will evolve, as well as the required supporting hardware. During such evolution, the economy will continue to be cyclical, impacting AI chip and hardware platform availability. Solution providers will have an opportunity to put forth AI systems that are scalable, modular, quickly updated, and robust to a cyclical economy.
PUBLISHED IN CIRCUIT CELLAR MAGAZINE • DECEMBER 2023 #401 – Get a PDF of the issue
J.M. Awach holds a BSEE from UMass Lowell and MSEE from Texas Tech. He has 33 years of experience in software and hardware, with designs produced in hundreds of thousands of circuit boards and tens of millions of ICs. Mr. Awrach may be reached by request when visiting www.seafire.com
