Syntiant: Overcoming the Challenges of Edge AI

By Kurt Busch

As devices become smarter and more interconnected, edge AI is transforming the way we connect the real and digital worlds. From earbuds and autonomous vehicles to industrial automation and healthcare, the convergence of artificial intelligence and edge computing is driving a new era of intelligent and responsive systems, enabling faster decision-making, reduced latency and enhanced privacy.

However, the integration of edge AI is not without its hurdles. Beyond the initial hardware challenge of providing ample computational power to constrained environments, three critical obstacles must be addressed for widespread deployment: 

Scarcity of Production Models

Substantial R&D efforts in AI have generated numerous models predominantly designed for cloud-based environments, often exhibiting basic performance. While creating demonstration models can be a swift process, developing production models demands significant time and effort. Adapting these models to operate efficiently at the edge, where computational resources are restricted, presents a formidable test. Moreover, the diversity of edge devices, each with its unique computational capabilities, introduces additional complexity. Creating lightweight and efficient models adaptable to various cloud-free environments is crucial for facilitating the widespread adoption of edge AI. 

Shortage of Modeling Knowhow

Crafting models that can effectively operate within the limitations of constrained resources requires a specialized skill set. Edge devices, often constrained in processing power, memory and energy, demand the optimization of algorithms and models. The shortage of experts possessing the knowledge to design and implement AI models tailored for edge deployment poses a substantial barrier. Closing this knowledge gap is imperative to unlock the full potential of edge AI, broadening its accessibility to a diverse range of applications. An efficient and adaptable Modeling Platform, designed to create production-quality models, represents one solution to empower a broader community of engineers with expert-level performance.

High Diversity of Applications

Edge AI finds applications across diverse fields, such as healthcare, manufacturing, smart cities and autonomous vehicles. The difficulty stems from the varied requirements and constraints of these applications. Designing a one-size-fits-all solution for edge AI proves impractical due to the unique demands of different use cases. Customization and adaptation of models for specific applications are essential, making it challenging to establish standardized frameworks. The deployment of a versatile edge AI Modeling Platform, easily tailored to diverse applications, ranging from simple sensors to large language models (LLMs), is pivotal for realizing the full potential of edge AI across various industries.

Making Edge AI a Reality

Before Syntiant entered the market in 2018, running machine learning at the edge was inconceivable. Artificial intelligence was relegated to the cloud, unconstrained by area and power. Syntiant first tackled these fundamental challenges with custom silicon delivering best-in-class performance, while meeting size, power and cost constraints. Syntiant’s Neural Decision Processors are at least 100x more efficient with typically 10x the throughput compared to existing low-power MCU solutions. Using at-memory compute and built-in standard CMOS processes, Syntiant’s devices directly process neural network layers without the need for any secondary compilers, which shortens time to market and offers unprecedented performance for low-power solutions.  

Syntiant also provides compute-efficient software solutions with proprietary model architectures that enable world-leading inference speed and minimized memory footprint across a broad range of processors. Capable of running on most hardware platforms including CPUs, GPUs, DSPs, FPGAs, and ASICs, Syntiant’s edge AI algorithms are being deployed in security and IP cameras, 360/VR cameras, video doorbells, video conferencing systems and other use cases.  

For teams seeking to develop their own algorithms, the Syntiant Modeling Platform provides extensive augmentation, rapid iteration and quality assurance, significantly expediting the deployment of production models across platforms spanning from DSPs to GPUs. Addressing the high diversity of applications, the Syntiant Modeling Platform supports various modalities, including audio, vision and radar.

With Syntiant’s technology, many machine learning workloads using high-powered application processors or cloud servers can now run in a low-power, always-on domain at the edge of the network. These deep learning solutions solve critical problems on compute-constrained embedded devices, making edge AI a reality, bringing orders of magnitude greater neural compute than previously possible. 

Kurt Busch is CEO and a cofounder of Syntiant.