AI at the edge is different from AI in the data center

Today’s pace of business requires companies to find faster ways to serve customers, gather actionable information, increase operational efficiency and reduce costs. Edge-to-cloud solutions supporting AI workloads at the edge help meet this demand. Placing computing power at the edge, close to the point of data creation, makes a significant difference in near real-time use cases. Running functions such as computational algorithms and data processing on local devices, rather than moving these workloads to a cloud data center, dramatically reduces latency.

In production, locally run AI models can quickly interpret data from sensors and cameras to perform critical tasks. For example, automakers scan their assembly lines with computer vision to identify potential vehicle defects before they leave the factory. In this case, very low latency and continuous demands make transmitting data across a wide network impractical. Even small delays can hamper quality assurance processes. On the other hand, low-power devices are ill-equipped to handle heavy AI workloads, such as training the models on which computer vision systems rely. Therefore, a holistic edge-to-cloud approach combines the best of both worlds. Backend cloud instances provide scalability and processing power for complex AI workloads, while front-end devices physically place data and analytics close together to minimize latency.

For these reasons, cloud solutions, including: Amazon, Google and Microsoft play a key role. Flexible and powerful instances with purpose-built processors, such as the Intel Xeon processor family with built-in AI acceleration capabilities, can handle the heaviest tasks such as model creation. From there, the prepared models are transferred to lightweight edge systems that collect and interpret data.

Low power devices on the edge

Device manufacturers like Arduino are designing rugged, low-power edge devices. Many of them cost less than $100, and Arduino users can connect several or thousands of devices that support machine learning models. Arduino complements its devices with Arduino Cloud, supported by efficient and scalable Amazon EC2 instances. Arduino also uses the Intel Distribution of OpenVINO toolkit, which provides developers with previously published models, making the cloud-to-edge development process less labor-intensive.

For example, an agricultural company used an Arduino solution to maximize crop yields. The solution includes sensors that feed data such as soil moisture and wind conditions to edge devices to determine how much water is needed for the healthiest crops. This technology helps farmers avoid over-watering and reduces the operating costs of electric water pumps.

In another case, a manufacturer dependent on its precision lathes used sensors in conjunction with Arduino devices to detect anomalies such as tiny vibrations that signal an impending hardware problem. Scheduling scheduled maintenance is much more cost-effective for a company than an unexpected failure that stops production.

Recruitment of exceptional athletes

AiScout, an application developed by British company, is another great example of edge-to-cloud connectivity. The company’s AiScout app bridges the gap between amateur athletes who want to be discovered by talent scouts and recruiters who need exceptional team players. Historically, recruiters traveled extensively to evaluate athletes’ skills, making the process expensive and time-consuming. The players faced a different problem. Even people with exceptional skills may go unnoticed by a talent scout if they live in other countries or remote areas.

Any athlete can use the free aiScout app on their edge device – in this case a camera phone – to record themselves, showcase their technical skills and upload video. At the back, cloud instances store and present millions of athlete videos, including those with suboptimal lighting or unintended camera movement. The motion capture platform can then analyze the videos, collect performance data, and develop 3D visualizations to help recruiters. Talent scouts can then browse videos from anywhere on their devices and find athletes who will best complement their existing team.

After extensive evaluation to find the ideal cloud solution to run its application, selected a combination of Amazon EC2 instances with Intel Xeon processors and Amazon DL1 instances with Intel Gaudi accelerators instead of GPUs to train the models. According to, a selection of different Intel-based instances increased inference performance by 50 to 200 percent while reducing costs by 40 percent compared to GPU-based solutions. Today, the widespread use of the aiScout edge-to-cloud solution brings benefits to gamers and recruiters. Talent scouts, who often spent 18 months evaluating and signing players, can complete the same process in just two weeks.

Enabling digital pathology

In the healthcare space, edge-to-cloud solutions also serve important functions. Traditionally, highly trained doctors examined patients’ tissue biopsies, X-rays, CT scans and other tests for possible health problems. However, no human is perfect. Overwhelmed radiologists and pathologists reviewing hundreds of images a day may inadvertently miss a small detail that matters. Thanks to the use of artificial intelligence, doctors gain an additional pair of eyes, thanks to which they can quickly and extremely thoroughly evaluate scans. If the AI ​​detects any anomaly, doctors can devote more time and attention to the image to determine whether it requires further medical intervention. Artificial intelligence can help pathologists achieve better patient outcomes without disrupting their work.

An edge-to-cloud solution is ideal for this medical application, enabling physicians to quickly and securely share images with others on the hospital network using 5G.

Because each type of disease indicator requires a different AI model, resources like Intel Smart Edge help manage multiple trained AI models across a hospital system. The HepatoAI platform also leverages the Intel OpenVINO toolkit and OpenVINO Model Server, which help with AI processing and inference in many medical applications.

The solution enables pathologists to securely transfer image data from edge workstations to a centralized private cloud. This backend server solution is powered by Intel Xeon Scalable processors for high throughput and built-in AI acceleration. Collectively, these technologies handle enormous amounts of medical image data, so other doctors working on the network can access this information from their workstations using a secure browser. More details about this solution can be found in the article.

Getting started with Edge-to-Cloud solutions

As the use cases in this article illustrate, AI powered by edge-to-cloud capabilities can deliver tremendous value to organizations of all sizes. A comprehensive approach connects the physical and virtual worlds to help companies improve existing processes, build new revenue streams, gain near real-time information, collaborate better, reduce costs and much more.

Companies evaluating or preparing to implement end-to-end AI solutions have many issues to consider, but a few suggestions can help make the process easier.

Some companies prefer a more complete turnkey solution, while others build a custom solution from scratch. Sometimes the most significant benefits come from a combination of both approaches. Successful implementations typically start with considering the desired business outcomes and working backwards. Evaluating a proven edge-native software platform is a great starting point for any approach your organization chooses.

The use of solutions based on open standards will help integrate heterogeneous components and avoid dependence on a single supplier. An open approach can also help future-proof cloud and edge investments. If one component needs to be updated, it remains compatible with the other components of the solution. It helps to choose proven technologies supported by multiple ecosystem partners with ready-made libraries and tools for rapid AI implementation. In addition to other technologies described in this article that facilitate artificial intelligence in distributed environments, components such as 13th Gen Intel Core processors based on the Intel vPro platform can provide an excellent combination of speed and security. For visualization tasks, Intel also offers a converged multimedia platform, combining the open Intel Datacenter GPU Flex series of GPUs, Intel Xeon processors and high-speed Ethernet connections to connect components.

Cloud service providers such as Amazon, Microsoft, and Google offer enormous assistance to customers in deploying AI solutions from the edge to the cloud. They bring expertise and robust solutions, as well as security, speed and scale, to help customers profitably leverage the power of AI.

At Vision in April, Intel announced several products that expand its edge platform and silicon roadmap. This year, Intel is looking to improve AI PCs with the latest Intel Core Ultra client processors (codenamed “Lunar Lake”), enabling more AI processing at the edge. For backend edge infrastructure, the upcoming Xeon 6 “Granite Rapids” processors will include several optimizations for GenAI.

Companies and research institutions can choose different approaches to their AI solutions from edge to cloud. The planning and implementation process may take some time, but the long-term benefits are worth the journey.