Onboard artificial vision capabilities in public transportation are critical to ensuring the safety and well-being of passengers and personnel, as well as for maximizing the overall customer experience. Now, AI-enabled Deep Learning is bringing new possibilities to the attention of rail, tram and bus operators. How can embedded developers cost-effectively maximize the potential of AI for their onboard applications and what are the key considerations when accelerating neural networks? This blog post discusses the design considerations and challenges facing embedded systems designers when developing AI-enabled vision applications for deployment onboard trains and other public transportation vehicles. If you would like to read the full white paper, click here.
EXECUTIVE SUMMARY
In the new age of the connected train – or bus or metro – operators, passengers and staff can now have greater peace of mind, thanks to increasingly sophisticated onboard artificial vision systems providing real-time monitoring and alerting as well as live, artificial intelligence-enabled video analytics.
With these developments, embedded systems designers and developers often prefer to run AI algorithms on the edge to reduce data streams going to the Internet and cloud, especially in mobile onboard applications where the wireless bandwidth, data throughput and data transit costs are key considerations. This is why onboard AI-enabled systems need efficient inference hardware with low power dissipation (<10 W) and the ability to function reliably in temperature ranges of -40 °C/+70 °C while also being able to withstand continuous vibration.
This blog discusses the design considerations and challenges facing embedded systems designers when developing AI-enabled vision applications for deployment onboard trains and other public transportation vehicles. The critical role of neural network acceleration is examined for delivering the full potential of Deep Learning for systems performance and functionality. This is followed by a brief overview of Kontron’s latest family of Intelbased rail-certified embedded industrial PCs featuring neural network acceleration for public transportation.
ONBOARD AUTOMATED VISION: A CHANGING MARKET
CCTV on trains and other public transport is no longer limited to passive onboard recording. To ensure maximum efficiency as well as passenger and staff protection, transport operators and their onboard staff now expect the support of real-time vision solutions. Moreover, the advent of driverless trains, buses and metros and fewer onboard staff will mean increased surveillance functionality becomes an even more pressing requirement.
Cameras are already deployed onboard trains as well as other public transportation vehicles for supporting a wide range of applications: passenger counting, abandoned/lost luggage detection, monitoring risk situations and passenger behaviour. Crowd monitoring on station platforms is another important application.
Now, however, in response to ever-growing security threats and the ongoing need to offer exceptional passenger experience, more powerful and sophisticated surveillance systems are required, for empowering staff to manage and respond to events or incidents as or before they happen. Such systems are also necessary for providing effective support in investigations with the use of powerful AI video analytics.
Privacy is a fundamental requirement to achieve: all biometric and personally identifiable data must be fully anonymized at the source. Then, stored data and transmitted data are encrypted using state-of-the-art algorithms powered by on board hardware accelerators.
Consequently, public transportation operators are looking to deploy the latest generation of real-time AI-enabled vision solutions leveraging the potential of Deep Learning Neural Networks. With this, embedded designers must specify and implement fit-for-purpose ruggedized embedded vision systems comprising the optimal combination of compute and video graphics capabilities, throughput performance, and digital security.
Typically, these must be contained within ruggedized, highly compact, fanless solid-state industrial PCs that can balance power consumption, cooling capacity, cost and system reliability. This is where increasingly powerful neural network accelerators, whether integrated with the CPU or not, are becoming more critical to enabling extremely high-speed processing while reducing dissipated power.
HARNESSING THE POTENTIAL OF NEURAL NETWORKS
Motherboards and embedded PCs used for enabling onboard artificial vision equipment must deliver real-time processing, data analytics, strong video graphics capability and low power consumption – all in a small footprint. For onboard visual surveillance applications, embedded PCs using the latest x86 based processors are particularly well-suited, enabling flexible solutions when combined with the varied choice of form factors available.
With the advent of Artificial Intelligence technologies designers must now look to cost-effectively harness the object recognition capabilities of neural networks – the algorithms that perform the clustering and classification of data in Deep Learning applications – to provide accurate alerts on a range of events or incidents including, for example, suspicious/unattended baggage and overfull carriages.
There is a need for speed as well as accuracy when considering the processing ‘hierarchy’ governing object detection, classification and tracking: from the data collected via camera sensors, GPS and perhaps inertial sensors, to their passing through vision processing algorithms for enhancement and filtering, before being sent to the neural network. To do this accurately and efficiently the system will need to “experience” tens of thousands of different scenarios in order to ‘train’ the software to recognize and distinguish between a multitude of different scenarios, including facial emotion recognition to detect potentially distressed or violent passengers. While simulation and modelling can help, much of the learning will come from day-to-day onboard experiences, to allow ongoing improvements concerning the accuracy of real-time analysis, reporting and alerting.
Fortunately, the growing availability of increasingly affordable dedicated network accelerators can now help achieve the optimal balance between performance, power usage, system cost, size, and flexibility. By optimizing the management of neural workloads through rapid processing, clustering and classification of data, neural network accelerators help facilitate unprecedented Deep Learning ‘intelligence’ for the latest generation of onboard embedded video surveillance systems. In addition, by offloading the CPU, they help reduce overall power consumption and heat dissipation. This is of key importance when considering low profile and ruggedized embedded PC designs are now perquisites for ensuring reliability in systems typically deployed in harsh, space-constrained environments.
For example, the Intel® Movidius™ Myriad™ X vision processing unit (VPU), features a dedicated neural network compute engine, offering 1 tera-operations per second (TOPS) of dedicated deep neural network (DNN) compute. This directly interfaces with a high-throughput intelligent memory fabric to avoid any memory bottleneck when transferring data.
EFFICIENT NEURAL NETWORK ACCELERATION
When designing onboard vision systems for use on public transportation, there are essentially three principal components involved for ensuring efficient neural network acceleration:
Network Training
This first step is the training phase of the neural network. This is to determine the weight coefficients of each neuron of the network by performing the necessary inference calculations. It entails applying each video stream image to the input layer of the neural network, propagating the calculations through the intermediate layers, and finally receiving the output of the last layer of neurons. Only the inference deployment phase requires optimization on the actual embedded platforms being deployed onboard. Therefore, the training phase of the neural network can be achieved offline on servers in the cloud using popular frameworks such as TensorFlow, Caffe, and PyTorch. Here, the model consisting of the interconnect topology of the neurons and the weights for each neuron input connection is defined by selecting an appropriate existing neural network and training it. This defines the optimal neuron weights based on analysis of several thousands or millions of images with known expected results. Calculation can be done in single precision 32 bits floating point, 16 bits floating point or 8-bit integer, depending on the precision required to apply the model versus the time and dissipated power required to perform the calculation. The performance is measured in frames per second, based on the time it takes to apply a new image extracted from a real-time video to the neural network.
Network accelerators
The second step concerns the type of neural network accelerator to be utilized in the embedded system. This is demonstrated when using the ResNet-50 standard benchmark for comparing inference execution times and associated CPU workload.
The CPU is a Intel® Xeon® Processor D-1539 with 8 cores providing 1.6 GHz and a TDP dissipation of 35 W. The optional accelerator circuit is an Intel® Movidius™ Myriad™ X with a wattage of 2.5 W. As shown, the Intel® Xeon® Processor D-1539 alone is twice as fast as the accelerator, but to achieve this, the main CPU is loaded at 100 %, and dissipates at 35 W. However, with the accelerator, the CPU load is negligible and the overall power for CPU plus accelerator is five times lower. However, there are various accelerator options but there is no one size fits all:
- Use a vector engine inside the CPU: For example, the Intel® Advanced Vector Extensions 512 (Intel® AVX512) inside the latest-gen Intel processors includes dedicated 512 bits vectors instructions to handle int8 and fp32 specialized operators. Called, VNNI (Vector Neural Network Instructions), these instructions increase processing performance compared to regular instructions and may be sufficient depending on the workload. However, while the CPU is busy running those loops of vector instructions on all processing cores, it cannot process other demanding tasks such has crypto instructions dedicated to communication security, or the main applications.
- GPU, either integrated or discrete: By nature, the GPU is a highly parallel structure able to run a diversity of workloads based on parallel languages such as CUDA or OpenCL. However, power dissipation is often not as efficient as running a neural network over dedicated accelerator circuits, so therefore GPUs are perhaps better suited to the training phase or to high-end image processing applications. For onboard surveillance systems in confined spaces, one does not want system cost, temperature, size and performance to be limitations.
- VPU (Vision Processor Unit): This is less programmable than a GPU when a custom processing is needed, but more efficient when applied to neural networks and totally dedicated to processing video images with neurons. VPUs take the load off the CPU by assigning tasks to an application-specific coprocessor for object, facial analysis and machine learning functions using less heat and power than a GPU. It will enable low powered systems to run inferences trained with TensorFlow and Pytorch frameworks for object and face detection and machine learning using only 2 to 3 watts of power, therefore generating significantly less heat than typical GPUs which can use perhaps 75 watts of power and require active cooling to operate effectively. Therefore, ruggedized embedded computers with VPUs can offer efficient, reliable low power platforms with a low thermal and power footprint enabling vision processing to move further out on the edge and stay protected within compact fanless industrial enclosures. Because of the low cost and accessibility of a VPU, it is also highly scalable as more compute is added throughout the application. There is also the flexibility to process more data onboard at the edge before transmitting to the cloud thereby reducing the data transit costs associated with centralized cloud computing.
- Use a FPGA: While more versatile than the other options, this approach frequently creates overheads in developing and maintaining the code, both on the FPGA and the host CPU. For running a standard neural network, this might prove too complex compared to one of the previous options.
Inference Framework
Finally, there is the inference framework. This phase can be particularly complex. However, all Intel devices capable of calculating inferences are supported over a single inference framework, the Intel Distribution of OpenVINO toolkit. With this inference framework, trained neural networks are converted and optimized depending on the target hardware, whether it is a CPU, a CPU with vector acceleration, a CPU with an integrated VPU or GPU, a CPU hosting a dedicated external accelerator, or a FPGA.
With a powerful and dedicated API supported by the OpenVINO™ toolkit, only a few lines of code of Python or C++ are necessary to acquire successive images from a video, scale them appropriately, do some early classic image processing (light, contrast, resizing…), run through the network, and finally rebuild a video stream annotated with detections performed by the neural network.
One of the major benefits of the toolkit is its ability to optimize AI inference performance on any Intel-based platforms, without changes to the hardware. Later, during the application design, designers can decide whether to include external hardware depending on the actual total workload and overall power dissipation.
KONTRON TRACe™: EN50155 BOX PCs WITH NEURAL ACCELERATION
Kontron is deeply involved in helping OEM and SI customers take advantage of the new fast-emerging ‘Transportation 3.0’ market opportunities including next generation onboard intelligent vision capabilities. The company’s next generation of secure off-the-shelf TRACe™ EN50155-certified transportation box computers now include models with built-in VPU network accelerators. These platforms support the OpenVINO™ inference framework from Intel® with acceleration based on Intel® Core™ i7 or Quad Core Intel Atom® CPUs. There is also the option of selecting motherboards with accelerators directly integrated with the latest CPUs from Intel including Intel® 11th Gen Core™ and Intel Atom® x6000E processors.
KONTRON TRACe™ MICRO-CLOUD PLATFORM WITH GENETEC™ SECURITY CENTER
Designed for harsh environments, the Kontron TRACe™ V40x is fully compatible with the latest available video technologies and offers a modern software deployment solution. By leveraging Genetec™ Security Center in a virtual machine, the Kontron TRACe™ V40x provides all the support necessary for a wide range of industry-leading IP cameras and CCTV equipment through one unified IP security platform.
Genetec™ Security Center allows the deployment of industry-leading video analytics; integration with vehicle telematics and other onboard systems; as well as data correlation and decision support. Crucially, this system allows first responders – the authorities or law enforcement agencies – to be alerted remotely of developing situations and react accordingly from a single, trusted data source.
Genetec™ Security Center also unifies operations of on-board rolling stock systems to wayside locations, thanks to features like automatic video transfers. This means thousands of cameras in fleets of trains, buses, and metros can be controlled and monitored along with all equipment located in stations or depots. This allows for information to be easily shared and actioned across entire fleets and transit networks. Together, the Kontron TRACe™ V40x and Genetec™ Security Center provide a unified platform from which data from transit vehicles can be collected and centralized alongside other transit resources. Leveraging the integrated virtualization capability of the TRACe™ V40x, applications can run the Genetec™ environment on Windows and other software packages on Linux if necessary.
Summary
As the requirements for AI-enabled onboard embedded real-time video analytics computer systems become increasingly demanding and complex, the right choice of embedded industrial PC or motherboard solution is becoming even more important. Especially so when it comes to optimizing neural networks and power dissipation. While the acceleration of Deep Learning applications is still comparatively new, there is a variety of processors available to accelerate almost any neural network workload. From the major processor manufacturers to some of the newest startups in the industry, the choice is growing for accommodating more vertical markets, application areas, power requirements and price points.
As a leading global embedded computing manufacturer experienced in the specific needs of OEMs and developers designing embedded solutions for the transportation market, Kontron can greatly reduce project risk with standards-based, purpose-designed industrial embedded PC products, supported by strong microprocessor manufacturer relationships, optimized lifecycle and support, and flexible, state-of-the-art manufacturing facilities.
For more information about Kontron embedded AI solutions for onboard transportation systems, please visit: www.kontron.com/en/railway-embedded-computing and our e-showroom: https://kfrlabs.kontron.com/ Please contact: sales.KFR@kontron.com or +33 498 163 400.