Browse Topic: Cameras

Items (641)
To address the challenges of binocular vision ranging under complex environmental conditions—such as illumination variations, occlusion, and textureless regions, which result in unreliable and non-robust performance—this paper proposes a multi-source heterogeneous sensor fusion ranging method integrating 4D millimeter-wave radar with the YOLOv5-Monster framework. This method is capable of overcoming the issue of limited ranging accuracy in monocular or binocular vision algorithms under non-ideal imaging conditions. This study achieves high-precision spatial perception through the following specific pipeline: First, Zhang’s calibration method is used to obtain the intrinsic and extrinsic parameters of the binocular camera, and stereo rectification is performed on the raw images. Next, a lightweight YOLOv5 network is employed for object detection, while a high-performance Monster network is utilized to generate dense disparity maps, thereby accomplishing initial depth estimation. To mitigate the inherent depth estimation errors of vision-only systems, 3D point cloud data from a 4D millimeter-wave radar is further introduced. By applying a Kalman filter algorithm, the millimeter-wave radar point cloud and visual outputs are fused, achieving spatiotemporal synchronization and optimal state estimation across modalities and effectively correcting biases in visual ranging. Experimental results show that within the full range of 4 to 150 meters, the relative error of the proposed method remains below 5%. Specifically, the relative errors are 1.25% (absolute error: 0.05 m) at 4 meters, 1.40% at 5 meters, 2.99% at 75 meters, and 4.91% at 150 meters. Compared with the vision-only Monster-YOLOv5 baseline method, the relative error at 150 meters is reduced from 13.16% to 4.91%, representing an accuracy improvement of over 60%. Meanwhile, in terms of long-distance error control, the proposed method significantly outperforms traditional stereo matching approaches such as SGBM+YOLOv5 and BM+YOLOv5, reducing errors by more than 20 percentage points. These results verify that deep multi-modal fusion can enhance environmental adaptability and measurement reliability, providing a high-precision and highly robust solution for distance estimation in intelligent perception systems, which holds important theoretical and engineering significance.
Li, FugaiXie, YuwenSu, HaoLiu, DongleiWu, Qiong
To address the challenges faced by micro flapping-wing flying robots in visual navigation—specifically, the large volume of visual information and the difficulty in transforming it into usable intelligent visual data—this paper proposes a clustering-based data-driven approach for directional and image perception. The aim is to enable intelligent visual navigation for flapping-wing robots. The proposed method performs clustering analysis on gyroscope data from the flapping-wing robot to extract directional features. Simultaneously, it applies clustering techniques to visual images captured by the robot to identify intelligent features such as edges. This approach enables the robot to acquire multiple optimized perceptual data types, thereby enhancing the behavior control system. Through the use of clustering analysis, the method not only improves the effectiveness of visual navigation but also extracts features related to visual targets and environmental information, providing technical support for visual target tracking. The experimental platform consists of a flapping-wing robot equipped with an onboard camera, and the proposed clustering-driven visual image perception approach has been experimentally validated. Experimental results demonstrate the high feasibility and effectiveness of the method in practical applications. The main contributions of this study lie in two aspects: (1) a clustering-driven visual image perception method for flapping-wing robots, and (2) a clustering-based approach for identifying posture and behavioral patterns of flapping-wing flying robots.
Li, ZixuanDing, WeiZhang, FengSong, MinLiu, ZhaomingMiao, LeiLiu, HaotianBai, NingTian, ShenCui, LongWang, Hongwei
This paper presents an innovative study in exploring, evaluating, and implementing deep-learning architectures for the calibration of multimodal sensor systems. The aim of this paper is to leverage the use of sensor fusion to achieve dynamic, real-time alignment between 3D LiDAR and 2D camera sensors. Static calibration methods are tedious and time-consuming, which is why we propose utilizing conventional neural networks (CNNs) coupled with geometrically informed learning to solve this issue. We leverage the foundational principles of extrinsic LiDAR–camera calibration tools such as RegNet, CalibNet, and LCCNet by exploring open-source models that are available online and compare our results with their corresponding research papers. Requirements for extracting these visual and measurable outputs involved tweaking source code, fine-tuning, training, validation, and testing of each of these frameworks for equal comparisons. This approach aims to investigate which of these advanced networks produces the most accurate and consistent predictions. Through a series of experiments, we reveal some of their shortcomings and areas for potential improvements. We find that LCCNet yields the best results among all the models that we validated.
Karramreddy, Venkat Sai RaxitMitchell, Liam
This paper presents a monocular vision-based system for high-precision missile pose measurement using ArUco markers and Perspective-n-Point (PnP) algorithms. By deploying 6 × 6 ArUco markers on a cylindrical missile mock-up, the system establishes 3D-2D correspondences between structured-light-scanned models and camera images to solve the PnP problem. The proposed approach integrates optimized ArUco marker recognition — leveraging adaptive thresholding, contour simplification, and grid-based validation — with the Efficient PnP (EPnP) algorithm to achieve real-time pose estimation. Experimental validation demonstrates angular accuracy of ± 0.3° in roll/pitch/yaw and positional accuracy of ± 2 mm within a 2 m range under controlled conditions. The system exhibits robustness against partial occlusions and motion blur, with degraded performance (± 1.2°, ± 5 mm) in extreme scenarios. Key innovations include a streamlined marker detection pipeline and adaptive pose refinement using Levenberg-Marquardt optimization. This work provides a cost-effective, non-contact solution for flight tests, with potential applications in weapon separation testing.
Wang, RuiyangZhang, Chaofan
In order to reduce traffic accidents caused by cars straying from lanes, a lane line recognition and deviation warning system based on machine vision is designed. It mainly includes image preprocessing, lane line detection, and the design of a deviation warning model. “In this study, an ROS-based intelligent vehicle-mounted camera is adopted for road image collection. To reduce the computational load of data processing while guaranteeing the algorithm’s accuracy and reliability, grayscale conversion and region of interest (ROI) extraction are implemented to finish the image preprocessing stage. Additionally, a fusion strategy of global and local thresholds is introduced to enhance both the operational speed and detection accuracy of the algorithm” use the Canny operator for the edge feature extraction; and complete the fitted lane lines with the improved Hough transform. Finally, based on the Kalman filter and camera viewpoint conversion coefficient algorithm, the lane line offset is detected in real time, and the deviation is judged in combination with the monitoring interface. Simulation experiments show that the system is able to effectively recognize the lane line and judge the deviation status under the condition of setting the offset threshold of 70 pixels, which significantly improves the accuracy and real-time performance of the lane deviation warning and provides effective technical support for reducing traffic accidents.
Wang, XufengZhang, ChunshuWang, YanChen, YihuiJi, Rui
The aging of the population has been a key issue worldwide, with mobility and fall of the elderly an important problem to be solved. In this paper, we propose an elderly mobility assist system based on the intelligent power-assisted device consisting of an assistive cane and an intelligent companion. It has the functions of standing support after falling, daily support and on-site rest. The assistive cane adopts a two-stage expansion mechanism of crank and slider structure, which forms a stable triangular support after unfolding, so that the patient can stand safely. The intelligent companion platform is driven by drive wheels, equipped with pushrod motors and vacuum suction devices, it can automatically approach the user and form an stable support column when the cane is in the out-of reach range; the control system is designed by combining microcontroller, camera object recognition, wristband remote control, to realize automatic steering and autonomous navigation at differential speed. The overall design satisfies the requirements of safety and strength through mechanical verification and stress analysis. The proposed system can help the elderly people to recover from falls better and enhance their independence and safety in their daily walks.
Yu, ChenxiWang, LongyiZhu, HuayunDong, YanMi, RuixueZhu, Lihong
End-to-end autonomous driving in urban environments faces three core challenges. First, camera and LiDAR sensor heterogeneity causes cross-modal perception inconsistencies and sensor fusion instability. Second, diffusion models suffer from training instability due to scale variance and distribution changes, which limits generalization. Third, traditional trajectory decoders lack structured interaction with semantic elements, thereby undermining planning rationality. To address these issues, CMFPNet introduces an integrated framework with three key modules. The HGCF-Backbone integrates LiDAR and camera features using channel focus, deformable cross-focus, and state space modeling to enhance semantic alignment. The NST module maps physical trajectories to normalized space, employing truncated diffusion sampling for stable generation in just 2–4 steps. The NDA models trajectory generation as a semantic narrative, utilizing a six-stage semantic attention flow incorporating BEV context, interactive dynamics, and self-states. Experiments on the NAVSIM dataset demonstrate CMFP Net’s superiority over existing baselines, showing outstanding generalization and trajectory stability in challenging scenarios. Notably, the truncated sampling strategy achieves an 8–10× acceleration during inference while maintaining decision accuracy and reducing computational costs. CMFPNet provides a scalable, semantically consistent solution for diffusion-based autonomous driving with significant potential in both research and practical deployment.
Qu, YanweiMo, Hangjie
Ultrasonic welding (UW) provides a rapid and efficient method for joining composite components by inducing resin flow through thermally driven diffusion and crystallization at the bonded interface. However, in the absence of a multiphysics modeling framework or a digital twin approach, current practice still depends on extensive trial-and-error testing to determine key welding parameters such as vibration amplitude, weld time, weld pressure, hold time, and downspeed. While in-situ thermal cameras can monitor surface temperatures, the internal temperature at the bonded interface is often significantly higher, introducing the risk of thermal degradation and inconsistent bond quality. To overcome these limitations, GEM developed a high-fidelity multiphysics model to establish a quantitative relationship between process parameters and the evolving temperature field within welded thermoplastic parts. The model integrates coupled mechanical, thermal, and acoustic physics to simulate high-frequency vibrations and static pressure, capture the generation and spatial distribution of heat, and represent the temperature-dependent viscoelastic response that governs bond formation. A validation test matrix was designed by systematically varying weld time and vibration amplitude. Through-thickness temperature distributions were measured using infrared thermal imaging, enabling direct comparison with model predictions. Upon validation, the model was applied for process tailoring, allowing precise control of temperature distribution to achieve target bond strength. This integrated modeling and validation approach demonstrated substantial benefits, including reduced design iterations, accelerated process optimization, and improved quality and performance of welded composite structures.
Walthers, MarkLi, RuiWei, QingxuanLua, Jim
Pedestrian fatalities in traffic accidents continue to rise, with severe injuries often resulting from both vehicle impact and subsequent ground contact, frequently occurring outside the field of view of vehicle-mounted cameras. This study presents a proof-of-concept (PoC) approach for reconstructing three-dimensional pedestrian motion—including occluded regions—using dashcam video. The method integrates 2D human pose estimation (MMPose) and monocular depth estimation (Depth Anything V2),the latter was fine-tuned on a custom dataset, to generate 3D skeletal coordinates.To evaluate motion matching, the reconstructed pedestrian poses were quantitatively compared with a database of vehicle collision simulations using the THUMS human body model and skeletal data representing real-world crash scenarios generated in PC-Crash. Composite similarity indices based on thoracic center of gravity trajectory and torso orientation vectors were employed for this comparison. Preliminary results indicate that the fine-tuned system achieves an average RMSE of approximately 0.1 m for key skeletal points, enabling accurate depth estimation for 3D pose reconstruction. Matching experiments with 11 PC-Crash cases demonstrated high similarity scores, and reconstructed sequences successfully identified critical injury events such as head-to-ground contact in occluded regions, confirming the feasibility of this approach for accident reconstruction and injury risk assessment. However, this study remains preliminary, limited to controlled indoor experiments with a single vehicle type and few subjects. Real-world crash footage and diverse vehicle geometries were not considered, and skeletal reconstruction from actual accident videos has not yet been implemented. Future work will expand the simulation dataset, refine similarity weighting, and validate the approach using real crash video. Ultimately, this technology may support forensic analysis and emergency response, but further validation is required before real-world application.
Onishi, KojiWang, KewangUno, ErikoIchikawa, KojiTanase, NoboruAndo, Takahiro
Edge detection is fundamental for intelligent vehicle applications, directly supporting ADAS functions such as lane detection, obstacle recognition, and scene understanding. The conventional Canny edge detection method exhibits notable shortcomings, especially in color-image processing, adaptive threshold selection, and preserving edge integrity under noisy conditions. In this study, we present an enhanced Canny edge detection framework tailored for ADAS-oriented intelligent vehicle systems, incorporating a quaternion-based weighted averaging scheme for color preservation, adaptive thresholds derived from gradient-amplitude histograms, multiscale edge localization via scale multiplication, and a novel gravitational-field-intensity operator for improved gradient robustness. Moreover, we extend the method to vanishing-point estimation an essential ADAS capability by performing precise intersection calculations combined with clustering techniques such as DBSCAN and RANSAC. Experimental evaluations demonstrate that the proposed algorithm markedly outperforms traditional approaches in edge clarity, localization accuracy, and noise resilience, underscoring its promise for strengthening ADAS perception modules in intelligent vehicles.
Uppala, Rohit RajKaye, MuraliZadeh, MehrdadTan, Teik-Khoon
Reliable environmental perception under adverse and contaminated conditions is a critical requirement for autonomous driving systems. Although LiDAR sensors play a central role in such perception, their performance is significantly degraded by surface contamination caused by environmental factors such as rain, snow, dust, anti-icing materials, and bug splatter impacts. However, most existing public datasets and prior studies rely on simulated or laboratory-generated contamination scenarios, which limit their applicability to real-world autonomous driving. To address this gap, we construct a large-scale real-world dataset collected from approximately 22,000 km of on-road driving across diverse regions of the United States, covering a wide range of naturally occurring environmental contamination conditions. The dataset was acquired using a multimodal sensing platform integrating LiDAR, perception RGB cameras, infrared camera sensors, and external monitoring systems, enabling comprehensive observation of sensor behavior under realistic operating environments. Based on this dataset, we propose a scalable contaminant classification framework that focuses on LiDAR surface contamination. A key contribution of this study is the introduction and exploitation of near-field point cloud features, which capture backscattered laser signals caused by surface contamination and exhibit a strong correlation with contamination severity and type. Using raw LiDAR signals, we utilize sixteen feature functions and train supervised learning models to classify seven distinct contaminant categories. Experimental results demonstrate that the proposed approach achieves classification accuracy exceeding 95% under real-world driving conditions, significantly outperforming prior laboratory-based studies. Furthermore, the framework is designed for practical deployment and can be extended to additional contaminant types and geographic regions through incremental data collection and learning. The proposed methodology enables real-time identification of LiDAR contamination sources, providing a critical foundation for adaptive sensor-cleaning strategies. By supporting contamination-aware sensor maintenance, this work contributes to cost- and weight-efficient sensor system design and represents an essential step toward achieving reliable Level 4 autonomous driving.
Kim, Hunjae
Accurate perception of the surrounding environment is fundamental and essential to safe and reliable autonomous driving. This work presents an integrated vision-based framework that com bines object detection, 3D spatial localization, and lane segmentation to construct a unified bird’s-eye-view (BEV) representation of the driving scene. The pipeline provides geometric information on object position and orientation by employing Omni3D to infer 3D bounding boxes of objects from monocular camera frames. Detections are subsequently projected onto a 2D BEV canvas, where object instances are represented with respect to the ground plane for enhanced interpretability. To complement the object-level perception, we utilized YOLOPv2 to perform lane segmentation, producing both lane masks and lane line masks in the image domain for future coordinate transformation. By adopting a pinhole camera model, the coordinate transformation of these masks from the perspective image plane into the BEV canvas can be performed. The fusion of 3D object detections and geometrically transformed lane representations yields a coherent and structured spatial map of the vehicle’s surroundings. In addition, the BEV space is integrated into a local 2D map generated from Mapbox tool. This unified environment model enables explicit reasoning about drivable space and surrounding obstacles, facilitating its integration into downstream modules such as path planning and trajectory prediction. The framework demonstrates the feasibility of leveraging recent advances in monocular 3D perception and deep learning-based lane segmentation to construct a computationally efficient and semantically rich BEV representation, which is a potential core perception component in real-time autonomous driving systems.
Tan, LinArjmanzdadeh, ZibaWang, HanchenLi, TaozheHajnorouzali, YasamanBurch, CollinLee, VictoriaXu, Bin
This paper presents a comparative study of three widely used cloud platforms, Google Colab, Microsoft Azure, and Amazon Web Services (AWS), for running a real-time cooperative perception system based on roadside unit (RSU) cameras. The goal is to evaluate the performance, scalability, and cost-efficiency of each platform when handling high-volume video data for object detection, a key task in autonomous driving. A unified perception pipeline using the YOLOv8 Small model was deployed on all platforms, with the same dataset and settings to ensure fair comparison. The evaluation focused on key metrics such as latency, frame processing rate, detection accuracy, cost, scalability, and reliability. The results show that Google Colab is a cost-effective starting point but has limitations in uptime and scalability. Azure offers stable performance and balanced cost, making it suitable for medium-scale applications. AWS delivers the best scalability and speed but at a higher cost. This study provides practical guidance for choosing the right cloud platform for deploying cooperative perception and intelligent transportation systems.
Alkharabsheh, EkhlassAlawneh, ShadiRawashdeh, Osamah
The increased integration of radar and vision sensors in modern vehicles has significantly improved environmental perception, safety, and automation. Nevertheless, conventional camera modules capture images in fixed, continuous frames, leading to unnecessary data processing, power consumption, and heat generation in the limited space of small sensors. The paper discusses the technology of Radar Based Dynamic Pixel Activation (RDPA); whereby radar data can be used to dynamically activate specific pixels on the camera sensor, optimizing image capture and processing. Through a systematic literature review of peer-reviewed articles published between 2021 and 2025, we examined the literature on radar-camera fusion, adaptive imaging, and sensor design that is efficient in power consumption. The review indicates a research gap that there is no current paradigm that dynamically activates sensor pixels at the hardware level using radar data. We aggregated ten topical studies and proposed a conceptual model where radar-determined Regions of Interest (ROIs) trigger localized pixel activation. The framework reduces the computational load, improves power efficiency and enhances thermal performance without affecting image fidelity. The paper also explains how RDPA may affect the Driver Monitoring Systems (DMS), Occupant Monitoring Systems (OMS), and Advanced Driver Assistance Systems (ADAS), and how it is more beneficial than traditional full frame imaging. Difficulties with synchronization, hardware interpenetration, and algorithmic synchronization are discussed. Altogether, RDPA is an excellent prospect to intelligent, energy-saving, and thermally stable vehicle perception systems of the next generation.
Kasarla, Nagender Reddy
This paper proposes ProGuard, a novel approach to preemptive pinch detection systems for buses. ProGuard utilizes state-of-the-art AI object detection algorithms to identify potential pinching events in bus entryways before pinching occurs. Modern conventional anti-pinch systems, such as pressure sensors or hall effect sensors, often rely on mechanical contact before triggering. While these systems are established safety mechanisms, they are reactive and therefore require some level of pinching before triggering. This reactive approach presents numerous safety concerns for passengers, especially when considering children on school buses. Existing preemptive detection methods, such as infrared or ultrasonic sensors, solve the problems presented by these reactive detection systems. However, these systems either lack the range or environmental resilience needed for reliable operation in buses. The critical nature of anti-pinch systems requires a robust and reliable solution that can adapt to various applications and environments. Our study investigates an AI-based approach that leverages the YOLOv11 nano object detection model to detect people and backpacks in real-time. We performed a comparative study on various model formats to find the best-performing format on the chosen edge compute hardware. Our experimental results revealed that when using the IMX model format on an AI-accelerated camera, ProGuard can achieve 24 frames per second and an inference time of 125ms while running on a Raspberry Pi computer. Performance tests on this model showed a mAP@0.5-0.95 of 0.522, putting ProGuard on par with baseline YOLOv11 nano performance. These results demonstrate that ProGuard offers an efficient and real-time alternative to current pinch detection approaches while operating on low-cost consumer hardware.
Bradley, HudsonZadeh, MehrdadTan, Teik-Khoon
Spectral imaging tools — cameras that capture colors beyond the RGB spectrum visible to our eyes — are vital for gleaning information about an object’s material and structural properties. Marrying them with machine learning has provided a powerful pipeline for identifying features in real-world applications including semiconductor fabrication, pollutant tracking, and crop monitoring.
Researchers now say that the best-performing materials in sustainable energy applications — e.g., converting sunlight or waste heat to electricity — often use collective fluctuations of clusters of atoms within a much larger structure; this process is often referred to as “dynamic disorder.”
A team of researchers at Penn State have devised a new, streamlined approach to design metasurfaces, a class of engineered materials that can manipulate light and other forms of electromagnetic radiation with just their structures. This rapid optimization process could help manufacture advanced optical systems like camera lenses, virtual reality headsets, holographic imagers and more, the team said.
Researchers at the University of Tokyo developed a framework to enable decentralized artificial intelligence-based building automation with a focus on privacy. The system enables AI-powered devices like cameras and interfaces to cooperate directly, using a new form of device-to-device communication. In doing so, it eliminates the need for central servers and thus the need for centralized data retention, often seen as a potential security weak point and risk to private data.
Accurate trajectory prediction of traffic agents is critical for enabling safer and more reliable autonomous driving, particularly in urban driving scenarios where close-range interactions are most safety critical. High-definition (HD) and standard-definition (SD) maps play a vital role in this process by providing lane topology and directional cues for forecasting agent movements. However, HD maps are expensive and resource-intensive to create, often requiring specialized sensors, while SD maps lack the precision needed for reliable autonomous navigation. To address this, we propose a novel framework for trajectory prediction that leverages online reconstruction of HD maps using vehicle-mounted cameras, offering a scalable and cost-effective alternative. Our method achieves improvements in predicting accuracy, particularly in close-range scenarios, the most crucial for urban driving, while also performing robustly in settings without pre-built maps. Furthermore, we introduce a new safety-aware evaluation metric that incorporates heuristic weights based on agent relevance and distance, enhancing traditional metrics like Brier-minFDE with a stronger focus on safety-critical scenarios. Extensive experiments demonstrate that our approach outperforms state-of-the-art map-less methods, particularly in close-range prediction, while our proposed metric establishes a more domain-relevant benchmark for assessing trajectory prediction in autonomous driving.
Upreti, MinaliGirijal, RahulB A, NaveenKumarThontepu, PhaniGhosh, ShankhanilChakraborty, BodhisattwaBhardwaj, Ritik
Vehicle door-related accidents, especially in urban environments, pose a significant safety risk to pedestrians, infrastructure and vehicle occupants. Conventional rear view systems fails to detect obstacles in blind spots directly below the Outside Rear View Mirror (ORVM), leading to unintended collisions during door opening. This paper presents a novel vision-based obstacle detection system integrated into the ORVM assembly. It utilizes the monocular camera and a projection-based reference image technique. The system captures real-time images of the ground surface near the door and compares them with calibrated reference projections to detect deviations caused by obstacles such as pavements, potholes or curbs. Once such an obstacle is detected the vehicle user is alerted in the form of a chime.
Bhuyan, AnuragKhandekar, DhirajJahagirdar, Shweta
The automotive industry is rapidly advancing towards autonomous vehicles, making sensors such as Cameras, LiDAR, and RADAR critical components for ensuring constant information exchange between the vehicle and its surrounding environment. However, these sensors are vulnerable to harsh environmental conditions like rain, dirt, snow, and bird droppings, which can impair their functionality and disrupt accurate vehicle maneuvers. To ensure all sensors operate effectively, dedicated cleaning is implemented, particularly for Level 3 and higher autonomous vehicles. It is important to test sensor cleaning mechanisms across different weather conditions and vehicle operating scenarios to ensure reliability and performance. One crucial aspect of testing is tracking the trajectory of the cleaning fluid to ensure it does not cause self-soiling of vehicles and affects the field of view or visibility zones of other components like the windshield. While wind tunnel tests are valuable, digitalizing this process is vital for making design decisions early in vehicle development. This work presents a digital methodology to test the self-soiling of a vehicle due to the cleaning systems present on vehicle exterior components, e.g. during mud cleaning at different vehicle speeds. The cleaning mechanism involves multiple water nozzles positioned above, below, or on the sides of these components, which spray water jets to remove dirt or mud deposits. The developed numerical method models the motion of cleaning fluid and contaminants after component cleaning. Steady-state aerodynamic simulations using the Finite Volume Method (FVM) are used to capture airflow, while the interaction of air with cleaning fluid and components is analyzed using a Smoothed Particle Hydrodynamics (SPH) solver. Correlations from this study and wind tunnel tests reveal potential optimization opportunities for existing cleaning systems by inspecting surrounding airflows at various vehicle speeds. Preliminary design evaluations indicate a specific vehicle speed range where self-soiling of vehicle components such as the windshield occurs due to mud cleaning. The proposed numerical method provides the capability to evaluate and qualitatively compare vehicle self-soiling due to various cleaning system designs of exterior components, offering valuable insights for optimizing cleaning mechanisms in autonomous vehicles.
Mane, SuvidyaMakam, Sri Lalith MadhavVarghese, RixsonDesu, Harsha
This paper presents a comprehensive survey and data collection study on the adaptability of Camera Monitoring Systems (CMS) for passenger vehicles. With the growing demand for enhanced safety, automation, and driver assistance technologies, Camera Monitoring Systems (CMS) has emerged as a key component in modern automotive design. This study aims to explore the current state of camera-based monitoring in passenger vehicles, focusing on their adaptability through survey data collection of various driving population and analysis. This paper evaluates the acceptance of CMS configurations in replacement to conventional rear-view mirrors through Position of Monitor, Clarity, CMS Adaptiveness to eyes, Comfort while turning, Merging into moving traffic, Monitoring Rear Traffic, while Getting Out of Car, while Overtaking, Coverage Area and Overall Acceptance. The findings offer valuable insights for manufacturers, engineers, and researchers working toward the evolution of intelligent vehicle systems.
Sinha, AnkitTambolkar, Sonali AmeyaBelavadi Venkataramaiah, ShamsundaraKauffmann, Maximilian
Computer vision has evolved from a supportive driver-assistance tool into a core technology for intelligent, non-intrusive occupant health monitoring in modern vehicles. Leveraging deep learning, edge optimization, and adaptive image processing, this work presents a dual-module Driver Health and Wellness Monitoring System that simultaneously performs fatigue detection and emotional wellbeing assessment using existing in-cabin RGB cameras without requiring additional sensors or intrusive wearables. The fatigue module employs MediaPipe-based facial and skeletal landmark analysis to track Eye Aspect Ratio (EAR), Mouth Aspect Ratio (MAR), head posture, and gaze dynamics, detecting early drowsiness and postural deviations. Adaptive, driver-specific thresholds combined with CAN-bus data fusion minimize false positives, achieving over 92% detection accuracy even under variable lighting and demographics. The emotional wellbeing module analyzes micro-expressions and facial action units to estimate stress, calmness, and agitation, contextualizing these states with fatigue indicators for holistic assessment. All computation occurs on the Jetson Nano edge platform with has a Quad-core ARM Cortex-A57 CPU and 128-core Maxwell GPU, optimized with TensorRT quantization for real-time operation (≤150 ms latency). The architecture ensures on-device privacy, aligning with GDPR and ISO/SAE 21434 cybersecurity principles. Compared with Tier-1 radar camera solutions, the proposed framework is fully software-driven, cost-efficient, and privacy-preserving. Field validation confirms strong correlation between model predictions and physiological HRV metrics. Future extensions include extreme fatigue detection and multimodal sensor fusion toward a self-adaptive, wellness aware vehicle ecosystem.
Iqbal, ShoaibImteyaz, Shahma
Perceiving the movement characteristics of specific body parts of a driver is crucial for determining their activity. Moreover, the driver’s body posture significantly impacts personnel safety during collision. This study investigates the creation of a dataset using Kinect depth camera for acquiring, organizing, annotating with skeleton tracking assistance, and optimizing interpolation. The pose recognition methods enhanced through an anchor regression mechanism, leading to the refinement of a lightweight anchor regression network capable of end-to-end learning ability from depth images. The improved backbone neck head structure offers advantages of reduced model parameters and enhanced accuracy. This engineering optimization makes it better suited for practical applications within vehicles with limited computational resources limitations and high real-time demands.
Xu, HailanLi, WuhuanLu, JunWang, XinHe, WenhaoChen, ZhenmingLiu, Yunjie
With the rapid development of autonomous driving technology, environmental perception, as its core module, has attracted much attention. Among them, the pure visual bird's-eye-view (BEV) 3D detection scheme has become a research hotspot due to its high spatial resolution and excellent semantic recognition ability in specific scenarios. Existing methods mainly utilize the Transformer encoder structure to perform position encoding in the BEV domain to achieve 3D perspective transformation, but they often fail to fully exploit the potential value of multi-perspective image information. To address this challenge, this paper proposes an improved Transformer-based visual BEV vehicle perception method that enhances perception performance by deeply fusing BEV domain and image domain information: an innovative multi-perspective position encoding mechanism is designed, which decouples camera parameters to more efficiently learn the mapping from images to 3D space; at the same time, a cyclic interaction attention mechanism is introduced to enhance the fine-grained association and fusion ability of pixel-level features, effectively improving the discriminability of features. In addition, to deal with challenges such as target occlusion in dynamic scenes, this method further proposes a long-term temporal perception framework that fuses multi-frame temporal information and designs a cross-time guidance module, significantly improving the robustness of target localization by injecting historical geometric constraints. Experiments on the nuScenes dataset verify the effectiveness of this method, and the results show that it achieves excellent performance in both spatial perception accuracy and temporal modeling capability, providing an innovative and practical solution for autonomous driving environmental perception.
Chen, PengyuWei, XiaoxuChen, Zhenwei
In low-light driving scenarios, in-vehicle camera images encounter technical challenges, including severe brightness degradation and short exposure times. Conventional driving image enhancement algorithms are susceptible to issues such as the loss of image features and significant color distortion. The proposed solution to this problem is a multi-scale attention fusion network (MAF-NET) for the enhancement of images captured during low-light driving conditions. The network’s structural design is uncomplicated. The model incorporates a meticulously designed multi-scale attention fusion module (MAFB), along with all essential components for network connectivity. The MAF is predicated on a heavy parameter residual feature block design and incorporates a multi-scale channel attention mechanism to capture richer global/local features. A substantial body of experimental evidence has demonstrated that, in comparison with prevailing algorithms, MAF-NET exhibits superior performance in low-light enhancement, detail retention, and color reproduction. Moreover, it attains commendable results in both subjective visibility assessments of nighttime driving scenarios and objective image quality metric tests, such as PSNR and SSIM.
Pan, DengChen, YuhanShi, YicuiLi, JieLi, Guofa
Vehicle trajectories encapsulate critical spatial-temporal information essential for traffic state estimation, congestion analysis, and operational parameter optimization. In a Vehicle-to-Infrastructure (V2I) environment, connected automated vehicles (CAVs) not only continuously transmit their own real-time trajectory data but also utilize onboard sensors to perceive and estimate the motion states of surrounding regular vehicles (RVs) within a defined communication range. These multi-source data streams, when integrated with fixed infrastructure-based detectors such as speed cameras at intersections, create a robust foundation for reconstructing full-sample vehicle trajectories, thereby addressing data sparsity issues caused by incomplete CAV penetration. Building upon classical car-following (CF) theory, this study introduces a novel trajectory reconstruction framework that fuses CAV-generated trajectories and infrastructure-based speed detection data. The proposed method specifically aims to reconstruct the unobserved trajectories of RVs located between successive CAVs within the same lane, ensuring continuity and accuracy in trajectory estimation. To validate the framework’s effectiveness, extensive SUMO simulations were conducted under different CAV penetration rates (PRs: 5%, 10%, 15%, and 20%) with a controlled traffic flow rate of 1000 veh/h. Key findings indicate that the proposed method maintains stable reconstruction accuracy across all tested penetration rates, with errors remaining within acceptable thresholds. Furthermore, comparative analysis against state-of-the-art CF-based reconstruction approaches reveals substantial improvements in accuracy, achieving reductions of 84.51% (LE), 97.07% (QLE) and 95.55% (TE), respectively. The result highlights the proposed method potential for enhancing real-time traffic state estimation, optimizing signal control strategies, and improving overall traffic management in V2I-enabled urban networks.
Bai, WeiFu, ChengxinYao, Zhihong
As I'm wont to do come December, with work well underway on the first issue of the new year, I like to take stock of upcoming venues for innovative product reveals and thought-provoking presentations on emerging trends and technologies. Come the first week of January, that means CES in Las Vegas. Traditional equipment manufacturers have increasingly used the event to demonstrate to the broader public that they not only deal in metal but also the digital realm. For example, earlier this year at CES, John Deere revealed its second-generation tech stack featuring camera pods, Nvidia Orin purpose-built processors and Deere's VPUs (vision processing units), along with four new autonomous machines including the 9RX 640 tractor for open-field ag operations. The company is exhibiting again this coming year.
Gehm, Ryan
Stoneridge displayed its vision for the future of commercial vehicle technology on the SAE COMVEC 2025 exhibit floor. The Innovation Truck showcases the Tier 1 supplier's next-generation vision and driver-assistance technologies designed to enhance driver safety and fleet optimization. Mario Gafencu, product design and evaluation specialist at Stoneridge, gave Truck & Off-Highway Engineering a tech truck walkaround at the event. The first technology Gafencu detailed was the second-generation MirrorEye camera monitor system that's designed to replace the glass mirrors on the sides of a truck.
Gehm, Ryan
Planetary and lunar rover exploration missions can encounter environments that do not allow for navigation by typical, stereo camera-based systems. Stereo cameras meet difficulties in areas with low ambient light (even when lit by floodlights), direct sunlight, or washed-out environments. Improved sensors are required for safe and successful rover mobility in harsh conditions. NASA Goddard Space Flight Center has developed a Space Qualified Rover LiDAR (SQRLi) system that will improve rover sensing capabilities in a small, lightweight package. The new SQRLi package is developed to survive the hazardous space environment and provide valuable image data during planetary and lunar rover exploration.
This article suggests a validation methodology for autonomous driving. The goal is to validate front camera sensors in advanced driver-assist systems (ADAS) based on virtually generated scenarios. The outcome is the CARLA-based hardware-in-the-loop (HIL) simulation environment (CHASE). It allows the rapid prototyping and validation of the ADAS software. We tested this general approach on a specific experimental application/setup for a vehicle front camera sensor. The setup results were then proven to be comparable to real-world sensor performance. The CARLA simulation environment was used in tandem with a vehicle CAN bus interface. This introduced a significantly improved realism to user-defined test scenarios and their results. The approach benefits from almost unlimited variability of traffic scenarios and the cost-efficient generation of massive testing data.
Cardozo, Shawn MosesHlavác, Václav
Waiting for a wound to heal is incredibly frustrating. First, it must clot; then an immune system response is needed; followed by scabbing and scarring — and that’s not even getting into the pain part.
Measuring the volume of harvested material behind the machine can be beneficial for various agricultural operations, such as baling, dropping, material decomposition, cultivation, and seeding. This paper aims to investigate and determine the volume of material for use in various agricultural operations. This proposed methodology can help to predict the amount of residue available in the field, assess field readiness for the next production cycle, measure residue distribution, determine hay readiness for baling, and evaluate the quantity of hay present in the field, among other applications which would benefit the customer. Efficient post-harvest residue management is essential for sustainable agriculture. This paper presents an Automated Offboard System that leverages Remote Sensing, IoT, Image Processing, and Machine Learning/Deep Learning (ML/DL) to measure the volume of harvested material in real-time. The system integrates onboard cameras and satellite imagery to analyze the field and top layers of residue, correlating this data with elevation maps to compute harvested material volume. This innovation supports operations such as baling, residue decomposition time and thereby contributing to land preparation. This technique offers benefits like reduced operational costs, labor independence, and enhanced soil nutrient planning.
Singh, Rana ShaktiStallin, Saravanan
This paper presents a novel approach to automated robot programming and robot integration in manufacturing domain and minimizing the dependency on manual online/offline programming. Traditional industrial robots programming is typically done by online programing via teach pendants or by offline programming tools. This presents a major challenge as it requires skilled professionals and is a time-consuming process. In today’s competitive market, factories need to harness their full potential through smart and adaptive thinking to keep pace with evolving technology, customer demand, and manufacturing processes. This requires ability to manufacture multiple products on the same production line, minimum time for changeovers and implement robotic automation for efficiency enhancement. But each custom automation piece also demands significant human efforts for development and maintenance. By integrating the Robot Operating System (ROS) with vision-based 3D model generation systems, we address these challenges effectively. A ROS-based framework has been developed to automate the manual offline robot programming and enable real-time task optimization for performing manufacturing operations such as painting, welding, and torquing. The proposed framework—Capture → Connect → Compile → Create—using RGBD camera systems to record 3D point cloud data and part details. It then connects the complete points, annotate features, interprets edges and tasks to be performed and then convert into executable robotic programs. This method significantly reduces manual programming efforts and enables rapid deployment of robotic systems across diverse tasks. The paper outlines the system architecture, implementation methodology, and integration strategy within existing manufacturing lines. Through autonomous robotic programming, this approach enhances mass customization and boosts overall manufacturing efficiency. The proposed system offers a scalable solution for smart factories aiming to achieve high productivity, flexibility, and reduced operational costs.
Hepat, Abhijeet
This study focused on the effects of hydrogen on the flame propagation characteristics and combustion characteristics of a small spark-ignition engine. The combustion flame in the cylinder was observed using a side-valve engine that allowed optical access. The fundamental characteristics of hydrogen combustion were investigated based on combustion images photographed in the cylinder with a high-speed camera and measured cylinder pressure waveforms. Experiments were conducted under various ignition timings and equivalence ratios and comparisons were made with the characteristics of an existing hydrocarbon liquid fuel. The hydrogen flame was successfully photographed, although it has been regarded as being difficult to visualize, thus enabling calculation of the flame propagation speed. As a result, it was found that the flame propagation speed of hydrogen was much faster than that of the existing hydrocarbon fuel. On the other hand, it was difficult to photograph the hydrogen flame directly in the lean region. In future studies, it will be necessary to reduce the shooting speed or to conduct investigations based on spectroscopic measurements. Regarding hydrogen combustion, it was found that the combustible range of hydrogen was broad even when the experimental conditions were substantially varied. It was also found that the combustible period was much shorter than that of the existing hydrocarbon fuel. Misfiring and after-firing were observed in hydrogen combustion where the ignition timing was close to top dead center as well as under a lean condition. This confirmed that there are issues concerning hydrogen combustion stability under such conditions.
Arai, YutoUeno, TakamoriSuda, RyosukeSato, RyoichiNakao, YoshinoriNinomiya, YoshinariMatsushita, KoichiroKamio, TomohikoIijima, Akira
Northwestern engineers have developed a new system for full-body motion capture — and it doesn’t require specialized rooms, expensive equipment, bulky cameras, or an array of sensors. Instead, it requires a simple mobile device.
Researchers have developed a prototype imaging system that could significantly improve doctors’ ability to detect cancerous tissue during endoscopic procedures. This approach combines light-emitting diodes (LEDs) with hyperspectral imaging technology to create detailed maps of tissue properties that are invisible to conventional endoscopic cameras.
In today’s digital age, the use of “Internet-of-Things” devices (embedded with software and sensors) has become widespread. These devices include wireless equipment, autonomous machinery, wearable sensors, and security systems. Because of their intricate structures and properties there is a need to scrutinize them closely to assess their safety and utility and rule out any potential defects. But, at the same time, damage to the device during inspection must be avoided.
Image sensors built into every smartphone and digital camera, distinguish colors like the human eye. In our retinas, individual cone cells recognize red, green and blue (RGB). In image sensors, individual pixels absorb the corresponding wavelengths and convert them into electrical signals.
The emergence of SUAS as a threat vector introduces significant challenges in surveillance and defense due to their potential for low cross section and high speeds, defeating or evading many existing detection and tracking capabilities. This paper presents two algorithms—one for detection and one for tracking—developed for event cameras, which offer substantial improvements in temporal resolution, dynamic range, and low-light performance compared to traditional imaging systems, all of which are critical for effective UAS defense. These advancements address current limitations in using event cameras and pave the way for a new generation of robust robotic vision based on event cameras.
Anthony, DavidChambers, DavidTowler, Jerry
Our research focuses on developing a novel loss function that significantly improves object matching accuracy in multi-robot systems, a critical capability for Safety, Security, and Rescue Robotics (SSRR) applications. By enhancing the consistency and reliability of object identification across multiple viewpoints, our approach ensures a comprehensive understanding of environments with complex layouts and interlinked infrastructure components. We utilize ZED 2i cameras to capture diverse scenarios, demonstrating that our proposed loss function, inspired by the DETR framework, outperforms traditional methods in both accuracy and efficiency. The function’s ability to adapt to dynamic and high-risk environments, such as disaster response and critical infrastructure inspection, is further validated through extensive experiments, showing superior performance in real-time decision-making and operational effectiveness. This work not only advances the state of the art in SSRR but also addresses the practical needs of end-users, providing a more robust tool for mission-critical operations.
Brown, Taylor J.Vincent, GraceNakamoto, KyleBhattacharya, Sambit
For further elucidation of the extremely complex mechanism of wall heat transfer during diesel flame impingement, heat flux measurement results based on two different relatively new approaches, high-speed infrared thermography and Micro Electro- Mechanical Systems (MEMS) heat flux sensor, were compared. Both measurements were conducted on the chamber wall impinged by a diesel flame achieved in constant volume combustion vessels under similar experimental conditions. Infrared thermography was conducted using a high-speed infrared camera (TELOPS M3k, 13,000 fps, 128×128 pixels), allowing the capture of time-series temperature and heat flux distributions on the wall surface with a spatial resolution of 70 μm (9 mm / 128 pixels). This high-resolution imaging also enables detailed estimation of near-wall turbulent structures, which are considered to significantly influence the heat flux distributions. The MEMS sensor is composed of closely aligned (520 microns separated) multiple highly sensitive thin-film Resistance Temperature Detectors (RTDs) of 235×235 microns, enabling estimation of near-wall turbulent fluid motion based on a cross-correlation analysis of measured heat flux fluctuations. The comparison between these measurements allows for mutual complementation of limited temporal resolution and quantitative accuracy of high-speed thermography and limited ability for spatial comprehension of the near-wall turbulence structure using the MEMS sensor. The time-series heat flux distribution obtained via high-speed thermography exhibited distinctive radial striped patterns. These patterns initially appeared as fine streaks with a high advection velocity immediately after wall impingement. As time progressed, they gradually increased in scale and exhibited a decrease in advection velocity. This behavior likely reflects the development of a boundary layer on the wall surface during the highly transient diesel flame impingement. Similarly, the heat flux measured by the MEMS sensor showed a comparable trend: the measured oscillation frequency corresponded well with the behavior of the striped patterns observed in the thermography, and the estimated fluid motion velocity was high immediately after wall impingement but gradually decreased over time.
Shimizu, FumikaMorooka, MasatoAizawa, TetsuyaDejima, KazuhitoNakabeppu, Osamu
Engineers have developed a smart capsule called PillTrek that can measure pH, temperature, and a variety of different biomarkers. It incorporates simple, inexpensive sensors into a miniature wireless electrochemical workstation that relies on low-power electronics. PillTrek measures 7 mm in diameter and 25 mm in length, making it smaller than commercially available capsule cameras used for endoscopy but capable of executing a range of electrochemical measurements.
In order to comply with increasingly stringent emission regulations and ensure clean air, wall-flow particulate filters are predominantly used in exhaust gas aftertreatment systems of combustion engines to remove reactive soot and inert ash particles from exhaust gases. These filters consist of parallel porous channels with alternately closed ends, effectively separating particles by forming a layer on the filter surface. However, the accumulated particulate layer increases the pressure drop across the filter, requiring periodic filter regeneration. During regeneration, soot oxidation breaks up the particulate layer, while resuspension and transport of individual agglomerates can occur. These phenomena are influenced by gas temperature and velocity, as well as by the dispersity and reactivity of the soot particles. Renewable and biomass based fuels can produce different types of soot with different reactivities and dispersities. Therefore, this study focuses on the influences of soot dispersity and reactivity by varying the reactive particle system. A model wall-flow filter channel is used for the investigation, enabling the observation of the layer break-up and detachment of particle structures from the filter surface over the entire channel length using a high-speed camera operating at 1000 fps. Recordings reveal significant differences in the behavior of the different soot types. Image analysis provides insights into the visual layer break-up, evolution of structure sizes, detachment dynamics, agglomerate velocities, and the spatial and temporal distribution of detachment events. General parameters, such as regeneration time and pressure drop, further characterize the regeneration process. The results provide insights into the detachment and rearrangement behavior of reactive particle structures with varying dispersities and reactivities in wall-flow filters, demonstrating that these characteristics can significantly influence the regeneration process.
Desens, OleHagen, Fabian P.Meyer, JörgDittler, Achim
The U-Shift IV represents the latest evolution in modular urban mobility solutions, offering significant advancements over its predecessors. This innovative vehicle concept introduces a distinct separation between the drive module, known as the driveboard, and the transport capsules. The driveboard contains all the necessary components for autonomous driving, allowing it to operate independently. This separation not only enables versatile applications - such as easily swapping capsules for passenger or goods transportation - but also significantly improves the utilization of the driveboard. By allowing a single driveboard to be paired with different capsules, operational efficiency is maximized, enabling continuous deployment of driveboards while the individual capsules are in use. The primary focus of U-Shift IV was to obtain a permit for operating at the Federal Garden Show 2023. To achieve this goal, we built the vehicle around the specific requirements for semi-public road operations which includes narrow streets and pedestrians. This involved integrating necessary modifications across multiple domains, including the e/e-architecture, sensor setup, software stack, and even the design of the driveboard and capsule. By utilizing systematic methods to address regulatory and safety challenges, we ensured that the vehicle met the standards required for autonomous driving in semi-public environments. In this paper, we explore the methodologies employed to achieve regulatory compliance, focusing on sensor integration, software- and e/e-architecture. We discuss our multi-modal sensor setup, which combines camera, lidar and radar to archive redundancy and enhanced environmental perception. Additionally, we provide an overview of our software architecture, emphasizing its role in ensuring safe driving functions and enabling autonomous operations.
Pohl, EricScheibe, SebastianMünster, MarcoOsebek, ManuelKopp, GerhardSiefkes, Tjark
With 2D cameras and space robotics algorithms, astronautics engineers at Stanford have created a navigation system able to manage multiple satellites using visual data only. They recently tested it in space for the first time. Stanford University, Stanford, CA Someday, instead of large, expensive individual space satellites, teams of smaller satellites - known by scientists as a “swarm” - will work in collaboration, enabling greater accuracy, agility, and autonomy. Among the scientists working to make these teams a reality are researchers at Stanford University's Space Rendezvous Lab, who recently completed the first-ever in-orbit test of a prototype system able to navigate a swarm of satellites using only visual information shared through a wireless network. “It's a milestone paper and the culmination of 11 years of effort by my lab, which was founded with this goal of surpassing the current state of the art and practice in distributed autonomy in space,” said Simone D'Amico, associate professor of aeronautics and astronautics and senior author of the study. “Starling is the first demonstration ever made of an autonomous swarm of satellites.”
In October 2024, Kongsberg NanoAvionics discovered damage to their MP42 satellite, and used the discovery as an opportunity to raise awareness on the need to reduce space debris generated by satellites. Kongsberg NanoAvionics, Vilnius, Lithuania Our MP42 satellite, which launched into low Earth orbit (LEO) two and a half years ago aboard the SpaceX Transporter-4 mission, recently took an unexpected hit from a small piece of space debris or micrometeoroid. The impact created a 6 mm hole, roughly the size of a chickpea, in one of its solar panels. Despite this damage, the satellite continued performing its mission without interruption, and we only discovered the impact thanks to an image taken by its onboard selfie camera in October of 2024. It is challenging to pinpoint exactly when the impact occurred because MP42's last selfie was taken a year and a half ago, in April of 2023.
Items per page:
1 – 50 of 641