A Smart Helmet Framework Based on Visual-Inertial SLAM and Multi-Sensor Fusion to Improve Situational Awareness and Reduce Hazards in Mountaineering

Sensitivity to surrounding circumstances is essential for the safety of mountain scrambling. In this paper, the authors present a smart helmet prototype equipped with visual SLAM (simultaneous localization and mapping) and barometer multi-sensor fusion (MSF), IMU (inertial measurement unit), omnidirectional camera, and global navigation satellite system (GNSS). They equipped the helmet framework with SLAM to produce 3D semi-dense pointcloud environment maps, which are then discretized into grids. Then, the novel danger metrics they proposed were calculated for each grid based on surface normal analysis. The A* algorithm was applied to generate safe and reliable paths based on minimizing the danger score. This proposed helmet system demonstrated robust performance in mapping mountain environments and planning safe, efficient traversal paths for climbers navigating treacherous mountain landscapes.


INTRodUCTIoN
Mountain scrambling, often called alpine scrambling, is a recreational sport that entails ascending mountain peaks and ridges while sometimes employing one's hands to scale rock walls and navigate challenging terrain (Whymper, 1871).While less technical than rock climbing, scrambling can still be quite dangerous, given the exposure to heights and objective hazards.Some of the potential dangers of mountain scrambling include falling from narrow ledges and cliffs; rockfall when climbing steep rock faces, with loose rocks potentially dislodged onto climbers below; rapid weather changes like rain, wind, and lightning (which can lead to slippery conditions, loss of visibility and hypothermia); getting lost due to lack of distinctive landmarks in mountainous terrain; and exposure to altitude sickness, sunburn, dehydration, and cold weather injuries due to the high altitude environment.
The underlying mountaineering severity can lead to tragic consequences.Analysis of The International Alpine Trauma Registry (IATR) data reveals that out of 306 recorded mountain accidents resulting in multisystem trauma, the majority were due to falls onto solid ground (51.4%), followed by falls in snowfields (10.8%), falls into crevasses (8.1%), and being struck by stones (5.4%).In 24.3% of cases, the cause was unknown (Rauch et al., 2019).These examples remind us of the dangers climbers who venture onto precipitous terrain face.The field would benefit from continued efforts to improve safety protocols and leverage technological innovations to mitigate the risks inherent to these physically and mentally demanding sports.
In this regard, a novel helmet prototype has been invented that presents a viable solution by alerting individuals through a loudspeaker about potential hazards and suggesting the safest route while minimizing travel time.The equipped SLAM technology offers a revolutionary approach to managing the objective risks inherent in mountaineering through the collaborative integration of multiple sensors.At the same time, the Multi-Sensor Fusion (MSF) has great promise to address the limitations of SLAM systems alone for enhanced precision and reliability.This helmet showcases the ability to use real-time data from the surrounding environment to show the best route to the destination.

ReLATed woRK
Throughout human history, helmets have played a crucial role in safeguarding the lives of individuals facing dangerous environments and challenges.From ancient civilizations to the present day, the evolution of helmet technology has been driven by a relentless pursuit of safety and protection.In the early days, helmets were forged from basic materials such as leather and bronze.Warriors and adventurers wore them to guard against head injuries in combat and dangerous terrain.As the centuries passed, helmet design and materials advanced, incorporating iron, steel, and carbon fiber.These improvements significantly enhanced protective capabilities, but the helmet is still primarily focused on protecting against physical impacts.
The rise in amateur mountaineers has increased the risk of fatal accidents, necessitating advanced safety equipment.In the 21st century, mountaineers face numerous environmental and navigational hazards, requiring protection beyond physical impacts.Helmets equipped with computer vision and guidance capabilities are essential, aiding in decision-making under extreme conditions and ensuring safer, less strenuous routes, especially in altitude sickness and hypothermia cases.
A principal method used in our helmet framework is Simultaneous Localization and Mapping (Durrant-Whyte & Bailey, 2006;Engel et al., 2018), which is widely used to enable a robot or autonomous vehicle to construct a map of an unfamiliar environment while simultaneously recognizing its position in that environment.The recent advancement in 3D map reconstruction (Grisetti et al., 2010) and SLAM (Ebadi et al., 2022b) not enable robots to precisely positioning and make autonomous decisions in a scalable approach (Kohlbrecher et al., 2011) that can be applied to extreme environments.
This technique facilitates the helmet system in navigating through intricate and unexplored areas by continuously updating its spatial awareness.By utilizing Simultaneous Localization and Mapping for generating detailed 3D terrain mapping and recognizing cliffs, edges, and overhangs, scramblers could more effectively evaluate potential risks of falling or rockfall and choose paths that are less exposed.
Moreover, another primary function of SLAM systems is to map surroundings while locating oneself, which makes this technology highly suitable for mitigating disorientation.Since the lack of noticeable reference points within rugged mountainous settings can frequently result in hikers becoming disoriented, the utilization of precise, up-to-date positioning technology within a constantly evolving diagram of the mountains could significantly enhance understanding of the current situation and lower the incidence of unintentional deviations from the intended path.A few Simultaneous Localization and Mapping (SLAM) algorithms have integrated supplementary data like weather patterns in their cartography protocols (Hong et al., 2020).Combining meteorological knowledge with topographical maps presents an opportunity to identify locations of shelter or alternate routes with less susceptibility to unfavorable weather patterns.Such data would significantly mitigate risks associated with precipitation, wind, and lightning strikes while undertaking demanding expeditions.
Though SLAM's capabilities show immense promise in enhancing mountain scrambling safety, relying solely on SLAM for localization and mapping during mountain scrambling has limitations, as the algorithm can accumulate errors over time, leading to imprecision and posing uncertainty.Therefore, in this case, we have incorporated sensors such as GNSS, IMU, and barometer to perform the MSF algorithm to ultimately obtain a more accurate, reliable, and stable position and orientation estimation.This is where MSF (Luo et al., 2002) can complement SLAM to create more robust and resilient systems for practical use in unpredictable real-world alpine environments.
In MSF, data from multiple heterogeneous sensors are combined using intelligent algorithms to leverage the strengths and mitigate the weaknesses of each sensory modality.For instance, visual SLAM may be fused with inertial measurement units (IMUs) containing accelerometers and gyroscopes.While visual SLAM is prone to drift, IMU inputs can provide short-term motion constraints to correct and bind these errors.However, IMUs alone are inaccurate over longer durations due to integration drift.Fusing the two modalities creates a system more reliable than either sensor in isolation.Additionally, other complementary sensors can be integrated, such as LIDAR for longrange mapping, GNSS (Global Navigation Satellite System) (Kaplan & Hegarty, 2017;Zaliva & Franchetti, 2014) coupling SLAM with this array of proprioceptive and exteroceptive sensors through MSF provides multi-faceted observations of the environment to overcome the weaknesses of any individual sensor channel.This leads to greater accuracy and resilience for localization and mapping in challenging mountainous terrain.
Furthermore, MSF also enables redundancy and cross-validation between sensor streams.When one modality is compromised, such as loss of visual tracking, others may be able to provide continuity and observe environmental constraints until the degraded signal recovers.This fail-soft characteristic is essential for robustness in safety-critical mountain scrambling.

MeTHodoLoGy
The core functionality of the helmet (Figure 1) is provided by 360-degree cameras (Scaramuzza, 2014) around the rim to provide a detailed 360-degree visual understanding of the surroundings, eliminating blindspots.The video feeds are analyzed using algorithms such as image segmentation (Haralick & Shapiro, 1985) to classify terrain, identify hazards like cliffs and overhangs.The vision system builds a geospatially-registered point cloud and mesh model of the environment updated continuously with wearer movement.This allows assessing slope, proximity to drop-offs, safe paths, and other navigational insights.• GNSS: GPS and other GNSS provide an absolute geographical position fix, which helps build an accurate map of the surroundings and locate the climber precisely on it.This enables planning optimal traversal routes.

• IMU (Ahmad et al., 2013):
The IMU uses accelerometers and gyroscopes to track the climber's movement and orientation.This aids localization and mapping by complementing the cameras with information on movement and perspective.It can also detect falls by recognizing sudden accelerations.• Barometer: The barometer measures atmospheric pressure changes to estimate altitude, complementing the visual mapping.This improves vertical positioning when visual references are inadequate.
The 360-degree vision achieves comprehensive hazard detection, the IMU tracks climber motion for localization, the GNSS provides precise global positioning to situate the climber on the map, and the barometer aids with altitude estimates when visual cues are limited.The combination enables robust situational awareness in challenging mountain environments.

System Integration
The diverse sensor suite enables the building of a detailed 360-degree model of the environment.The vision algorithms integrate footage from omnidirectional cameras queue depth and inertial data to the appropriate scene region.Then, the hazardous zones, navigation markers, and other real-time overlays are rendered on a compact heads-up display inside the helmet.The algorithm continuously analyzes the 360-degree camera feeds and is designed to be low latency to ensure the augmented environment view responds immediately to user movement and terrain changes.
When the helmet perceives an area of instability like loose rock, thinning ice, or a steep dropoff, it marks the danger zone in the heads-up display.As the climber moves, these warnings remain geospatially fixed, allowing them to be visually tracked.The system can also provide audible alerts when in proximity to hazards.By constantly monitoring the scene, the helmet acts as a vigilant lookout for potentially deadly situations the climber could miss in the high cognitive load of navigation.
The combination of a 3D semi-dense map, motion tracking, and real-time hazard overlay allows navigating mountain faces while continuously assessing upcoming terrain and proximity to cliffs or crevasses.

Rugged design
The helmet employs a hardened design to handle the harsh conditions of mountaineering.The housing encasing the cameras, sensors, and other fragile electronics is impact-resistant and reinforced to withstand falls, stones, and compressive forces.The optics employ scratch-resistant glass lenses and hydrophobic coatings to maintain visibility in rain and snow.Internal components feature moisture protection, heat sinking, and shock absorption.The helmet maintains reliable operation across a broad thermal range, functioning equally well on frozen summits or under blazing sun.Automated self-tests validate all sensors and indicators to catch failures (Henry & Clarke, 1993).The helmet continues providing vital assistance even after harsh impacts or prolonged exposure by employing resilience engineering.

User data Collection and Privacy Considerations
The development of our smart helmet is grounded in a conscientious approach to user data collection, ensuring rigorous compliance with privacy standards and explicit user consent (Mertens, 2018).We employ robust anonymization techniques to safeguard climbers' personal information, upholding their privacy rights.Simultaneously, the helmet's real-time hazard analysis algorithms are designed to maximize user safety, meticulously balancing the dual imperatives of risk mitigation and privacy preservation.Additionally, the Gaussian distribution-based machine learning scheme could be applied to anomaly detection and ensure the security of our data analysis processes (Dwivedi et al., 2021).
We must also address the critical aspect of data collection and sensor reliability.The integration of IoT-based sensors for big data collection, as discussed by Plageras et al. (2018), highlights the necessity of efficient data processing and analysis to enhance the functionality and reliability of the smart helmet.Similarly, the work of Chawra and Gupta (2022) on optimizing wake-up scheduling in 3D-wireless sensor networks, and the efficient data collection scheme for underwater linear sensor networks proposed by Ahmed et al. (2022) underscore the significance of ensuring sensor robustness and data accuracy.

ALGoRITHM
To carry out localization, our approach initially involves implementing SLAM to construct a semidense map.Subsequently, the map is divided into grids, and we compute a dangerous score for each grid.The subsequent path planning is founded on the computed grid graph utilizing the A* algorithm.
Figure 3 displays the whole SLAM and MSF process.The input to the algorithm is a stream of images from the 360 camera as it moves through the environment.The first step is to perform image projection to undistort those images according to camera intrinsic parameters, and feature extraction and feature matching are applied to get the poses between two keyframes via geometrybased function essential matrix and fundamental matrix (Luong & Faugeras, 1996), followed by the triangulation to get the 3D map points.Furthermore, those expanded 3D map points and the estimated camera pose are fed back into bundle adjustment for further optimization (Triggs et al., 2000).This incremental process repeats as new keyframes are acquired to build a consistent map while estimating the camera trajectory.
The optimized 3D map points and camera poses are combined with additional sensor data from other sensors using MSF techniques.This sensor fusion also allows for estimating the initial camera pose for the next keyframe, which is then used to triangulate and expand the 3D map (SemiDense Map) via features matched to previous keyframes through image projection.In summary, image projection, bundle adjustment optimization, multi-sensor fusion, and triangulation are combined in a loop to generate an optimized 3D map along with estimated camera poses at each keyframe.

Image Projection
Map points can be projected into the 360 camera's equirectangular image plane based on the currently estimated camera pose to provide a detailed 360-degree visual understanding of the surroundings.The projected pixel coordinates can then be matched to feature points detected in the equirectangular image.
To establish 2D-3D correspondences between features in the catadioptric omnidirectional camera system, Geyer and Daniilidis (2000) first proposed a four-step projection process to model the geometric relationship between a 3D scene point and its corresponding 2D pixel location in the camera image.
Consider a scene point P x y z = ( ) , , in the mirror's reference frame.For convenience, we assume the axis of symmetry of the mirror aligns with the camera's optical axis, and the x and y axes of the camera and mirror are also aligned.Hence, the only difference between the camera and mirror reference frames is the translation along z.
Projecting the scene point onto the unit sphere: The The point m is then mapped to the camera image point p u v

= ( )
, ,1 through the intrinsicparameter matrix K . Therefore: where K is: However, Geyer and Daniilidis' fisheye projection model varies from camera to camera and depends on the field of view of the lens, and approximation of fisheye lenses by catadioptric cameras has limited accuracy.
To tackle that problem, Scaramuzza (2014) proposed a unified model that uses the Taylor polynomial to overcome the lack of knowledge of a parametric model for fisheye cameras, whose coefficients and degrees are found through the calibration process.Accordingly, the relation between the normalized image point m x y m m

= ( )
, ,1 and the unit vector P s in the fisheye (mirror) reference frame can be written as: Scaramuzza (2014) has also emphasized that polynomials of the third or fourth order can accurately represent all catadioptric and several types of fisheye cameras currently available in the market.The ability of this model to be used effectively with a diverse range of commercial cameras is the fundamental reason for its great future potential.

Feature Point detection (FAST)
Feature point detection extracts distinctive keypoints for tracking across frames.A commonly used detector is Features from Accelerated Segment Test (FAST).FAST (Mair et al., 2010) examines local pixel neighborhoods to rapidly identify high-contrast corners.This efficiency suits real-time SLAM, especially in complex mountain environments.Real-time performance is critical but challenging due to factors, such as variable lighting and terrain.FAST's speed enables fast keypoint extraction despite these difficulties.By only considering local pixel circles rather than expensive smoothing.

oRB descriptor
While FAST efficiently detects keypoint locations, we also need descriptors to characterize the local visual appearance at each keypoint.Some Robust descriptors can be used to facilitate matching keypoints between frames by representing the distinct image patch around each point.
A highly effective descriptor that complements FAST detection is ORB (Oriented FAST and Rotated BRIEF), which builds upon FAST to extract oriented FAST corners, followed by describing these keypoints using a compact binary string via the BRIEF (Binary Robust Independent Elementary Features) method (Calonder et al., 2010).
BRIEF compares pixel intensities in a smoothed image patch at predefined locations relative to the keypoint.The comparisons produce a bit string that summarizes the salient gradients.ORB modifies BRIEF to make it rotation-invariant for matching under viewpoint changes.Together, the oriented FAST detection and rotated BRIEF description in ORB provide a feature that is fast to compute, invariant to rotation, and robustly matched across frames.Moreover, those characteristics make ORB ideal for real-time SLAM in dynamic environments with continuously changing camera orientation.

Feature Matching
Descriptor matching between keyframes is essential for tracking visual features across different viewpoints in mountain visual SLAM.As the camera explores mountainous terrain, the perspective changes dramatically, causing the appearance of features to vary significantly across frames.The ORB descriptor vectors are compared using Hamming distance (Norouzi et al., 2012) to identify matches between keyframes.
We can establish a keypoint match between frames if the descriptors have a small Hamming distance, indicating they likely correspond to the same physical 3D point viewed from different mountain terrain perspectives.We can track the same features across multiple keyframes taken at distinct camera poses and lighting conditions by finding descriptor matches.This would allow incrementally estimating the camera motion and constructing a map of the mountain environment by tracking matches over sequential views.
Initially, we have two keyframes I 1 and I 2 that depict overlapping views of a scene.We can then establish 2D-to-2D correspondences between points in the two frames by leveraging epipolar geometry constraints to estimate the fundamental matrix F and initial relative pose (Zhang et al., 1995).For example, pixel P 1 in I 1 matches to P 2 in I 2 .
Using these feature matching, we can compute the fundamental matrix F that encodes the epipolar constraint: Decomposing F gives us the essential matrix E and an initial estimate of the relative camera motion between the two poses P 1 and P 2 .

By Single Value Decomposition
, where T is the translation and R is the rotation from frame I 1 to I 2 .From this, we get our initial pair of camera matrices P 1 and P 2 .
Next, we triangulate the feature matches to reconstruct 3D points X j in the scene.For a matched pair of pixels P 1 and P 2 : = By back-projecting rays from the feature bearings in both frames, their intersection gives the 3D coordinate X j .
Repeating this process for all matches allows the construction of a map of 3D points corresponding to tracked visual features.
We now have a set of 3D-to-2D correspondences between reconstructed points X j and their pixel coordinates in the keyframes.
As the camera explores new viewpoints I n , we can utilize the Perspective-n-Point (Lu, 2018) algorithm to estimate an initial pose P n using established 3D-to-2D matches with points visible in frame I n .
Finally, bundle adjustment globally refines all camera poses P P P P n minimizing the total reprojection error between the 3D points and corresponding observed pixel coordinates across all keyframes.This jointly optimizes the pose estimates to obtain a globally consistent reconstruction.

Back-end Fusion with GPS and IMU Pre-Integration
Every keyframe state at time t is defined as x p q v ba bw compromising position p , orientation q as a quaternion, velocity v , and IMU biases ba for the accelerometer and bw for the gyroscope.
IMU measurements between times t and t + 1 are pre-integrated into a relative motion increment z: t → t + 1 (Förster et al., 2017): where Δ denotes the change in each state variable between t and t + 1 predicted by integrating the IMU.This pre-integration accounts for the IMU biases by including them in the state as: , ,
å denotes the Mahalanobis distance (De Maesschalck et al., 2000) with covariance Σx for SLAM, Σy for GNSS (Global Navigation Satellite Systems) and Σz for the pre-integrated IMU increment.This optimizes over x t and x t +1 , adjusting the state estimate to best satisfy the constraints from both GNSS and IMU pre-integration.Critically, the pre-integrated IMU measurements are not recomputed, as the optimization only touches the biases ba t , bw t , avoiding unnecessary recalculations.This way, GNSS global position measurements are fused with locally estimated pose increments through an optimization leveraging pre-integration.This provides globally accurate and locally precise state estimates.
In circumstances where GNSS suffers from intermittent errors caused by temporary obstruction of the satellites by mountains, groves, etc. and fails to achieve precise positional data, there is a possibility of experiencing drift in the vertical position estimation.To tackle this issue, adding a barometer can be utilized to measure atmospheric pressure.By incorporating pressure measurements, we can obtain a constraint for the altitude h from the z direction, thus enhancing the accuracy of pose measurements.
The barometer measures atmospheric pressure, providing an independent altitude estimate f x ( ) which convert the pressure signal to height value and may be varied according to a different model: We can get the pose state by: We then fuse the barometer into the optimization, and minimize the cost: In summary, fusing barometer altitude GNSS, especially for vertical positioning, when GNSS references are impaired due to lack of internet connectivity.The barometer helps maintain 3D accuracy when external data is limited.

Normal Vector Calculation From the Semi-dense Map
To extract normal vectors from a semi-dense Point Cloud map, we first divide the 3D space into grids at a resolution that captures local surface elements.With the grid produced from the segmentation, the next step is analyzing each neighborhood's point distribution by the covariance matrix.
The core analysis (Sanchez et al., 2020) phase involves Principal Component Analysis (PCA) on each region.For each local point set P P P P n 1 2 3 , , , ,  { } , the 3x3 covariance matrix C v is computed between the x y z , , coordinates.C v encapsulates the correlation structure of the surface points.Eigen decomposition of C v yields eigenvalues l l l , , { } .The eigenvector v i paired with the minimum eigenvalue l i corresponds to the axis of least variance, as the normal vector estimate.

Calculation of each Grids' dangerous Score
After converting the mountainous terrain into grids and obtaining the normal vectors for each grid, the subsequent stage involves examining the data related to the normal vectors to evaluate the danger level.Various indicators of danger can be derived from the normal vectors, including multiple metrics: • Incline angle: The angle between the normal vector and the global vertical axis.Steeper slopes generally denote a greater risk of falling or sliding.The dot product between the normal and vertical yields the cosine of the incline angle.• Roughness: Measured by the variance of normal vectors within a grid.High variance implies an irregular, bumpy surface that can be treacherous.To integrate these factors into an overall danger score D for each grid v, we propose a weighted function: where: Models with increasing risk with slope angle A v exponentially, where β is a scaling factor: Incorporates the roughness of neighboring grids R vn : Edge proximity E v weighted by cliff score C v .
The cliff score C v is computed as follows.
For each grid v, cast rays outward in directions (such as front or sides), and for each ray, find intersecting grids w via raycasting.
Compute dot product of v's normal N v and ray direction D.
where Dh = height difference between v and w, α is a scaling factor: We use the ratio of isotropy to grid surface area as an indicator of loose terrain: Logarithmic scale obstacle density: This danger metric integrates additional grid information like volume and cliff scores with non-linear transformations of the metrics to better model danger response and the exponential and logarithmic functions to help account for sensitivity and saturation effects.
The metrics can be normalized before combining based on expected value ranges.The weighting coefficients w i are optimized per environment using techniques, such as grid search and crossvalidation against hazard data (Yan et al., 2022).
Once calculated for all grids, the danger scores are classified into qualitative levels like low, medium, and high.The environment map can be visualized with risk-coded colors to support planning safe navigation routes and activities.As the normal vectors get updated dynamically from new In summary, A* search optimized for minimum danger is used to plan optimal safe paths.The path is refined and adjusted incrementally as the environment and danger scores change.Danger is monitored during execution, and replanning occurs if thresholds are exceeded.
eXPeRIMeNT ReSULT The helmet system was tested in simulations and real-world environments, with experiments demonstrating robust capabilities in obstacle detection, 3D mapping, localization, and route planning for mountain navigation.Quantitative results showed improved localization accuracy by fusing SLAM, GNSS, and barometric data, while the system also consistently identified optimal safe traversal routes through analyzed terrain hazards.
We started the experiment in a real-world environment.Figure 7 is the moving object detection under this keyframe, and the semi-dense point cloud is then reconstructed from each keyframe, showcasing the helmet's ability to create a detailed 3D representation of the mountain terrain, providing critical situational awareness.The following experiment highlighted the improvements in localization accuracy when fusing SLAM, GNSS, and barometric data.These enhancements significantly reduce position and orientation errors, vital for precise navigation in mountainous terrain.The route planning experiment demonstrated the helmet's ability to calculate safe traversal paths by analyzing terrain hazards.It utilizes the generated grid graph to consistently find optimal minimum-risk routes, balancing safety and efficiency.Collectively, these experiments validate the smart helmet's capacity to enhance situational awareness through real-time sensor data fusion.
The following are details for each experiment respectively.

Moving object detection
An essential capability demonstrated was precise moving object detection and image projection to distortion input from the 360-degree camera input.As shown in Figure 7, the system identified and tracked multiple people during mountaineering.By projecting image features onto the frame using the unified projection model, robust detection was achieved.

Localization Accuracy
The position errors show the drift as a percentage of the total 100m traversal distance.With SLAM alone, the position drifts 1.2m over 100m of travel, giving a 1.2% error.Adding GNSS reduces this to 0.8% over the same distance.Integrating the barometer further improves accuracy to 0.4% drift.
The orientation errors are measured in absolute degrees of deviation from the ground truth.SLAM has 4.5° drift, improved to 2.1° with GNSS and 1.2° with barometer added, where the GNSS provides periodic position corrections and the barometer constraints drift along the vertical axis.This results in more precise and robust state estimation in the challenging mountain environment.

Semi-dense Map Reconstruction
The reconstructed semi-dense point cloud provided a detailed representation of the mountain environment geometry.As depicted in Figure 8, the incremental bundle adjustment process leveraged matched features across keyframes to estimate camera poses and triangulate world points.This produced a geospatially registered 3D model reflecting the mountain structure.

Route Planning
The semi-dense 3D point cloud generated via SLAM was divided into grids, and we computed danger metrics, including slope, roughness, and proximity to cliffs.Figure 9 illustrates the resulting grid graph with danger color coding.The optimal minimum-risk route was identified using A* search optimized for safety.The system could consistently find safe trajectories, balancing hazard avoidance with traversal efficiency.
These results validate the helmet prototype's ability to enhance situational awareness through real-time sensory data fusion.Combining computer vision, SLAM mapping, and environmental analytics provides a comprehensive platform for informed navigation in hazardous mountain regions.

CoNCLUSIoN
This innovative smart helmet prototype demonstrates promising capabilities in enhancing mountaineer safety through visual-inertial SLAM and Multi-Sensor Fusion.The main contributions of this work lie in its comprehensive framework, which incorporates algorithms to achieve precise localization, robust mapping, and safe path-finding in extreme terrains, ensuring that mountaineers can navigate safely even in the most challenging conditions.While further refinements are needed, the prototype showcases an essential step toward wearable enhanced reality assistance in hazardous alpine settings.The low cost of visual SLAM technology, compared to its LiDAR counterparts, presents an economic advantage, making it a more feasible option for large-scale production.Visual SLAM components potentially range around 50-500 USD, compared to over 2000 USD for LiDAR-based systems.Enhanced machine learning techniques, such as reinforcement learning, could be vital in optimizing system behaviors for user safety and comfort.The system could adapt and improve its performance over time by continuously learning from user interactions and environmental conditions, leading to a more intuitive and user-friendly experience.
However, our algorithm is dependent on clear visual conditions, and its performance could be compromised in fog, heavy snow, or low-light scenarios.Integrating multiple sensors and advanced processing capabilities demands significant power, potentially resulting in limited battery life.The helmet's capability to collect and process vast amounts of data in real-time could lead to information overload, making it challenging for users to make quick and informed decisions.
The potential applications of this technology extend far beyond mountaineering, promising benefits across various high-risk activities and industries.In search and rescue operations, the helmet's precise localization and mapping capabilities could significantly enhance rescue teams' efficiency and safety when navigating challenging terrains.The construction and mining sectors could see improved worker safety with the helmet's real-time situational awareness and hazard detection.Outdoor adventure enthusiasts, including rock climbers and backcountry skiers, could leverage the helmet for safer navigation in unfamiliar terrains.

Figure 1 .
Figure 1.The prototype design of the helmet

Figure 3 .
Figure 3.The pipeline of the entire SLAM and MSF process

Figure 4 .
Figure 4.The image projection for the omnidirectional camera

•
Edge proximity: Distances to nearby grids with substantially differing normals may mark a cliff or overhang.Smaller distances flag a potential edge hazard.• Isotropy (Kamash & Robson, 1978; Rivlin & Ericksen, 1997): How close the normals are to an ideal plane normal.High isotropy indicates loose, unstable terrain like gravel or sand.• Obstacle density: The number of distinct objects distinguished by separate normals.More obstacles mean more hazardous navigating and an increased chance of tripping.

Figure 8 .
Figure 8. Semi-dense grid map reconstruction via SLAM

Figure 9 .
Figure 9. Planned safe traversal route by A* search

Table 1 . Comparative analysis of smart helmets Causes of Injuries Strengths Problems
Note: The smart helmets listed in this table were all launched from 2016 to 2019.The features and limitations of various smart helmet models are outlined, highlighting the common issues of high cost and lack of intelligent decision-making.Our helmet framework is equipped with visual SLAM technology, ensuring a robust and economical solution scalable for mass production.

Figure 5. Triangulate to obtain map point depth
When a GNSS measurement y t +1 arrives, it observes an absolute position that can constrain pose drift.Furthermore, the v t +1 is the pose stemmed from our visual SLAM bundle adjustment; we then incorporate this by minimizing the cost: = ( ),

Table 2 . Localization accuracy with different sensor combinations
Note: The values presented in this table represent the localization errors measured over a traversal distance of 100m.Scene 1 is located in Bodafon Mountain (Wales), providing a certain set of challenges and environmental conditions, while Scene 2 is situated in Garnedd Ugain Mountain (Wales), offering a different landscape and set of obstacles.