High definition (HD) maps provide fundamental data support for intelligent connected vehicles (ICV). Light detection and ranging (LiDAR) has become an essential technique for HD map construction, environmental perception, localization, and other ICV tasks due to its advantage of high scanning accuracy and dense point cloud generation. LiDAR-based simultaneous localization and mapping (SLAM) technology is one prevailing method in HD map construction. However, in a SLAM algorithm, the pose estimation error is prone to accumulate and result in the map’s drift and structural error after long-distance travel. In order to avoid such problems, it is necessary to associate closed-loop data and correct the poses. This paper proposes a feature descriptor to detect loop closure, use a two-phase registration method to match closed-loop data, and optimize the map based on factor graph optimization. First, we calculate the feature direction based on the local distribution of planar feature points and construct the point cloud submap’s descriptor. Loop closure can be detected by calculating the similarity between feature descriptors. We filter the feature direction and set the weight to augment the vertical obstacles’ description in the structured environment. Second, we use the coarse transformation calculated from loop closure detection as the initial value to calculate a finer transformation between the closed-loop submaps by the normal distribution transform (NDT) method. Third, by adding a closed-loop constraint, the map’s error is reduced by factor graph optimization. Finally, based on the KITTI Odometry dataset and our data collected in an industrial park, the proposed method is validated and analyzed.