High definition (HD) maps can provide comprehensive and accurate knowledge of the environment for highly autonomous vehicles. However, HD maps have large data volumes, which pose high demands on data acquisition, analysis, and storage. To balance data volume and map definition between HD maps and low-accuracy road network maps, this paper is focused on the construction of lightweight HD maps for autonomous vehicles running with non-paved roads. Firstly, we introduce the representation of a lightweight HD map which contains several special elements, e.g., operating areas, parking lot, junction, borders, and centerlines, to describe the closed environment. Then, we propose a border generation algorithm and a border expansion algorithm to find the real border of the map and construct the drivable area. Meanwhile, a two-step method is proposed to extract the centerline of the road. An experiment is conducted to demonstrate the effectiveness of the proposed algorithms. The proposed lightweight HD map has less data volume than HD maps and higher precision than low-accuracy road network maps, and can be widely applied to autonomous vehicles working in non-paved roads.