Deep learning-driven semantic segmentation and spatial analysis of quarry relic landscapes using point cloud data: insights from the Shaoxing quarry relics

The use and preservation of stone reflect an important aspect of early human culture, particularly concerning stone quarrying. Therefore, studying Chinese quarry relics is vital for understanding the impact of stone culture. This research sheds light on how ancient people consciously interacted with and transformed their natural environment. Establishing a database of Chinese quarry relics is especially significant in the context of contemporary climate change and frequent geological disasters.

Dating back to the pre-Qin Dynasty, the Shaoxing quarry relic landscape exemplifies the essential characteristics of Chinese quarry sites. These relics, products of human history and environmental interactions, form a foundational basis for studying the historical evolution of China’s geographical environment and the development of traditional Chinese landscapes. Current research primarily comes from Chinese scholars, and the methods and technical tools predominantly focus on qualitative analysis of art forms. However, these approaches are insufficient to fully capture the complexity and diversity of quarry relic landscapes^1,2. There are several challenges to address, including the limitations of two-dimensional analytical methods and the absence of advanced techniques for three-dimensional modeling and quantitative analysis. To overcome these challenges, it is essential to integrate advanced digital technologies such as 3D laser scanning, Building Information Modeling (BIM), virtual reality, 3D reconstruction, point clouds, and UAV photogrammetry.

In the realm of architecture and engineering, the preservation of tangible cultural heritage, including historical buildings, village settlements, cultural landscapes, and archaeological artifacts, has become increasingly critical. In recent years, the application of digital technologies to document and assess cultural heritage has become a global trend in heritage conservation³. The main digital preservation technologies include 3D laser scanning, BIM, virtual reality, 3D reconstruction, point cloud, and drone photogrammetry⁴, which offer a broad and diverse range of functions in the study of historical buildings and complexes^5,6,7, village settlements⁸, archaeological artifacts^9,10, and cultural landscape^11,12,13 in the field of cultural heritage research.

3D scanning technology promotes interdisciplinary research across multiple fields. Currently, 3D scanning technology is mainly applied in five areas: built and urban environments, cultural heritage and archaeology, subsurface environments, environmental monitoring, and forestry and agriculture¹⁴. These applications include the development of 3D web services for smart city concepts¹⁵, the identification of real-world subsurface infrastructural physics¹⁶, the non-contact assessment and prediction of environmental stability in damaged and polluted environments¹⁷, and plant phenotyping tasks¹⁸. In landscape architecture, 3D scanning technology is particularly focused on cultural heritage and archaeology. The point cloud data obtained through 3D scanning can precisely identify historical relics and garden spaces. The applications of 3D point cloud data are primarily seen in the following areas: studying the microstructure of cultural heritage^19,20, analyzing the spatial patterns and structures of garden rockeries^21,22,23,24, and creating a 3D point cloud database that provides essential information for restoration and reconstruction efforts^25,26. Deep learning has emerged as a crucial research direction in machine learning, with wide applications in image and natural language processing, speech recognition, and online advertising²⁷. Remote sensing image land class recognition technology based on deep learning significantly enhances recognition accuracy, acquiring land-class information more intelligently²⁸ and facilitating the screening, classification, and output of subsequent information. In the field of cultural heritage, deep learning can be used to explore the representativeness and diversity of heritage information³.

Existing research combining 3D scanning and deep learning has focused on engineering and medical research but has not yet been applied to historical sites with complex spatial patterns^29,30,31. There are significant research gaps, particularly the need for studies on the three-dimensional spatial morphology and semantic segmentation of quarry relic landscapes. This study aims to enhance the process of acquiring 3D point cloud data and to perform semantic segmentation of the elements within the quarry relic landscape. By calculating the volume ratio of each element, this research will provide more accurate and intuitive data for theoretical studies, thereby deepening the understanding of the artistic concept behind the Shaoxing quarry relic landscape. Additionally, this study seeks to expand the application of deep learning techniques in traditional landscape research, broadening the scope of investigation within this field.

Building on the preceding discussion, we propose the following two research questions:

(1) How can 3D scanning technology precisely capture the three-dimensional spatial structure and internal elements of quarry relic landscapes?

(2) How can deep learning techniques be employed for semantic segmentation and quantitative analysis of quarry relic landscapes?

Addressing these research questions will offer new insights and methods for preserving and studying quarry relic landscapes, promoting technological advancements in cultural heritage conservation.

Study site

This study was conducted in Shaoxing City, Zhejiang Province, Eastern China. Shaoxing, located in the north-central part of Zhejiang Province, south of the Qiantang River estuary, is an ideal site for studying quarry relic landscapes due to its historical and cultural significance. As the capital of the Yue State during the early period (770-221 BC), Shaoxing has over 2,500 years of urban development, showcasing a rich interplay of historical craftsmanship, culture, and art. The region’s diverse topography includes the southwestern mountainous and hilly areas, the central river and lake plains, and the northern coastal mudflats. This varied landscape, combined with a complex water system and elevated water levels, made the stone a more suitable material for construction than wood, leading to extensive quarrying activities. These historical quarrying activities have left a legacy of well-preserved quarry relics, including East Lake, Keyan, Houshan, and Yangshan, which are rich in historical and cultural significance and represent major existing quarry relics (Fig. 1).

Shaoxing natural geography and quarry relics location map.

Full size image

The favorable landscape environment, stable political situation, and developed commodity economy in Shaoxing fostered urban construction, water conservancy, and transportation needs, stimulating the growth of the quarrying industry and the construction of quarry relics (Fig. 2). Quarry sites are predominantly located in plain areas near isolated hills along the river, divided into eastern and western regions, with notable sites including Raomen Mountain, CaoShan Mountain, Houshan Mountain, Daxi Mountain, Niutou Mountain, Keshan Mountain, Ma’an Mountain, and Yangshan Mountain³². East Lake, Keyan, Houshan, and Yangshan quarry relic landscapes are well-preserved, and rich in historical and cultural significance, representing the four major existing quarry relics (Fig. 3). The demand for urban construction spurred the growth of quarrying activities, leading to the creation of quarry ruins that served as the foundation for quarry relic landscape construction, a tradition that continued from the Qin Dynasty through to the modern era, shown in Fig. 4. These well-preserved sites provide invaluable insights into the historical and cultural quarry landscape of ancient China.

Stone and application of four quarry relics in Shaoxing.

Full size image

Existing Four quarry relics in Shaoxing.

Full size image

History of quarrying in Shaoxing.

Full size image

The selection of Shaoxing as the study site is highly relevant to the objectives of this research. The region’s quarry relics offer a unique combination of historical significance, cultural richness, and diverse landscapes, making it an unparalleled site for exploring advanced digital technologies in the study and preservation of quarry relic landscapes. By leveraging Shaoxing’s quarry relics, this research aims to develop innovative methods for capturing three-dimensional spatial structures and performing quantitative analysis. Advanced digital technologies such as 3D laser scanning, BIM, virtual reality, 3D reconstruction, point clouds, and UAV photogrammetry can precisely capture the intricate details of the quarry landscapes, allowing for comprehensive three-dimensional spatial analysis and quantitative evaluation. The integration of digital twin technology further enhances the ability to monitor and analyze these relics in real time, providing dynamic and interactive models for conservation efforts.

Acquisition and processing of 3D point cloud data

This study examines four quarry relic landscapes located in existing quarry sites in Shaoxing, each capable of constituting an independent enclosed space: Shifo Temple in Yangshan Quarry Relic, Quarry in Caoshan Mountain, Shengshui Pond in Houshan Quarry Relic, and Seven-star Rock in Keyan Quarry Relic. For the acquisition of three-dimensional data, a Focus S70 laser scanner was used. This device operates effectively in temperatures ranging from −20 °C to 55 °C, with a scanning range of up to 70 meters. It has a measurement speed of up to 976,000 points per second, a point ranging error of ±2 mm, and a scanning range of 360° horizontally and 300° vertically. UAV photogrammetry was conducted using the MAVIC AIR 2, which features a camera with a 1/2.0-inch CMOS sensor and 48 effective megapixels, an f/2.8 aperture, a maximum flight altitude of 500 meters, and a range of approximately 18.5 kilometers.

The study developed a foundational mapping method for quarry relic landscapes, as illustrated in Fig. 5. The method includes the following steps:

1. a.

   Environmental Data Collection: Efficient and comprehensive quarry relic data acquisition was achieved by strategically arranging scanning stations and routes, utilizing both a 3D laser scanner and a UAV.

2. b.

   Data Processing: The raw point cloud data and aerial imagery were processed separately.

3. c.

   Data Representation: The processed 3D point cloud data were used to produce CAD maps, orthophoto maps, and orthophotos.

Basic mapping process of quarry relic landscape.

Full size image

Data collection

(1) TLS Data Collection

Due to the complex structure, large scale, and limited field of view of quarry relic, collecting point cloud data using a 3D laser scanner cannot be achieved in a single pass. Therefore, multiple scans from different viewpoints are necessary. The collected data from these various scans must be unified into a single coordinate system through computer-based recognition, alignment, and integration to generate comprehensive spatial information. Additionally, RGB values and texture information for each scanned object are captured using 1920×1920 pixel photos. A suitable number of scanning sites are chosen to ensure the successful creation of a 3D information model of the quarry relic. The arrangement of scanning sites should meet the following criteria:

1. 1.

   The scanning range must cover the entire object. Additional scanning sites should be added in areas with complex structures and significant obstructions.

2. 2.

   Scanning sites should be situated on flat terrain. If the terrain is uneven, built-in leveling should be used to calibrate the sites.

3. 3.

   To minimize alignment errors in the point cloud data, the overlap between neighboring scanning sites should be at least 15%.

A drawing of the 3D laser scanning sites in the quarry relic landscape should be prepared, with sites arranged according to the numbering sequence on the drawing. A total of 58 3D laser scanning sites were established across four quarry relic landscapes: Shifo Temple, Shengshui Pond, Quarry in Caoshan Mountain, and Seven-star Rock. Specifically, Shifo Temple had 29 scanning stations, Shengshui Pond had 12, Quarry in Caoshan Mountain had 9, and Seven-star Rock had 8 (Fig. 6). The scanning concentrated on the following areas:

1. 1.

   Shifo Temple: Main architectural features include Chenghuang Peak, Shifo Peak, and Pudu Peak.

2. 2.

   Shengshui Pond: Water features, rock formations, caves, and the Vine Embracing Rock.

3. 3.

   Quarry in Caoshan Mountain: Water features, Shizi Rock, and Xiangbi Rock.

4. 4.

   Seven-star Rock: The existing deep pool and rock formations.

3D laser scanning site arrangement schema.

Full size image

(2) UAVDP Data Acquisition

The use of UAV aerial photography data covers the shortage of the 3D scanner in capturing the top views of quarry relic. To acquire UAV aerial photography data, a detailed route planning design was first conducted using Rocky Capture software for intelligent route planning. After defining the flight range, aerial photography parameters were set, including flight altitude, speed, and overlaps in both the flight direction and side direction. Due to the varying heights of quarry relic rocks, the flight altitude was adjusted accordingly. The maximum flight speed was set to 5 m/s. In compliance with Chinese national specifications, the heading overlap for low-altitude aerial photogrammetry should be between 60–80%, and the side lap overlap should be between 15–60%. For higher accuracy in subsequent 3D reconstruction, the heading overlap for this aerial photogrammetry was set to 80%, and the side lap overlap was set to 70%.

A total of 20 flight routes were designed for the four quarry relic landscapes, with 5 routes for each mapping site. The average ground resolution of the acquired images was set to 3 cm, with the actual shooting altitude at approximately 50 meters. The image acquisition process involved four flights and lasted about 3 hours and 40 minutes (Fig. 7). In total, 681 photographs were captured, each covering an area of approximately 6.2556 hectares.

Drone route schema.

Full size image

Data processing

(1) TLS Data Processing

TLS data processing can be divided into three main steps: point cloud alignment, point cloud denoising, and point cloud sampling (Fig. 8). 1) Alignment: Trimble RealWorks uses the overlapping areas of two scan stations to align the data. The scanned *.fls files are imported station by station into Trimble RealWorks. Automatic alignment is performed, followed by manual adjustments to correct any errors in the automatic alignment process. 2) Denoising: To denoise the point cloud, the “Point Cloud Splitting” tool in Trimble RealWorks is employed. This tool removes large-scale noise, such as ground noise, using straight-pass filtering, while smaller-scale noise, like outliers around gardens, is handled through statistical filtering. It is crucial to adjust the parameters of the denoising algorithm carefully to ensure that the denoised point cloud accurately reflects the spatial characteristics of the quarry ruins. This involves reducing errors by combining algorithm adjustments with manual identification of noise, as well as optimizing downsampling parameters, such as sampling rate, to preserve key features. 3) Sampling: To perform sampling, first select the area to be cropped and switch to the top view. Adjust the size of the crop box and rotate it to align with the features of the scanned point cloud. Next, switch to the front view to control the crop height. Define the boundaries of the cropped point cloud by dragging the edges of the crop box, and then extract the selected point cloud through the sampling process.

TLS data processing flow.

Full size image

(2) TLS & UAVDP Data Fusion

Using ground control points, point cloud data from UAVDP and TLS were fused using Geomagic Studio software. The process involved the following steps: 1) Manually pinpoint seven ground control points in the TLS point cloud. Using the “feature point alignment” function in Geomagic Studio, these control points were aligned with the corresponding points in the UAVDP point cloud. 2) The deviation of each control point pair was calculated according to the model in Eq. (1), The alignment order was then adjusted step by step by removing pairs with large deviations and retaining those with smaller deviations. Ultimately, at least three control point pairs with the smallest deviation are selected as the feature points for the coordinate transformation of the point cloud model to complete the accurate alignment of the TLS with the UAVDP point cloud model.

σi denotes the correction bias of control point i; (n) denotes the number of control points involved in the final coordinate transformation.

Deep learning for point cloud semantic segmentation

PointNet is proposed by QI, and the basic idea is: firstly, use a 3×3 T-Net network matrix to rotate and transform the original point cloud data to ensure that the processed point cloud data are aligned to a regular space to improve the segmentation accuracy; secondly, use a 64×64 T-Net network matrix to align the extracted features of the point cloud to ensure that the features are processed uniformly; and finally, apply the Maximum Pooling (MaxPooling) technique to aggregate the features of the points³³. MaxPooling technique has translation, rotation, and scaling invariance, which can reduce the network parameters and computation while retaining the main features of the point cloud, prevent the point cloud network from overfitting, and improve the ability of generalization of the point cloud network model, to solve the problem of the disorder of the point cloud, and PointNet + + introduces a point cloud partitioning method based on PointNet. The basic idea of PointNet + +, which introduces point cloud hierarchical sampling and local relationship modeling on the basis of PointNet, is: firstly, the farthest point sampling (FPS) method is used to iteratively complete the sampling of the point cloud in the original 3D field attraction point cloud data, and a number of points are selected as the center points³⁴; secondly, the BallQuery method is used around each center point (i.e., take the given center point as the ball center, include all the points in the given radius, and at the same time give the number of neighboring points) to select the surrounding points to form a cluster, each cluster as an input sample of PointNet network and get a set of local features; then keep the center point fixed, expand the radius of the area, and use the local features obtained in the previous step as the input of PointNet again, and so on, by continuously expanding the local range, extracting local features, and finally obtaining global features, thus realizing the purpose of scene segmentation. In the challenging 3D point cloud benchmark test, PointNet + + obtains significantly better results than the state-of-the-art³⁵. Therefore, when applied to complex quarry relics spatial elements, the use of the PointNet + + network for recognition and classification can relatively accurately categorize the elements of quarry relic landscape. The dataset, model training, evaluation indexes, and comparative validation of this deep learning method will be discussed later.

Production and division of the dataset

Based on the results of point cloud data acquisition from the previous section, we constructed a dataset for classification experiments using data obtained from the quarry relic landscape scene. This dataset includes a variety of targets and non-targets collected from different scenes, categorized into five types: rocks, architecture, water, plants, and pavement. After data preprocessing and enhancement, a total of four pieces of point cloud data were obtained, and K-Fold cross-validation was adopted for training and testing. In this method, each point cloud scene is treated as a subset (or fold), where each iteration involves selecting one subset as the test set while using the remaining three subsets for training. This process is repeated for four iterations, with a different subset designated for testing each time. Finally, the average performance of the model across different datasets is calculated by taking the mean of the results from the four iterations. Sample images from the categorized dataset are shown in Fig. 9, alongside the corresponding sample point cloud in Fig. 10.

Corresponding samples of the quarry relic landscape classification dataset

Full size image

Corresponding to the pre-processed sample point cloud.

Full size image

Model training and analysis

(1) Experimental environment

The hardware and software environments for classification experiments and the various parameters set for training are shown in Table 1. The PointNet + + network is used in this experiment, and the parameters are set as shown in Table 2.

(2) Evaluation metrics

To evaluate the classification results of different networks in the quarry relic landscape scene, the mainstream point cloud classification accuracy metrics include Overall Accuracy (OA), Multiply–Accumulate per Cycle (mAcc), etc. OA refers to the ratio of the number of correctly predicted classes of point clouds in the scene to the total number of point clouds in the scene and is a commonly used metric for the multiclass classification problem; mAcc refers to the average of all class’s classification accuracy of the evaluation model. OA refers to the ratio of the number of correctly predicted categories in the scene to the total number of clouds in the scene, which is a commonly used metric in multi-category classification problems. To reasonably evaluate the classification performance of the classification network on samples, this paper uses two metrics, OA and mAcc, as the criteria for judging the network performance. The calculation of the evaluation metrics is shown in (2) and (3).

Where TP represents the number of correct prediction results; W represents the total number of point clouds in the dataset; and n represents the number of classes.

To examine the performance of the model for multi-class target segmentation in the point cloud of quarry relic landscape, four evaluation metrics, including Precision (symbolized as I_pre), Recall (Recall, I_rec), intersection over union (loU), and F1 score (symbolized as I_F1), are used to evaluate each class segmentation. The precision measures the proportion of the samples predicted by the model to be positive examples, which shows the accuracy of the model in identifying positive examples, while positive examples refer to those points that are correctly labeled as a specific category, such as a point that belongs to the category of “rocks” (the real semantic labeling is “rocks” category). For example, if a point belongs to the category of “rock” (the real semantic labeling is “rock”), and it is also labeled as “rocks” in the segmented point cloud, then this point is a positive example; recall is the proportion of samples that are positive examples correctly identified by the model; intersection over union is a measure of the degree of overlap between the model predicted region and the real region, the closer to 1, the model’s prediction of the degree of overlap between the model predicted region and the real region. 1, indicating that the model’s prediction region overlaps with the real region to a higher degree, thus reflecting the superiority of the model’s prediction performance; the F1 score is the reconciled average of precision and recall, which is used to comprehensively evaluate the model’s performance, and the higher the F1 score is, the better the model’s performance is. Their functional expressions are as follows:

Where Q is the number of samples, i represents different classes, i = 1,2,…,n; TP denotes True Positive, i.e. the model predicts that a sample is a positive case and this prediction is correct; FP denotes False Positive, i.e. the model predicts that a sample is a positive case but this prediction is incorrect; TN denotes True Negative, i.e. the model predicts that the points do not belong to the category, and in fact, they do not belong to the category. FN means false negative, i.e., the model predicts that the points do not belong to the category, but in fact, they do belong to the category.

Digital information management of quarry relic point cloud

Using the described method, point cloud data for the four quarry relic landscapes (Shifo Temple, Shengshui Pond, Quarry in Caoshan Mountain, and Seven-Star Rock) was acquired, processed, and presented. This approach, combined with documentary evidence, facilitates the digitization of historical information related to the quarry relic landscape. It provides a framework for the digital preservation and management of Quarry Relic Heritage, offering a means to preserve and manage these cultural landscapes effectively.

Shifo Temple (Fig. 11a) is situated at the eastern foot of Yangshan Mountain, also known as Yangshi Mountain. The landscape of Shifo Temple is shaped by the mined mountain and residual water, with the quarry relic lake encircled by the remnants of the mountain. The temple is constructed harmoniously with the leftover mountain, creating a picturesque scene where “Buddha is in the stone, the stone is in the water, and the water is in the mountain.” The current point cloud data model effectively illustrates the spatial relationships between the mountains, water, and architecture of Shifo Temple. Utilizing point cloud data allows for the creation of a comprehensive spatial structure of the landscape featuring quarry relics.

a The point cloud master plan of Shifo Temple. b The point cloud master plan of Shengshui Pond. c The point cloud master plan of Quarry in Caoshan Mountain. d The point cloud master plan of Seven-star Rock.

Full size image

Shengshui Pond (Fig. 11b) is situated in Houshan Mountain, and its name is derived from a line in Zhang Dai’s poem: “Who carved the mountain with ghosts and gods? It is merely the mined mountain with residual water.” The viewing of Shengshui Pond is based on one-sided cliffs, with various forms of artistic expression. The different artistic effects of Shengshui Pond are formed with the development of the tour route, which can be quantitatively analyzed in the point cloud model.

Quarry in Caoshan Mountain (Fig. 11c) is situated at the base of Caoshan Mountain and features several notable cliff carvings. On the west side of the pool, a stone beam extends across the water, measuring over 20 meters in length, and shaped like a green lion’s head. Compared with the other three quarry relic landscapes, the Quarry in Caoshan Mountain has more figurative rock forms such as Xiangbi Stone and Shizi Stone. The point cloud modeling can be used to further study the morphological treatment of cliffs and boulders in traditional Chinese landscapes and the restoration characteristics of the rock form in the quarrying wasteland.

Seven-star Rock (Fig. 11d) is located in the Keyan Quarry Relic and dates back 1300 years to the Quarry Relic era. Originally, the site comprised seven interconnected quarries, with features ranging from large stone chambers to smaller caves, arranged in a pattern resembling the Big Dipper, hence the name “Seven-star Rock.” According to the data of the Shaoxing Culture, Radio, and Television Tourism Bureau, among the four existing quarry relics in Shaoxing, Keyan has the highest visibility and visitor volume, so it is particularly important to balance its development and protection. As a closed inward space, Seven-star Rock is viewed by going deep into the interior, which requires consideration of the spatial capacity. With the help of spatial data from point cloud modeling, a visitor flow model can be built to calculate the visitor flow in a small area to achieve the most comfortable visitor viewing.

Overall, 1) Introducing the point cloud data model into the study of Chinese gardens can make up for the visual limitations of one-sided viewing and form a more complete and comprehensive viewing relationship (Fig. 12). 2) In terms of heritage data protection, there is complete data storage and batch storage of each element, which provides the basis for the construction of the digital twin system and contributes to the social and cultural dissemination of traditional quarrying art. 3) In terms of design or restoration, it emphasizes the grasp of the overall spatial form of quarry relic landscape and helps to excavate the inner logic of quarry relic landscape design. (Fig. 13).

Multi-faceted perspective of rock in quarry relic landscape.

Full size image

The significance of the establishment of the quarry relic landscape point cloud database.

Full size image

Semantic segmentation of quarry relic point cloud

Validation of point cloud semantic segmentation results

(1) Identification and classification of landscape elements in quarry relic

The art of quarry relic landscape emerges from the interplay between water, rocks, plants, etcetera. Each landscape element contributes uniquely to the creation of a quarry relic landscape, culminating in its distinctive artistic effect. Following the classification standards of traditional Chinese gardens³⁶, this study categorizes the quarry relic landscape into five elements: rock, water, plants, architecture, and pavement.

“Rock” encompasses the exposed rock formations on the cliffs resulting from quarrying activities. “Water” refers to the ponds formed by quarrying activities. “Plants” include naturally or artificially grown vegetation such as trees, shrubs, vines, bamboo, and herbs on the quarry relic mountains. “Architecture” covers various architectural structures or combinations designed for both aesthetic and functional purposes within the landscape. “Pavement” includes garden paths and ground surfaces.

Using this categorization, the point cloud data of the quarry relic landscape is semantically segmented. Point cloud models representing different elements from the four quarry relic landscapes are extracted, as illustrated in Figs. 14 and 15.

Deep learning results for quarry relic landscape elements.

Full size image

Semantic segmentation of the elements in quarry relic landscape.

Full size image

(2) Evaluation of results & comparative analysis

1) Evaluation of results

The calculation results of four evaluation indicators for five categories, including rocks, architecture, plants, water, and pavement, obtained by applying PointNet + + are shown (Table 3). It can be seen that the four key evaluation indexes of this paper’s method, such as F1 score, Precision, Recall, and IoU. According to the results of the four evaluation indexes, the PointNet + + network presents a superior classification performance in the recognition of all five elements of the quarry relic landscape, different elements are scored slightly differently in different evaluation indexes. In general, the classification effect of rock is the best, and it presents the best effect in the F1-score, Recall, and IoU.

2) Comparative analysis

Comparison of sub-elements:

To investigate the classification effect of the improved algorithm on the target and non-target in the quarry relic landscape scene, the classification experiments are carried out using the homemade dataset, and the classification accuracies of each category are computed separately as an additional metric to evaluate the performance of the improved network in this paper. Table 4 shows the classification accuracies of PointNet and PointNet + + for the five categories in the homemade dataset of the quarry relic landscape scene. It can be seen that compared with the PointNet network, the prediction accuracy of the PointNet + + network for most of the categories has been improved to varying degrees, the classification accuracy of each category is above 70%, and the accuracy of all categories is higher in the PointNet + + network, which indicates that the PointNet + + network has improved the classification accuracy of the network to a great extent, particularly for the classification of rocks, PointNet + + network shows a significant improvement in classification accuracy, which is helpful to provide reference information for the accurate identification of rock, the most important element, in quarry relic landscape.

Comparison of the overall dataset:

To verify that PointNet + + used in this paper is more advantageous compared to PointNet in segmenting the various elements of the quarry relic landscape, the results are shown in Table 5. The OA and mAcc of the PointNet + + network are improved by 46.74% and 48.75% compared to that of the PointNet network, and in terms of the throughput of the PointNet + + network improved by 59.82ins./s compared to the PointNet network, and in the Giga Floating-point Operations Per Second(GFLOPs) PointNet + + network improves 28.31 compared to PointNet network. Considering the combined accuracy and speed, the PointNet + + network used in this paper is more advantageous for the application of multi-objective in quarry relic landscape scenarios classification task.

Calculation and analysis of the volume of the quarry relic landscape

Using Trimble RealWorks software to perform volumetric calculations on each element of the point cloud, the triangulation mesh model is generated and repaired after importing the point cloud data file to ensure there are no holes or topological errors. Next, delineate the calculation area by drawing polygons or selecting the model surface and setting a suitable datum (Fig. 16). Adjust the resolution to 0.1 m. A resolution of 0.1 m preserves more details for objects with complex features on the surface, accommodates complex geometries, reduces errors, and improves the accuracy of volume calculations. Then, use the “Calculate Volume” function to calculate, and finally obtain the volume size of each element and the average height difference and other information.

Point cloud element and volumetric model.

Full size image

As shown in Table 6, the spatial volumes of different garden elements in the quarry relic landscape are calculated. It is worth stating that positive and negative space is an important concept in the study of quarry relic landscape: Positive space refers to the physically existing material elements that enclose a space, namely the rocks, buildings, plants, and so forth in this study. Negative space, on the other hand, refers to the virtual space enclosed by these elements, particularly the stone pits formed by rock quarrying³⁷. By introducing the concept of positive and negative space, the aim is to explore the spatial relationships of the existing quarry relic landscape (Fig. 17). The study of negative space mainly focuses on the virtual space above the water slough after single stone pits mining, while Yangshan Quarry. Relic does not belong to the space of inwardly shaped single quarrying mining, and the scope of the virtual space cannot be defined, so it is not included in the calculation.

Positive and negative spatial relationships in quarry relic landscape.

Full size image

The results reveal that the rock, plant, and architecture elements exhibit varying spatial occupancy ratios. Each landscape element has distinct characteristics.:

1. 1.

   The quarry relic landscape exhibits varying proportions of positive and negative space, ranging from a ratio of 1:0.27 to 1:1.11. Overall, the four quarry relic landscapes present composite spaces with a balanced sense of spatial enclosure. The quarried rocks create sub-spaces with containment relationships, such as the three kiln-cave-type gray spaces in Seven Star Rock and the gray space beneath the Xiangbi Stone in Caoshan Mountain. Among the landscapes, Quarry in Caoshan Mountain is the most open, followed by Shengshui Pond, and Seven-star Rock exhibiting the most pronounced spatial depression.

2. 2.

   In the positive space of the quarry relic landscape, plants and rocks are predominant. On average, plants constitute 60.49% of the positive space across the four quarry relic landscapes, while bare rocks account for 31.76%. This suggests that rocks and plants are the primary landscape features. The proportion of plants has surpassed the rocks, which as the main body of the quarry relic landscape, highlighting the importance of plant pruning for future quarry relics heritage conservation and restoration efforts.

3. 3.

   The architectural space generally occupies a small proportion. For instance, Shengshui Pond and Quarry in Caoshan Mountain have architectural proportions of only 0.93% and 0.76%, respectively. In contrast, in Yangshan Mountain’s Shifo Temple, architectural elements account for 19.86% of the flat area. Architecture on the quarry mountain primarily serves functional purposes such as providing areas for rest and viewing, with minimal impact on the overall spatial landscape effect.

The heritage data of the quarry relic includes both natural and man-made elements that are closely interconnected. It features naturally occurring landforms and material properties, alongside human-made additions such as cliff carvings and structures. These elements reflect the cultural concepts and design ideas of historical societies, serving as essential information for conservation efforts. Each of these different sources of information carries a unique value and needs to be effectively integrated and managed for conservation, which provides the possibility of a comprehensive understanding of the quarry relic heritage.

This study proposes a digital conservation strategy for quarry heritage relics and develops a Quarry Relics Landscape Information Modeling and Protection Management System (QRLIM&PMS). This system is designed based on the spatial ontology of quarry relics and incorporates data storage, analysis, and application. It facilitates the digital transformation, storage, and interactive display of landscape elements and their cultural connotations. The goal is to offer examples of the protection and utilization of these quarry relics, along with technical support (Fig. 18).

Digital heritage protection and management system for quarry relics.

Full size image

Construction and analysis of digital model information

This research showcases the heritage of quarry relics to the world. By utilizing digital scanning and deep learning techniques like semantic segmentation, we can now access integrated digital information that reveals issues previously unnoticed. This approach leads to discoveries that differ from traditional research methods and allows us to gather more geospatial, historical, and cultural information, such as spatial morphology, the shapes of quarry relics, and the stone carvings, etc.

Employing a multi-technology integration method enables us to obtain data on the garden entity, space, attributes, and more. Through digital analysis, we can examine the spatial layout of landscapes and the elements of quarry relics, unveiling the laws and mechanisms behind landscape creation. This research supports information technology applications like digital archiving of quarry relics and landscape restoration, facilitating the visualization and management of landscape data, which help to aid in identifying the core value of quarry relic heritage.

Spatial quantification

Spatial quantification involves the quantitative analysis of various aspects of the landscape scene at the quarry relic. By precisely measuring and analyzing the spatial layout and scale of each area within the quarry relic landscape, we can statistically evaluate the proportions of different landscape elements, such as rocks, plants, and architecture, within the overall space. This analysis aids in gaining a deeper understanding of the design’s logic and characteristics.

After concluding the previous section, the Shaoxing Quarry Relic features open-air mining, and depending on the mining methods used and the quality of the stone, the mining height of the quarry relic ranges from approximately 17 to 22 meters. The spatial characteristics of the quarry are classified as both positive and negative space, with a ratio of positive to negative space varying from 1:0.27 to 1:1.11. The quarry exhibits a composite spatial form, incorporating smaller spaces within a larger one. Rocks and plants make up the majority of the space, with rocks being the most significant element in terms of the heritage value of the quarry relics. Therefore, it is essential to protect the rocks effectively, while the vegetation should be managed in a targeted manner. The average proportion of architecture in the four quarry relic landscapes is just 7.75% (see Fig. 19). This contrasts with traditional garden environments, where architecture typically plays a dominant role.

Location and proportion of architecture in quarry relic landscape.

Full size image

Additionally, modern monitoring technology can be utilized to track pedestrian flow in these quarries in the future. This will allow for the analysis of tourist behavior patterns and spatial preferences, providing a scientific foundation for planning tours and arranging facilities within the landscape. Furthermore, by quantifying visitors’ spatial perception data—including visual accessibility and spatial comfort—there is an opportunity to optimize the design and enhance the overall experience quality of the quarry relic landscape.

Feature clustering

The clustering analysis focuses on classifying and extracting features from rock formations and stone carvings within the quarry relic landscape. By using a clustering algorithm, rocks with similar morphologies or features are grouped. This helps uncover the patterns and cultural significance embedded in the ancient stone excavation process. For example, rocks with specific shapes or combinations may be linked to particular quarrying techniques or uses. Similarly, rocks that share themes or styles may reflect the influence of a specific historical period or cultural movement. This perspective provides valuable insights into the cultural inheritance and evolution of the quarry relic landscape. The identification and classification of point cloud elements reveal that rocks, which form the core of the complex and varied spatial structure of quarry relic landscapes, are often obscured by vegetation, architecture, and other garden features. This makes it challenging to observe their distinct characteristics, resulting in a prolonged period during which artistic research on these rocks has been minimal.

Using deep learning methods, individual mountain stones were identified, revealing that these rocks exhibit certain clustering characteristics. Some rocks take on recognizable forms that resemble animals and constellations, while others showcase various geometric shapes resulting from different mining techniques, as illustrated in Table 7.

In ancient Chinese culture, symbols such as the bull, elephant, and lion represent auspicious images associated with wealth and good fortune. The layout that subtly corresponds to the constellations, such as the Big Dipper, has been involved in urban planning and garden design since the Qin and Han dynasties. These designs reflect ancient people’s pursuit of immortality and embody Chinese philosophical views on the unity of humanity and the cosmos. Additionally, geometric shapes such as L-shapes and fan shapes are more commonly found in quarry relics, a trend influenced by the open-air mining practices characteristic of the Shaoxing region.

Cultural feature

The excavation of cultural characteristics aims to showcase the rich cultural heritage associated with stone quarrying at the quarry relic site. By systematically analyzing and arranging aspects such as stone quality, color, and carving techniques, we can not only recreate the historical processes of quarrying and stone use but also provide valuable cultural resources for contemporary cultural revitalization. The stone Buddha and cliff carvings within the quarry relic landscape possess unique cultural value and significance. The quarry relic landscape is adorned with numerous rocky wall inscriptions. According to statistics, there are a total of 51 stone carvings spanning from the Ming Dynasty to modern times across the existing relics. The breakdown of these relics is as follows: the Houshan Mountain Quarry Relics have 8 carvings, the East Lake Quarry Relic features 7, the Keyan Quarry Relic has 14, and the Yangshan Quarry Relic contains 22.

In terms of distribution, the carvings from Houshan Quarry Relic are mainly found in the Caoshan Mountain area. The carvings from Keyan Quarry Relic are primarily located on the west side of the Stone Buddha and the cliffs of Seven-Star Rock. Meanwhile, the carvings from Yangshan Quarry Relic are concentrated around the Shifo Temple area. These cliff carvings boast high artistic value and significantly enhance the cultural and artistic aspects of the natural landscape (Fig. 20).

Statistics on cultural symbols in quarry relics landscape.

Full size image

1) Point cloud models can be utilized to measure the dimensions, sizes, carving forms, and depths of various cliff inscriptions. This allows for dynamic monitoring of these artworks, enabling timely and effective conservation and restoration efforts.

2) The stone Buddha statue, with its exquisite carving skills and solemn artistic expression, reflects the religious beliefs, aesthetic concepts, and the high level of skill of the craftsmen of that era. The inscriptions and patterns on the cliff carvings often document significant events, cultural exchanges, or anecdotes of celebrities from a specific historical period. For instance, based on the content of certain cliff carvings can be classified into four categories: religion, personal emotions, patriotism, and scenery praise. These categories reveal the local people’s aspirations for greatness or their longing for solitude amidst the beauty of nature.

By conducting an in-depth analysis of the cultural significance of stone Buddha statues and cliff carvings, we can integrate these ancient art forms with modern design concepts. For instance, we can incorporate the modeling styles and carving techniques of quarry relic Buddha statues into the restoration of contemporary quarrying sites. Additionally, we could draw inspiration from the layout and expression of cliff stone carvings in landscape design. This approach would allow us to create new cultural products and landscape elements that reflect the unique characteristics of quarry relic culture, ultimately preserving its value while encouraging innovation and development within this cultural heritage.

Multi-technology integration of data acquisition and processing

For the quarry relic, determining what information is critical and needs to be prioritized for conservation is one of the core issues in heritage conservation. This study introduces a multi-technology fusion method for data acquisition and processing, which effectively gathers the most comprehensive and accurate information for studying quarry relics. This approach also represents a significant methodological innovation within the research.

Geospatial information digitization

This study employs three-dimensional laser scanning to collect detailed point cloud data for the restoration of quarry relics. It also utilizes unmanned aerial vehicles (UAVs) to capture an overall image that illustrates the relationship between the relics and the surrounding environment. By combining these two methods, we aim to record heritage information comprehensively and in detail. Following this, professional software is used to classify and process the point cloud data to distinguish different materials and obtain high-quality mapping results. This approach provides valuable reference examples for the digital mapping of similar landscape types.

The three-dimensional data acquisition technology utilized in this study effectively captures the spatial coordinates, elevation, slope, aspect, and other geographical information of the quarry relic. This provides a high-precision data foundation for topographic and geomorphological analyses, as well as spatial modeling. When combined with remote sensing information, it allows for a broader understanding of the environmental context, including surrounding topography and vegetation cover. Additionally, the study involves the collection and monitoring of internal temperature, humidity, and light levels within the quarry relic. This data helps analyze the mechanisms impacting the rock and stone carvings, ultimately contributing to the development of a scientific protection plan.

Landscape element information digitization

By utilizing advanced semantic segmentation algorithms such as PointNet + +, we can accurately segment and analyze various landscape elements, including rocks, water, architecture, pavement, and plants. The PointNet + + algorithm has shown remarkable effectiveness, particularly in the segmentation of rocks and plants, achieving an accuracy rate exceeding 92%. Additionally, it operates at a significantly faster speed compared to PointNet. These results offer a dependable methodology for future analyses of landscape heritage, including quarry sites and landscapes primarily consisting of mountains and forests. This approach is anticipated to evolve into a unified and standardized method for managing data related to this type of heritage.

Textual, image and audio information digitization

In the research, the graphic and textual information related to the quarry relic is widely collected through various channels and digitized for storage. At the same time, The study emphasizes the integration of digitization technology with traditional methods such as surveying, mapping, and archaeology to complement and verify each other. This collected information includes rich graphic and textual information, such as historical records, cliff carvings, articles about travel, and stone inscriptions, as well as images, audio, video, etc. collected during the field research and interviews. These textual, image, and audio information are not only an important carrier of the history and culture of the quarry relic but also a key basis for in-depth study of its cultural connotation and social economy. Through digital storage, it is possible to realize the long-term preservation, convenient retrieval, and efficient use of such information.

Conservation of quarry relic heritage and development of a management system

The quarry relic landscape is a precious cultural heritage, and the protection and management system of the digitized results of the quarry relic will not only provide a solid technical foundation for heritage protection but also create new opportunities for transforming the results and public services. Ultimately, this approach will significantly promote the cultural value of quarry relics and support their preservation and development.

Heritage conservation

The integrated heritage data management system, utilizing digital technology, offers significant value in heritage application. In addition to storing heritage information, it can effectively monitor and prevent damage to these sites. For example, by analyzing data on the changes to quarry relics across different seasons and climatic conditions, it can accurately identify the factors causing damage, such as wind erosion, fluctuations in temperature and humidity, climatic influences, and inherent defects in the stone itself.

Moreover, this system equips policymakers and managers with precise assessment tools for both the value of heritage sites and the analysis of damage causation. This enables them to employ digital preview and virtual restoration techniques to develop and classify protection policies, ultimately enhancing the scientific basis and effectiveness of efforts to preserve quarry relics.

Achievement transformation

The heritage data of quarry relics offers designers a wealth of creative inspiration and data support, fostering a harmonious integration of traditional and modern design concepts. By deeply exploring the cultural significance and distinctive elements of quarry relics, we can highlight their unique value and encourage the revival and innovative development of quarry culture in contemporary society. For instance, integrating traditional stone carving art and other elements from the quarry landscape into modern landscape planning can create a new spatial environment that reflects regional cultural characteristics.

Additionally, analyzing visitor flow and behavioral preference data can optimize tour route planning for landscape heritage, enhancing the overall experience for visitors. The combination of point cloud data and VR technology allows visitors to immerse themselves in the historical changes and cultural atmosphere surrounding the quarry relics.

Furthermore, the heritage data of quarry relics has significant commercial development potential. It can lead to the creation of various products that embody regional cultural traits, such as miniature landscape models, cultural and creative items (including stationery, ornaments, and handicrafts), and games inspired by the cultural background of the quarry relic. This not only expands the dissemination channels for quarry relics but also provides new sources of economic income.

Public services

The use of digital technology to present information about cultural heritage related to quarry relics can take various forms, such as online virtual exhibitions, educational platforms, and mobile applications. These methods enhance public understanding and awareness of the cultural significance of the landscape, as well as the importance of its preservation.

For example, by developing science popularization applications on the intangible cultural heritage of mountain-chiseling techniques, and presenting the traditional quarrying techniques and cultural inheritance of the quarrying sites to the public in the form of vivid digital animations and video tutorials. Additionally, digital display technology is utilized to comprehensively present the historical appearance and cultural connotation of the ancient garden, building, and quarry relics, enhance the value and influence of these historical landscapes, and promote the widespread dissemination and inheritance of the culture associated with quarry relics.

Conclusion

This study developed a novel approach using point cloud data and deep learning to analyze and classify quarry relic landscape elements, with the Shaoxing Quarry Relics as a case study, revealing intricate spatial relationships within these heritage sites. The findings of the study indicate that:

1. 1.

   This study successfully acquired point cloud data from four quarry relic landscapes, demonstrating the feasibility of multi-technology fusion for data acquisition and providing a valuable example for similar studies.

2. 2.

   The PointNet + + deep learning network can efficiently and accurately recognize the landscape elements of quarry relic sites, particularly rocks and plants. This technology can be utilized as a semantic segmentation method for analyzing this type of heritage data, enhancing the accuracy and efficiency of element analysis.

3. 3.

   The space of the Shaoxing quarry relic features a composite space pattern, with the ratio of positive and negative space predominantly at 1:1. The positive space is dominated by plants and bare rocks, with only a small amount of architecture present. The rocks and stone pits serve as the most significant landscape elements, embodying nearly all the cultural information and heritage value of the site.

4. 4.

   Propose digital protection strategies and develop the QLIM&PMS system to implement comprehensive digital treatment for quarrying landscapes. Provide examples and technical support for the protection and utilization of quarry relic sites, while promoting the digitalization of landscape architecture heritage.

Although the paper presents only four cases, its innovative approach to deep learning serves as a pioneering demonstration for creating heritage databases related to quarrying remains. This lays a strong foundation for future research and practical applications in heritage management. It is hoped that this research will eventually lead to the establishment of a comprehensive database of quarry relic landscapes, which possess significant cultural and scenic value. Such efforts aim to preserve the memory of these heritage sites and transmit their cultural importance.