2024_05_20_6272fd3eea1461074c11g

Maize stem-leaf segmentation framework based on deformable point clouds
基于可变形点云的玉米茎叶分割框架

Tao Yang

, Tongyu

陶阳

、童宇

College of Information and Electrical Engineering, Shenyang Agricultural University, Shenyang 110161, China

沉阳农业大学信息与电气工程学院,沉阳 110161

School of Mathematics and Computer Science, Zhejiang Agriculture and Forestry University, Hangzhou 311300,China

浙江农林大学数学与计算机科学学院,浙江杭州 311300

School of Information and Intelligence Engineering, University of Sanya, Sanya 572022, China

三亚大学信息与智能工程学院, 三亚 572022
}

A R T I C L E I N F O
文章信息

Keywords: 关键词：

Maize Point Cloud Dataset
玉米点云数据集

Physically based deformation
基于物理的变形

Point Cloud Data Augmentation
点云数据增强

Stem-Leaf Segmentation 茎叶分割

Abstract 抽象的

A B S T R A C T The efficacy of three-dimensional (3D) point clouds in studying crop morphological structures is based on their direct and accurate data presentation ability. With deep-learning integration, organ segmentation from point clouds could serve as the basis for tremendous advancements in organ-level phenotyping. However, despite the potential, the acquisition of a sufficient number of annotated plant point clouds for practical model training remains a major hurdle. To help overcome this limitation, we constructed a 3D point-cloud dataset specifically for maize stem-leaf segmentation encompassing 428 maize plants ranging from 2 to 12 leaves. We also developed a point cloud enhancement strategy that uses highly controllable deformations to improve the morphological diversity of the training set significantly, while preserving the local geometric features of organs. Our dataset supports the generation of abundant training data from a limited number of labelled data, and we also provide a segmentation framework based on the augmented data to validate the efficiency of our enhancement technique. Two labelled data items were randomly chosen from our plant dataset based on every leaf number, yielding 22 labelled data items total, to produce several deformed point clouds for training the PointNet++ semantic segmentation model, as well as the hierarchical aggregation for the 3D instant segmentation (HAIS) model. These models were tested on 406 datasets, where the PointNet++ model secured a mean intersection-over-union (mIoU) in semantic segmentation and the HAIS model obtained an mean average precision (mAP) in instance segmentation. Following post-processing, an instance segmentation result of mAP was achieved with the HAIS model. These findings demonstrate that our method allows for the efficient training of organ segmentation models with minimal labelled data input in a reduced timeframe. Moreover, it offers an effective tool for point-cloud parsing in maize phenotyping research. Our Maize dataset is available from https://github.com/syau-miao/SignleMaizePointCloudDataSet.git, and the source code of our method can be found at https://github.com/yangxin6/Deformation3D.git.
摘要三维（3D）点云在研究作物形态结构方面的功效基于其直接、准确的数据呈现能力。通过深度学习集成，点云的器官分割可以作为器官水平表型分析巨大进步的基础。然而，尽管有潜力，获取足够数量的带注释的植物点云用于实际模型训练仍然是一个主要障碍。为了帮助克服这一限制，我们构建了一个专门用于玉米茎叶分割的 3D 点云数据集，其中包含 428 株玉米植株，叶子数量从 2 到 12 片不等。我们还开发了一种点云增强策略，该策略使用高度可控的变形来显着提高训练集的形态多样性，同时保留器官的局部几何特征。我们的数据集支持从有限数量的标记数据生成丰富的训练数据，并且我们还提供基于增强数据的分割框架来验证我们增强技术的效率。根据每个叶子的数量，从我们的植物数据集中随机选择两个标记数据项，总共产生 22 个标记数据项，以生成多个变形点云，用于训练 PointNet++ 语义分割模型，以及用于 3D 即时分割的分层聚合（ HAIS）模型。这些模型在 406 个数据集上进行了测试，其中 PointNet++ 模型在语义分割中获得了平均交集 (mIoU)，HAIS 模型获得了平均平均精度（ mAP）在实例分割中。经过后处理，使用HAIS模型获得了 mAP的实例分割结果。这些发现表明，我们的方法可以在更短的时间内以最少的标记数据输入有效地训练器官分割模型。此外，它为玉米表型研究中的点云解析提供了有效的工具。我们的玉米数据集可从 https://github.com/syau-miao/SignleMaizePointCloudDataSet.git 获取，我们方法的源代码可以在 https://github.com/yangxin6/Deformation3D.git 找到。

1. Introduction 一、简介

Maize plays a crucial role in sustaining the global food supply, and high-throughput phenotyping is required for future varietal improvements in the species. In recent years, the ubiquitous use of threedimensional (3D) sensing technologies, such as laser scanning (Su et al., 2018), multi-angle imaging (Wu et al., 2020), and LIDAR (Jin et al., 2019), has resulted in revolutionary advancements in the measurement of plant phenotype parameters. Precise organ-level segmentation is essential for measuring plant phenotypic features (e.g. the leaf length and width), and substantial research has been conducted on this issue.
玉米在维持全球粮食供应方面发挥着至关重要的作用，未来该物种的品种改良需要高通量表型分析。近年来，三维（3D）传感技术的广泛使用，例如激光扫描（Su et al., 2018）、多角度成像（Wu et al., 2020）和激光雷达（Jin et al., 2019）），在植物表型参数的测量方面带来了革命性的进步。精确的器官水平分割对于测量植物表型特征（例如叶长和宽度）至关重要，并且已经针对这个问题进行了大量研究。

Traditional approaches primarily target plants with specific morphologies using the topological and morphological features of these plants as a priori knowledge to create hand-crafted segmentation features (Elnashef et al., 2019; Jin et al., 2019). These methods do not require data annotation and can yield effective results for plants with certain morphologies. However, they exhibit limited generalisability and present challenges when they are applied to different entities. In addition, their high sensitivity to parameters significantly complicates the determination of optimal parameters that are suitable for a wider range of data.
传统方法主要针对具有特定形态的植物，使用这些植物的拓扑和形态特征作为先验知识来创建手工分割特征（Elnashef 等，2019；Jin 等，2019）。这些方法不需要数据注释，并且可以对具有某些形态的植物产生有效的结果。然而，它们的通用性有限，并且在应用于不同实体时面临挑战。此外，它们对参数的高敏感性使确定适合更广泛数据的最佳参数变得非常复杂。

Deep-learning methods use data-driven strategies for the direct learning of point cloud feature extraction and combination schemes that aid in segmentation tasks. Methods for applying deep learning to pointcloud segmentation are generally categorised as supervised, semisupervised, and unsupervised learning. Recent 3D-based supervised deep-learning methods have exhibited great potential in improving the generality and accuracy of organ segmentation. Supervised learningbased models require a vast amount of labelled data for training, and labelling point-cloud data is extremely time consuming and labour intensive. Despite the continual innovations and improvements of neural network structures, the problem of missing annotated plant 3D data has not been effectively solved, which has become a key factor restricting progress in this field. Although point-cloud data augmentation can alleviate the data dependency pressure to a certain extent, existing methods still have room for improvement in increasing plant morphological diversity and improving method controllability. Several unsupervised and semi-supervised poin-cloud segmentation deeplearning models have been proposed in recent years (Zhang et al., 2023b; Wang and Wang, 2020; de Gélis et al., 2023). These methods require little or no annotated data, thereby offering new perspectives for plant organ segmentation. However, the segmentation accuracy of these current methods still lags behind that of supervised learning approaches and their applicability in plant point clouds necessitates substantial work for validation.
深度学习方法使用数据驱动策略来直接学习点云特征提取和有助于分割任务的组合方案。将深度学习应用于点云分割的方法通常分为监督学习、半监督学习和无监督学习。最近基于 3D 的监督深度学习方法在提高器官分割的通用性和准确性方面表现出了巨大的潜力。基于监督学习的模型需要大量的标记数据进行训练，而标记点云数据极其耗时且耗费人力。尽管神经网络结构不断创新和改进，但植物3D标注数据缺失的问题仍未得到有效解决，成为制约该领域进展的关键因素。虽然点云数据增强可以在一定程度上缓解数据依赖压力，但现有方法在增加植物形态多样性和提高方法可控性方面仍有改进空间。近年来提出了几种无监督和半监督的点云分割深度学习模型（Zhang et al., 2023b; Wang and Wang, 2020; de Gélis et al., 2023）。这些方法需要很少或不需要注释数据，从而为植物器官分割提供新的视角。然而，这些当前方法的分割精度仍然落后于监督学习方法，并且它们在植物点云中的适用性需要大量的验证工作。

This study aims to design a set of methodologies that can mitigate the dependence on labelled data of supervised learning segmentation models. The contributions of our current study are threefold: we provide a maize point-cloud dataset for organ segmentation; we improve the point-cloud augmentation based on physical deformations to increase the morphological variety of the maize point-cloud training data while maintaining the local geometric features of organs; and we develop a maize stem-leaf segmentation framework based on deformable pointcloud data to enable deep-learning model training with minimal annotated data items.
本研究旨在设计一套方法，可以减轻监督学习分割模型对标记数据的依赖。我们当前研究的贡献有三个：我们提供了用于器官分割的玉米点云数据集；我们改进了基于物理变形的点云增强，以增加玉米点云训练数据的形态多样性，同时保持器官的局部几何特征；我们开发了一个基于可变形点云数据的玉米茎叶分割框架，以便能够使用最少的注释数据项进行深度学习模型训练。

2.1. Traditional plant point-cloud segmentation
2.1. 传统植物点云分割

Numerous traditional methods have been proposed for the segmentation of 3D plant organs, which commonly utilise hand-crafted features such as local geometric features and global topological structures. Elnashef et al. (2019) employed tensor features of local point clouds to segment the stems and leaves of maize plants during the seedling stage. Jin et al. (2019) introduced a median normalisation vector growth algorithm that rapidly segments maize stems and leaves using the directional features of local point clouds. Li et al. (Li et al., 2018a) achieved the regional growth leaf segmentation of dense plants using the directional and curvature features of local point clouds. The global topological structure of plants delineates the spatial and connective relationships between various organs such as stems and leaves, thereby serving as pivotal prior knowledge for segmentation. Several studies have adopted point-cloud skeletons to represent these global topological structures (Chaudhury and Godin, 2020; Miao et al., 2021b; Xiang et al., 2019). These skeleton methods consolidate a specific range of point clouds into skeleton vertices, which are subsequently interconnected to form edges based on their relational bonds, ultimately culminating in a graph structure (Wu et al., 2019). Thereafter, plant segmentation is realised by extracting the shortest paths between these skeleton vertices from the graph structure, guided by the topological relationships between the plant stems and leaves (Zermas et al., 2020). Although these traditional methods eliminate the need for model pre-training, they are limited in their segmentation efficiency and highly sensitive to parameter changes.
人们提出了许多传统的 3D 植物器官分割方法，这些方法通常利用手工制作的特征，例如局部几何特征和全局拓扑结构。埃尔纳谢夫等人。（2019）利用局部点云的张量特征来分割幼苗阶段玉米植株的茎和叶。金等人。 (2019)引入了一种中值归一化向量增长算法，该算法利用局部点云的方向特征快速分割玉米茎和叶。李等人。 (Li et al., 2018a)利用局部点云的方向性和曲率特征实现了密集植物的区域生长叶片分割。植物的整体拓扑结构描绘了茎、叶等各个器官之间的空间和连接关系，从而成为分割的关键先验知识。一些研究采用点云骨架来表示这些全局拓扑结构（Chaudhury and Godin，2020；Miao et al.，2021b；Xiang et al.，2019）。这些骨架方法将特定范围的点云合并为骨架顶点，这些顶点随后相互连接以基于其关系键形成边，最终形成图结构（Wu et al., 2019）。此后，在植物茎和叶之间的拓扑关系的指导下，通过从图结构中提取这些骨架顶点之间的最短路径来实现植物分割（Zermas et al., 2020）。尽管这些传统方法消除了模型预训练的需要，但它们的分割效率有限并且对参数变化高度敏感。

2.2. Plant point-cloud segmentation via deep learning
2.2.通过深度学习进行植物点云分割

Among the supervised, semi-supervised, and unsupervised learning paradigms for point-cloud segmentation, supervised learning exhibits significant effectiveness. For example, Jin et al. (2020) developed a voxel-based convolutional neural network (VCNN) that classifies and segments maize at different growth stages. Turgut et al. (Turgut et al., 2022) used the PointNet++ model (Qi et al., 2017) to achieve the semantic segmentation of rose bushes. Li et al., (2022a) similarly developed simultaneous semantic and instance segmentation for plant point clouds that differentiates stems and leaves, as well as single leaf instances. Li et al., (2022b) used the legacy PointNet network in conjunction with a mean-shift algorithm to perform semantic and instance segmentation of stems and leaves. Ao et al. (2022) used the point cloud-based feature learning model (Li et al., 2018b) to segment maize stems and leaves successfully, and Zhang et al. (Zhang et al., 2023a) developed a PointNet++ variant that segments maize tassels based on their ear tip features. Du et al. (2023) developed PSTPointGroup, which is a novel deep-learning approach for highresolution instance segmentation of siliques in rapeseed plant point clouds, using dynamic voxel encoding and attention mechanisms. Sun et al. (Sun et al., 2023b) used point and voxel representations and a point-VCNN (Liu et al., 2019) to segment the main stems, branches, and bolls of cotton plants in an end-to-end manner. Guo et al. (2023) presented a novel feature-fusion network for semantic segmentation that consists of voxel- and point-branches, and demonstrated its efficacy through comprehensive experiments on large-scale maize and tomato point clouds. Unsupervised and semi-supervised methods that are grounded in deep learning offer novel perspectives for plant point-cloud organ segmentation, primarily owing to their reduced reliance on extensive (or minimal) manual annotations for neural network training. Luo et al. (2023) proposed a weakly supervised 3D organ-level plant shoot segmentation model using an annotation-efficient point-cloud framework. Nonetheless, the practical applicability of unsupervised and semi-supervised methods in plant point-cloud segmentation remains a ripe area for further exploration.
在点云分割的监督、半监督和无监督学习范式中，监督学习表现出显着的有效性。例如，金等人。（2020）开发了一种基于体素的卷积神经网络（VCNN），可以对不同生长阶段的玉米进行分类和分割。图尔古特等人。（Turgut et al., 2022）使用PointNet++模型（Qi et al., 2017）实现了玫瑰花丛的语义分割。 Li 等人（2022a）同样开发了植物点云的同步语义和实例分割，可以区分茎和叶以及单叶实例。 Li 等人 (2022b) 使用传统的 PointNet 网络与均值平移算法结合来执行茎和叶的语义和实例分割。敖等人。（2022）使用基于点云的特征学习模型（Li et al., 2018b）成功分割了玉米茎叶，Zhang et al. (Zhang et al., 2023a) 开发了一种 PointNet++ 变体，可以根据耳尖特征分割玉米穗。杜等人。 (2023) 开发了 PSTPointGroup，这是一种新颖的深度学习方法，使用动态体素编码和注意机制，对油菜植物点云中的长角果进行高分辨率实例分割。孙等人。 (Sun et al., 2023b) 使用点和体素表示以及点 VCNN (Liu et al., 2019) 以端到端的方式分割棉花植物的主茎、分枝和棉铃。郭等人。 (2023)提出了一种由体素和点分支组成的用于语义分割的新型特征融合网络，并通过对大规模玉米和番茄点云的综合实验证明了其功效。基于深度学习的无监督和半监督方法为植物点云器官分割提供了新颖的视角，主要是因为它们减少了对神经网络训练的广泛（或最少）手动注释的依赖。罗等人。 (2023) 使用注释高效的点云框架提出了一种弱监督的 3D 器官级植物芽分割模型。尽管如此，无监督和半监督方法在植物点云分割中的实际适用性仍然是进一步探索的成熟领域。

2.3. Point-cloud data augmentation
2.3.点云数据增强

The current lack of sufficient annotated plant-based point-cloud data is a severely limiting factor. As a result, several scholars have presented affine transformations to enhance and expand existing data collections. For example, Shi et al. (2020) and Chen et al.(2021) introduced random flipping, global scaling, global rotation, and global translation methods to enhance the training set, resulting in improved 3D object detection performance in street scenes. Xin et al. (2023) validated the effectiveness of these affine transformations, particularly translation and rotation, for the 3D semantic segmentation of tomato plants using point clouds. Affine transformation can be applied to the point cloud of the entire plant or a single organ, thereby increasing the diversity of plant pose and organ distribution in 3D space. However, these methods cannot deal with deformation such as bending and twisting, and thus, their ability to enhance the diversity of geometric forms of plant organs is limited. Jitter (Yu and Grauman, 2017) and elastic transformations (Gabrani and Tretiak, 1996) alter the morphological features of data, resulting in greater data variability (Chen et al., 2021; Jiang et al., 2020; Ngo et al., 2023). However, the predominant focus of these methods on the deformation of entire point-cloud scenes, coupled with the largely random and challenging-to-fine-tune nature of these deformations, means that the gain in data diversity still needs to be strengthened.
目前缺乏足够的带注释的基于植物的点云数据是一个严重的限制因素。因此，一些学者提出了仿射变换来增强和扩展现有的数据收集。例如，石等人。 (2020) 和 Chen 等人 (2021) 引入了随机翻转、全局缩放、全局旋转和全局平移方法来增强训练集，从而提高了街道场景中的 3D 对象检测性能。辛等人。 (2023) 验证了这些仿射变换（特别是平移和旋转）对于使用点云对番茄植株进行 3D 语义分割的有效性。仿射变换可以应用于整个植物或单个器官的点云，从而增加3D空间中植物姿态和器官分布的多样性。然而，这些方法无法处理诸如弯曲和扭曲等变形，因此它们增强植物器官几何形态多样性的能力有限。抖动（Yu 和 Grauman，2017）和弹性变换（Gabrani 和 Tretiak，1996）改变数据的形态特征，导致更大的数据变异性（Chen 等，2021；Jiang 等，2020；Ngo 等， 2023）。然而，这些方法的主要关注点是整个点云场景的变形，再加上这些变形的很大程度上随机性和难以微调的性质，意味着数据多样性的增益仍然需要加强。

3. Materials and methods
3 材料与方法

3.1. Dataset construction
3.1.数据集构建

Field experiments were conducted in the experimental maize field at
田间试验在玉米试验田进行

Shenyang Agricultural University

between May and July 2019, 2020 and 2021. Five maize varieties (Xian Yu 335, LD145, LD502, LD586 and LD 1281) were planted in plots with a total area of

and row-row and plant-plant distances of 60 and

, respectively. Maize samples

were randomly selected and transplanted to pots in an indoor laboratory. Raw point clouds were obtained using a 3D laser scanner (FreeScan X3; Tianyuan Inc., Beijing, China).
2019年5月至7月、2020年和2021年沉阳农业大学

。在总面积为

。随机选择玉米样品

并移植到室内实验室的盆中。使用 3D 激光扫描仪（FreeScan X3；天元公司，北京，中国）获得原始点云。

The unprocessed dataset comprised point clouds representing both the pot and various surrounding objects. Using CloudCompare Stereo software, these extraneous points were manually eradicated, thereby isolating only the point cloud associated with the plant. Further refinements to eliminate noise involved the use of pass-through and statistical outlier-removal filters. To preserve the morphological characteristics of each plant while minimising the data scale of the point-cloud database, the farthest point-sampling approach was employed. Hence, point clouds exceeding 20480 points were downsampled to 20 480, whereas those with fewer than 20480 points were maintained.
未处理的数据集包含代表罐子和各种周围物体的点云。使用 CloudCompare Stereo 软件，手动消除这些无关点，从而仅隔离与植物相关的点云。消除噪声的进一步改进涉及使用直通和统计异常值去除滤波器。为了保留每种植物的形态特征，同时最小化点云数据库的数据规模，采用了最远点采样方法。因此，超过20480点的点云被下采样到20480，而少于20480点的点云被保留。

The number of point clouds from various maize varieties having different numbers of leaves is listed in Table 1. In 2019, owing to manual operations, a batch of stem point-cloud data only retained half of the full point cloud. We included these data in our dataset to test the power of our method to handle incomplete point clouds.
不同叶数的各玉米品种的点云数量如表1所示。2019年，由于人工操作，一批茎点云数据仅保留了完整点云的一半。我们将这些数据包含在数据集中，以测试我们的方法处理不完整点云的能力。

In this study, our own Label3DMaize(Miao et al., 2021a) toolkit (Fig. 1) was used for semantic and instance annotations of maize point clouds. This toolkit achieves semiautomatic point cloud segmentation and annotation at different growth stages through a series of operations, including stem, coarse, fine and sample-based segmentations. Users only need to interactively select some key areas in the core point cloud (e.g. bottom or top of stems and the highest point of each non-stem organ), and the software automatically segments those instances using a series of unsupervised segmentation algorithms. If some areas are not accurately annotated, the toolkit provides a fine segmentation function that enables the correction of misclassified points through simple manual interactions. After using Label3DMaize to label the point cloud of the maize plants, the central axis of the stem point cloud coincided with the Z-axis, and the growth direction of the plant pointed towards the positive direction of the Z-axis. Each organ instance was then assigned an integer label (Fig. 1C), and the label of the stem instance was set to zero. As such, the leaf instances were assigned labels, beginning at one from the bottom to the top of the plant. The semantic label ("stem" or "leaf") of the point cloud can be determined directly from the instance label. Tests have shown that the toolkit takes approximately 4-10 min to segment a maize shoot, and the obtained annotation results are satisfactory for model training.
在本研究中，我们自己的 Label3DMaize（Miao et al., 2021a）工具包（图 1）用于玉米点云的语义和实例注释。该工具包通过茎分割、粗分割、细分割和基于样本的分割等一系列操作，实现不同生长阶段的半自动点云分割和标注。用户只需交互地选择核心点云中的一些关键区域（例如茎的底部或顶部以及每个非茎器官的最高点），软件就会使用一系列无监督分割算法自动分割这些实例。如果某些区域没有准确标注，该工具包提供了精细的分割功能，可以通过简单的手动交互来纠正错误分类的点。使用Label3DMaize对玉米植株的点云进行标注后，茎点云的中心轴与Z轴重合，且植株的生长方向指向Z轴的正方向。然后为每个器官实例分配一个整数标签（图1C），并将茎实例的标签设置为零。因此，叶子实例被分配了标签，从植物的底部到顶部从一个标签开始。点云的语义标签（“茎”或“叶”）可以直接从实例标签确定。测试表明，该工具包对玉米芽进行分割大约需要4-10分钟，获得的注释结果满足模型训练的要求。

3.2. The segmentation pipeline
3.2.分割管道

Our point-cloud enhancement pipeline includes four parts: plant annotation, physics-based plant deformation, point-cloud segmentation model training and post-processing of instance segmentation results.
我们的点云增强流程包括四个部分：植物注释、基于物理的植物变形、点云分割模型训练和实例分割结果的后处理。

3.2.1. Plant point-cloud deformation
3.2.1.植物点云变形

A physics-based elastic deformation simulation framework was used to perform point-cloud data augmentation. Our framework was developed based on the open-source VegFEM simulation library(Sin et al., 2013), which is computationally efficient.
基于物理的弹性变形模拟框架用于执行点云数据增强。我们的框架是基于开源 VegFEM 模拟库（Sin et al., 2013）开发的，计算效率很高。

The finite element method (FEM) was used to discretise the partial differential equations of solid continuum mechanics. The deformable maize plant was represented by a volumetric mesh consisting of 3D hexahedral elements. As such, the plant's point cloud was used to generate a hexahedral volumetric mesh. First, the minimum bounding box of the point cloud was calculated and segmented at a certain resolution to obtain a series of adjacent hexahedra; the volumes lacking point clouds were removed. In this study, the length of each hexahedron edge was set to 0.02 times the longest edge length of the bounding box. The semantic and instance labels of point clouds within each hexahedron were counted, and the semantic and instance labels of the hexahedron were designated as labels with the highest number of point clouds.
有限元法（FEM）用于离散固体连续介质力学的偏微分方程。可变形玉米植株由 3D 六面体元素组成的体积网格表示。因此，植物的点云用于生成六面体体积网格。首先计算点云的最小包围盒，并以一定的分辨率进行分割，得到一系列相邻的六面体；缺少点云的体积被删除。在本研究中，每个六面体边的长度设置为边界框最长边长度的0.02倍。统计每个六面体内点云的语义和实例标签，并将六面体的语义和实例标签指定为点云数量最多的标签。

In solid mechanics, three-dimensional deformable objects can be modelled using nonlinear elastic partial differential equations, which are represented by ordinary differential equations (Fig. 1):
在固体力学中，三维可变形物体可以使用非线性弹性偏微分方程进行建模，用常微分方程表示（图1）：

where

and

represent 3D displacement, velocity and acceleration of

vertices in the volume mesh, respectively.

is the mass matrix, which is formed by stacking the mass of each element. In this study, the mass of each element was set to 1000.

designate the internal elastic forces, and

designate the external forces specified by users to drive maize plant volumetric mesh deformations.

is the Rayleigh damping matrix, which is expressed by the following formula:
其中

和

分别表示体积网格中

顶点的3D位移、速度和加速度。

是质量矩阵，由各个元素的质量叠加而成。在本研究中，每个单元的质量设置为1000。

表示内部弹力，

表示用户指定的驱动玉米植株体积网格变形的外力。

为瑞利阻尼矩阵，用以下公式表示：

where

is the tangent stiffness matrix,

and

are scalar parameters that can be adjusted by the user to control damping effects,

is used to slow down global deformation and

is used to dampen the relative deformation velocity differences, which is useful in removing temporal high-frequency instabilities. In this study, these parameters are

and

.
其中

是切线刚度矩阵，

和

是用户可以调整以控制阻尼效果的标量参数，

用于减慢全局变形，

用于抑制相对变形速度差异，这对于消除时间高频不稳定性很有用。在本研究中，这些参数是

和

。

The internal elastic forces

depend on the chosen elastic material properties (i.e. isotropic, anisotropic, linear and nonlinear). In this study, the maize plant was set as a St. Venant Kirchhoff (StVK) material. Note that StVK uses the Green-Lagrange strain tensor,

, and is defined by the energy density function,

:
内部弹性力

取决于所选弹性材料特性（即各向同性、各向异性、线性和非线性）。在这项研究中，玉米植株被设定为圣维南基尔霍夫（StVK）材料。请注意，StVK 使用格林-拉格朗日应变张量

，并由能量密度函数

定义：

Table 1 表格1

Number of maize sample varieties with different numbers of leaves.
具有不同叶子数量的玉米样本品种的数量。

Number of

leaves

Number of

samples

XY335 data

bulk

LD145 data

bulk

LD502 data

bulk

LD586 data

bulk

LD1281

databulk

Number of stem- 茎数-

missingdata

Number of stem- 茎数-

completedata

100

Fig. 1. Maize plant point-cloud annotation: (A) plant point cloud, (B) Label3DMaize annotation software, and (C) visualisation of point-cloud annotation results. Each organ instance is assigned an integer label and is represented by a colour.
图 1. 玉米植株点云标注：(A) 植株点云，(B) Label3DMaize 标注软件，(C) 点云标注结果可视化。每个器官实例都分配有一个整数标签并用颜色表示。

where

is the

identity matrix,

is the vertex of the undeformed volumetric mesh and

is the position of the vertex,

, after deformation (i.e.

is the Young's modulus, and

is the Poisson ratio. In this study, for the stem material,

and

were set to

and 0.01 , respectively. For the leaf material,

and

were set to

and 0.45 , respectively. Symbol

represents the trace of the strain tensor matrix,

, and the colon (:) denotes the tensor contraction (i.e. element-wise dot product). The strain energy,

, of any given deformation is computed by integrating the energy density,

, over the entire body,

:
其中

是

单位矩阵，

是未变形体积网格的顶点，

是顶点的位置，

，变形后（即

）。

是杨氏模量，

是泊松比。在本研究中，对于茎材料，

和

分别设置为

和0.01。对于叶子材质，

和

分别设置为

和 0.45 。符号

表示应变张量矩阵

的迹，冒号 (:) 表示张量收缩（即逐元素点积）。任何给定变形的应变能

是通过对整个主体

上的能量密度

进行积分来计算的：

The internal elastic forces,

, equal to the gradient of strain energy,

, with respect to the components of the deformation vector,

. For STVK materials, the internal forces and tangent stiffness matrix are cubic and quadratic polynomials, respectively. The coefficients of these polynomials can then be derived analytically using the integration of the FEM shape functions over each hexahedron element.
内部弹性力

等于应变能梯度

，相对于变形矢量的分量

。对于 STVK 材料，内力和切线刚度矩阵分别是三次多项式和二次多项式。然后可以使用每个六面体元素上的 FEM 形状函数的积分来分析导出这些多项式的系数。

To time step the deformation simulation at runtime, we numerically integrated the system from Eq. (1). In this study, an implicit Newmark integrator was used for numerical integration. Given current deformations

at time

, the implicit Newmark method obtained a nonlinear equation for the next deformations,

, at time

:
为了在运行时对变形模拟进行时间步长，我们对方程式中的系统进行了数值积分。 (1).在本研究中，使用隐式纽马克积分器进行数值积分。给定时间

的当前变形

，隐式 Newmark 方法获得了时间

的下一个变形

的非线性方程:

where

and

are user-chosen parameters. In practice, these values are often set to

and

. To solve for

, the Newton-Raphson procedure was performed iteratively, beginning with

to generate the updated guesses,

, using the

and tangent stiffness matrix,

, at the current guess,

:
其中

和

是用户选择的参数。在实践中，这些值通常设置为

和

。为了求解

，迭代执行 Newton-Raphson 过程，从

开始，使用

，当前猜测

：

when

falls below a certain threshold or number of iterations exceeds a certain number, the iteration process is terminated.
当

低于一定阈值或迭代次数超过一定次数时，迭代过程终止。

When external forces are applied to the vertices of the maize plant's volumetric mesh, the mesh undergoes elastic deformation. Using a simple trilinear interpolation, the displacement of the vertices of each hexahedral element can be transferred to a point cloud inside the element. Through deformation, we only changed the coordinates of each point, without changing its other properties such as the index, colour, and intensity. As the normal vector of a point will change following deformation, it needs to be recalculated.
当外力施加到玉米植株体积网格的顶点时，网格会发生弹性变形。使用简单的三线性插值，每个六面体单元的顶点的位移可以转移到单元内的点云。通过变形，我们只改变了每个点的坐标，而没有改变其其他属性，例如索引、颜色和强度。由于变形后点的法向量会发生变化，因此需要重新计算。

To make the deformed maize plants more diverse in terms of plant shape, we carefully designed a set of methods for applying the necessary external forces. External forces were applied to key areas in the volumetric mesh, including the tip vertices of each leaf and the bottom vertices of the stem. The stem vertices were sorted from small to large according to the Z-coordinate, and the smallest eight vertices were set as the bottom vertices of the stem. To identify the tip vertices of each leaf, we computed the Euclidean distances of the volumetric mesh vertex of each leaf instance from the Z-axis. Vertices that were greater than fourfifths of the maximum distance were defined as the leaf tip area vertices.
为了使变形玉米植株的株形更加多样化，我们精心设计了一套施加必要外力的方法。外力被施加到体积网格中的关键区域，包括每片叶子的尖端顶点和茎的底部顶点。将茎顶点按照Z坐标从小到大排序，最小的8个顶点设置为茎的底部顶点。为了识别每个叶子的尖端顶点，我们计算了每个叶子实例的体积网格顶点距 Z 轴的欧几里德距离。大于最大距离五分之四的顶点被定义为叶尖面积顶点。

During deformation, we first applied a force to the stem. We randomly selected one vertex from the bottom vertices of the stem as the leader point for applying force and designated the five neighbourhood vertices of this point as auxiliary points according to the Euclidean distance. The external force exerted on the leader point was randomly assigned values ranging from

in the

-direction, from

in the Y-direction, and from

in the

direction, where

represent user-defined parameters. The magnitude of these parameters directly influences the random variability of the external forces applied, leading to more pronounced deformations in the point clouds. We calculated the distances from each auxiliary point to the leader point and sorted them in ascending order. If the random force at the leader point was

, then the force applied at the auxiliary point ranked

was

. This method of applying force causes changes in stem length and curvature.
在变形过程中，我们首先向茎施加力。我们从茎的底部顶点中随机选择一个顶点作为施力的引导点，并根据欧氏距离指定该点的五个邻域顶点作为辅助点。施加在引导点上的外力在

方向上随机分配从

到

的值，从

到

在 Y 方向上，以及在

方向上从

到

，其中

到

代表用户定义的参数。这些参数的大小直接影响所施加的外力的随机变化，导致点云的变形更加明显。我们计算每个辅助点到引导点的距离，并按升序对它们进行排序。如果引导点处的随机力为

，则排名为

的辅助点处施加的力为

。这种施力方法会导致阀杆长度和曲率发生变化。

As a result of stem deformation, leaf instances were deformed. Before deformation, we increased the material parameters of the stem (i.e. mass density, Young's modulus and Poisson's ratio) to

and 0.45 , respectively. We set the vertices of the fixed stem top and bottom points to displacements of zero to make the stem difficult to change. We then applied a force to deform the leaves. Like the stem, we selected leader and auxiliary points from the leaf tip vertices. The method of adding external force to the auxiliary point was the same as that of the stem.
由于茎变形，叶子实例也发生了变形。在变形之前，我们将茎的材料参数（即质量密度、杨氏模量和泊松比）分别增加到

和 0.45 。我们将固定阀杆顶部和底部点的顶点设置为零位移，以使阀杆难以改变。然后我们施加力使叶子变形。与茎一样，我们从叶尖顶点选择引导点和辅助点。辅助穴位加外力的方法与茎相同。

These effects are illustrated in Fig. 2.
这些效果如图 2 所示。

3.2.2. Point-cloud segmentation model
3.2.2.点云分割模型

The deep-learning models applied to point-cloud segmentation tasks can be broadly classified into point- and voxel-based methods. To verify the effectiveness of enhanced data through their deformations in actual training models, this study used PointNet++ (Qi et al., 2017), a pointbased model, and HAIS(Chen et al., 2021), a voxel-based model) to train semantic and instance segmentation models, respectively.
应用于点云分割任务的深度学习模型可以大致分为基于点的方法和基于体素的方法。为了通过实际训练模型中的变形来验证增强数据的有效性，本研究使用基于点的模型PointNet++（Qi et al., 2017）和基于体素的模型HAIS（Chen et al., 2021）来分别训练语义和实例分割模型。

PointNet++, which was developed by Qi et al. (2017), is a deeplearning framework that is tailored for point-cloud processing. This model extends the original PointNet (Charles et al., 2017) by introducing hierarchical neural network structures to capture local features at multiple scales. Its architecture is characterised by set abstraction layers, which recursively apply PointNet to partitions of the input point set. Each layer comprises sampling, grouping, and PointNet layers. The sampling process employs farthest-point sampling to determine the centroid points, around which grouping forms local regions. Subsequently, the PointNet layers process these regions, extracting features that are invariant to the order of points. These features are aggregated across the network, enabling PointNet ++ to learn complex geometric structures within point clouds of varying densities.
PointNet++，由 Qi 等人开发。（2017）是一个专为点云处理量身定制的深度学习框架。该模型通过引入分层神经网络结构来捕获多个尺度的局部特征，从而扩展了原始的 PointNet（Charles 等人，2017）。其架构的特点是集合抽象层，递归地将 PointNet 应用于输入点集的分区。每层都包含采样层、分组层和 PointNet 层。采样过程采用最远点采样来确定质心点，围绕该质心点分组形成局部区域。随后，PointNet 层处理这些区域，提取与点顺序不变的特征。这些特征在网络中聚合，使 PointNet ++ 能够学习不同密度的点云中的复杂几何结构。

HAIS, which was initially proposed by Chen et al. (2021), is a voxelbased model for instance segmentation in point clouds. It converts point clouds into a voxel grid to simplify the data structure for subsequent processing. At the core of HAIS is a hierarchical agglomerative clustering algorithm that incrementally merges voxels or voxel groups based on their spatial proximity and feature similarity. The process begins with individual voxels and progresses to larger clusters, with the identification of distinct object instances. A key feature of HAIS is the use of a learned feature embedding for each voxel, which ensures that clustering decisions are informed by relevant spatial and contextual data. This approach enables HAIS to segment individual instances in complex point clouds effectively.
HAIS最初由Chen等人提出。（2021），是一种基于体素的模型，用于点云中的实例分割。它将点云转换为体素网格，以简化后续处理的数据结构。 HAIS 的核心是一种分层凝聚聚类算法，该算法根据体素或体素组的空间邻近性和特征相似性增量合并体素或体素组。该过程从单个体素开始，并逐渐发展到更大的簇，并识别不同的对象实例。 HAIS 的一个关键特征是使用每个体素的学习特征嵌入，这确保聚类决策由相关空间和上下文数据通知。这种方法使 HAIS 能够有效地分割复杂点云中的各个实例。

3.2.3. Post-processing of instance segmentation results
3.2.3.实例分割结果的后处理

The HAIS model is prone to under-segmentation, often merging adjacent leaves into a single instance. The confidence in these undersegmented areas may be low, resulting in them being marked as unsegmented instances (a label value of 0 ); however, the model may also mistakenly identify these areas as a single leaf instance. The first scenario can be directly identified by verifying whether the label value is 0 . For the second scenario, the number of point clouds for each leaf instance is counted, and the point cloud is considered as undersegmented if the count exceeds a certain threshold.
HAIS 模型容易出现分割不足的情况，通常会将相邻的叶子合并到单个实例中。这些未分割区域的置信度可能较低，导致它们被标记为未分割实例（标签值为 0 ）；然而，模型也可能错误地将这些区域识别为单个叶实例。第一种场景可以通过验证标签值是否为 0 来直接识别。对于第二种情况，对每个叶实例的点云数量进行计数，如果计数超过某个阈值，则认为点云未分割。

After identifying all under-segmented point clouds, we applied the distance-filed based segmentation pipeline (DFSP) (Wang et al., 2023) that was previously developed by our team for further segmentation. DFSP can rapidly locate and segment organs by encoding the global spatial structure and local connections in maize plants. Upon entering a base point (set to

in this study), DFSP constructs a Minkowski distance field from other point clouds to the base point. QuickShift++ then estimates the local maximum regions from the distance field for graph segmentation.
在识别出所有未分割的点云后，我们应用了我们团队之前开发的基于距离场的分割管道（DFSP）（Wang et al., 2023）进行进一步分割。 DFSP 可以通过编码玉米植株的全局空间结构和局部连接来快速定位和分割器官。输入基点（在本研究中设置为

）后，DFSP 构造从其他点云到基点的 Minkowski 距离场。 QuickShift++ 然后根据距离场估计局部最大区域以进行图分割。

Both the HAIS model and DFSP may also result in over-segmentation. In this study, over-segmentation was identified simply by the number of point clouds contained within an instance, and over-segmented instances were directly merged into the nearest instance. It's important to note that the distance between two instances is measured as the Euclidean distance between the closest points of the instances.
HAIS模型和DFSP也可能导致过度分割。在本研究中，过度分割仅通过实例中包含的点云数量来识别，并且过度分割的实例被直接合并到最近的实例中。值得注意的是，两个实例之间的距离是通过实例最近点之间的欧几里得距离来测量的。

4. Results 4. 结果

4.1. Point-cloud deformation effect
4.1.点云变形效果

Fig. 3 presents a selection of plant point clouds that were randomly generated by the methods outlined in this paper using a physics-based deformation framework for plant point clouds. This approach enables the application of varying force locations, directions, and magnitudes to each organ, thereby allowing for completely distinct deformation trends and extents for each organ. Such control over the localised morphology of organs significantly enhances the morphological diversity of plant point clouds. In addition, as the deformation technique is rooted in elastic deformation, the application of force to localised regions of each organ affects the corresponding organ and the entire plant through the transmission of force within the medium, which results in overall plant deformation. Conversely, the deformation of an organ is also subject to the internal forces of the entire plant. This interplay between the local organ and global plant forces ensures that the stem-leaf connections, phyllotaxy, and other topological features remain accurate post-
图 3 展示了通过本文概述的方法使用基于物理的植物点云变形框架随机生成的植物点云选择。这种方法能够对每个器官施加不同的力位置、方向和大小，从而允许每个器官完全不同的变形趋势和程度。这种对器官局部形态的控制显着增强了植物点云的形态多样性。另外，由于变形技术植根于弹性变形，对每个器官的局部区域施加力，通过介质内的力传递影响相应的器官和整个植物，从而导致植物整体变形。反之，器官的变形也受到整个植物的内力的作用。局部器官和整体植物力之间的相互作用确保了茎叶连接、叶序和其他拓扑特征在事后保持准确。

一个

Fig. 2. Leaf and stem deformation process and its effects: (A) volumetric mesh of the maize plant point cloud from Fig. 1, (B) instance labels and key areas of the volumetric mesh, (C) force applied to the stem, (D) force applied to the leaves, and (E) deformed volumetric mesh and corresponding point clouds.
图 2. 叶和茎变形过程及其影响：(A) 图 1 中玉米植株点云的体积网格，(B) 体积网格的实例标签和关键区域，(C) 施加到茎上的力，（D）施加到叶子上的力，以及（E）变形的体积网格和相应的点云。

Fig. 3. Randomly generated plant point clouds using the method described. (A) and (C) show the original annotated point clouds. (B) and (D) display the point clouds deformed using the presented method. Note: Within each plant, points belonging to the same organ instance are depicted in identical colours, with different organs represented by different colours.
图 3.使用所描述的方法随机生成植物点云。 (A) 和 (C) 显示原始注释点云。 (B) 和 (D) 显示使用所提出的方法变形的点云。注意：在每株植物中，属于同一器官实例的点用相同的颜色表示，不同的器官用不同的颜色表示。

Fig. 4. Visualization of deformation results for stem and leaf organs: (A) original point cloud of the stem before deformation, (B) point clouds of the stem after five types of deformation, (C) unaltered leaf point cloud, and (D) point clouds of leaves after five types of deformation. Note: The top sub-figures in (A) and (B) show the frontal views of the stem point clouds, and the bottom sub-figures display the bottom views; in (C) and (D), green and yellow represent different leaf point clouds.
图4.茎叶器官变形结果可视化：(A)变形前茎的原始点云，(B)五种变形后茎的点云，(C)未改变的叶子点云，( D）五种变形后的树叶点云。注：（A）和（B）中顶部子图显示茎点云的正视图，底部子图显示底部视图； (C)和(D)中，绿色和黄色代表不同的叶子点云。

deformation 形变

Although some external forces of excessive magnitude may render the plants less 'realistic', the physics-based deformation preserves the local geometric characteristics of the organs well (as shown in Fig. 4). For instance, the cylindrical features of a bent stem remained intact and the undulating surface features of bent leaves are preserved, which is beneficial for the model to learn the local geometric features of organs.
尽管一些过大的外力可能会使植物不太“真实”，但基于物理的变形很好地保留了器官的局部几何特征（如图4所示）。例如，弯曲茎的圆柱形特征保持完整，弯曲叶子的起伏表面特征被保留，这有利于模型学习器官的局部几何特征。

This method provides significant flexibility, enabling the creation of maize plants with unique shapes through various external force application strategies. By adjusting the magnitude and application points of the forces, we simulated scenarios such as maize plant lodging (Fig. 5 (A)), variations in leaf inclination (Fig. 5(B)), and the state of the leaves after being bent or broken (Fig. 5(C)).
该方法提供了显着的灵活性，能够通过各种外力应用策略创造出具有独特形状的玉米植株。通过调整力的大小和作用点，我们模拟了玉米植株倒伏（图5（A））、叶片倾斜度变化（图5（B））以及叶片弯曲后的状态等情况或破损（图5（C））。

4.2. Data augmentation effects
4.2.数据增强效应

The data deformation capability of the presented method establishes it as a novel data-augmentation tool that is suitable for plant point-cloud segmentation. We compared the effects on enhancing the segmentation model performance using our method with eight commonly used pointcloud augmentation techniques (Table 5). We employed these dataaugmentation techniques to enlarge our dataset and train the segmentation models. The implementation details of these eight enhancement algorithms are provided in Appendix A.1. The setup of the training parameters is documented in Appendix A.2, while the strategy for data preparation for model training and testing is described in Appendix A.3. Table 2 details the number of training data points generated through these augmentation techniques and the segmentation accuracy of the PointNet++ and HAIS models trained with these augmented datasets. In all tables in this paper, the best result is in boldface, whereas the secondbest result is underlined.
该方法的数据变形能力使其成为一种适用于植物点云分割的新型数据增强工具。我们将使用我们的方法与八种常用的点云增强技术对增强分割模型性能的效果进行了比较（表 5）。我们采用这些数据增强技术来扩大数据集并训练分割模型。附录 A.1 提供了这八种增强算法的实现细节。训练参数的设置记录在附录 A.2 中，而模型训练和测试的数据准备策略在附录 A.3 中描述。表 2 详细介绍了通过这些增强技术生成的训练数据点的数量以及使用这些增强数据集训练的 PointNet++ 和 HAIS 模型的分割精度。在本文的所有表格中，最好的结果用黑体字表示，而次好的结果用下划线表示。

Visual representations of stem-leaf segmentation by PointNet++ and HAIS models trained with different augmentation methods are presented in Figs. 6 and 7, respectively.
图 2 和图 3 给出了使用不同增强方法训练的 PointNet++ 和 HAIS 模型的茎叶分割的视觉表示。分别为6和7。

An analysis of the segmentation metrics for PointNet , as outlined in Table 2, reveals significant findings. All augmentation methods substantially enhanced the accuracy of PointNet. Specifically, the method presented in this study achieved the highest segmentation accuracy with 22,000 data points and the second-highest accuracy with 2200 data points. Overall, the organ-level augmentation methods outperformed the plant-level methods in semantic segmentation tasks, especially with a smaller training dataset (2200 data points), where the segmentation enhancement, particularly of the stems, was markedly better for organ-level augmentations. This superiority was notably evident in the visual outcomes, especially in the stem segmentation, where the plant-level methods often resulted in under-segmentation, as illustrated in Fig. 6.
如表 2 所示，对 PointNet 分段指标的分析揭示了重要的发现。所有增强方法都大大提高了 PointNet 的准确性。具体来说，本研究中提出的方法在 22,000 个数据点上实现了最高的分割精度，在 2200 个数据点上实现了第二高的精度。总体而言，器官级增强方法在语义分割任务中优于植物级方法，特别是在训练数据集较小（2200 个数据点）的情况下，其中分割增强（尤其是茎的分割增强）对于器官级增强明显更好。这种优越性在视觉结果中尤其明显，特别是在茎分割中，植物级方法经常导致分割不足，如图 6 所示。

Instance segmentation, which is a more complex task than semantic segmentation, exhibited considerable improvement with our method, particularly in segmenting individual leaf instances, according to the HAIS instance segmentation metrics. The HAIS model trained with 22 real label data points tended to produce under-segmentation results, merging adjacent leaves into a single instance. All augmentation methods can address under-segmentation by increasing the diversity of the data. Our method with a model trained on 2200 data points surpassed models trained with other methods that were expanded to 22,000 data points. This tenfold difference in the data volume highlights the effectiveness of our approach in instance segmentation tasks. The capability of our method to deform organs in a diverse manner increases the geometric variability between adjacent organs, thereby enhancing the leaf segmentation outcomes. As can be observed from Fig. 7, the HAIS models trained with all augmentation methods still struggled to resolve under-segmentation completely, with adjacent leaf areas often being mistakenly segmented as a single instance, particularly in the lower or upper leaf regions. Compared to other augmentation methods, models trained with our method and rotation augmentation can handle under-segmentation of the upper leaves better. Although the model may not effectively segment closely packed upper leaves, identifying them as an unsegmented instance lays the groundwork for further postprocessing in these areas. The trained model of our method also significantly outperformed other augmentation techniques for the bottom leaves. As illustrated in Fig. 7, other methods mistakenly segmented multiple bottom leaves as one instance, whereas our method segmented these leaves more effectively. The experimental results indicate that the deformed data of our method enable the model to learn individual leaf features better than other augmentation methods, thereby improving the model segmentation accuracy.
实例分割是比语义分割更复杂的任务，根据 HAIS 实例分割指标，我们的方法显示出相当大的改进，特别是在分割单个叶实例方面。使用 22 个真实标签数据点训练的 HAIS 模型往往会产生分割不足的结果，将相邻的叶子合并到单个实例中。所有增强方法都可以通过增加数据的多样性来解决分割不足的问题。我们使用在 2200 个数据点上训练的模型的方法超越了使用扩展到 22,000 个数据点的其他方法训练的模型。数据量的十倍差异凸显了我们的方法在实例分割任务中的有效性。我们的方法以多种方式使器官变形的能力增加了相邻器官之间的几何变异性，从而增强了叶子分割结果。从图 7 中可以看出，使用所有增强方法训练的 HAIS 模型仍然难以完全解决分割不足的问题，相邻叶区域经常被错误地分割为单个实例，特别是在下部或上部叶区域。与其他增强方法相比，使用我们的方法和旋转增强训练的模型可以更好地处理上部叶子的欠分割。尽管该模型可能无法有效地分割紧密排列的上部叶子，但将它们识别为未分割的实例为这些区域的进一步后处理奠定了基础。我们方法的训练模型也显着优于其他底部叶子增强技术。如图所示如图7所示，其他方法错误地将多个底部叶子分割为一个实例，而我们的方法更有效地分割这些叶子。实验结果表明，我们方法的变形数据使模型能够比其他增强方法更好地学习单个叶子特征，从而提高模型分割精度。

4.3. Segmentation results
4.3.分割结果

The maize stem-leaf semantic segmentation PointNet++ model trained in this study achieved an mIOU of

, and the instance segmentation HAIS model achieved a mAP of

. We also compared these two models with some of the latest unsupervised segmentation methods, including the unsupervised deep-learning semantic segmentation method GrowpSP (Z. Zhang et al., 2023), and the traditional unsupervised instance segmentation algorithms SS (Miao et al., 2021b) and DFSP (Wang et al., 2023), which are specifically designed for maize stem-leaf segmentation.
本研究训练的玉米茎叶语义分割PointNet++模型取得了

的mIOU，实例分割HAIS模型取得了

的mAP。我们还将这两个模型与一些最新的无监督分割方法进行了比较，包括无监督深度学习语义分割方法 GrowpSP (Z.Zhang et al., 2023) 和传统的无监督实例分割算法 SS (Miao et al., 2023)。 2021b) 和 DFSP (Wang et al., 2023)，它们是专门为玉米茎叶分割而设计的。

Table 3 presents a comparison of the accuracy achieved in the maize stem-leaf semantic segmentation tasks between the PointNet++ model trained in this study and the GrowSP method. As a self-supervised deeplearning method, GrowSP saves time on data annotation but still requires time for model training. The accuracy of GrowSP slightly exceeded that of PointNet in indoor semantic segmentation ntasks; however, its accuracy was significantly lower than that of PointNet ++ in maize stem-leaf semantic segmentation tasks. It is evident from Fig. 8 that the segmentation results of GrowSP differed significantly from the ground truth, indicating that its self-learned features offer little aid in recognising plant organ semantics. The application pattern of unsupervised deep learning methods in plant point-cloud processing tasks remains to be explored.
表 3 比较了本研究中训练的 PointNet++ 模型与 GrowSP 方法在玉米茎叶语义分割任务中所实现的精度。作为一种自监督深度学习方法，GrowSP 节省了数据标注时间，但仍然需要时间进行模型训练。 GrowSP在室内语义分割ntasks中的准确率略高于PointNet；但在玉米茎叶语义分割任务中其准确率明显低于PointNet++。从图8中可以明显看出，GrowSP的分割结果与地面真实值存在显着差异，表明其自学习特征对识别植物器官语义几乎没有帮助。无监督深度学习方法在植物点云处理任务中的应用模式还有待探索。

Table 4 shows the accuracy comparison of the maize stem-leaf instance segmentation tasks between the HAIS model trained in this study and the SS and DFSP methods. SS and DFSP do not require learning features from data but incorporate extensive prior botanical knowledge in their algorithms, such as the topological connections between stems and leaves and the geometric features of organs. The segmentation performance of both methods was inferior to that of the the HAIS model (Fig. 9). Although DFSP and SS save time on learning from data, tuning parameters is a time-consuming task, and more importantly, their efficiency, accuracy, and stability in point-cloud segmentation could not compare with those of the HAIS model.
表4显示了本研究训练的HAIS模型与SS和DFSP方法之间的玉米茎叶实例分割任务的准确性比较。 SS 和 DFSP 不需要从数据中学习特征，而是在算法中融入广泛的先验植物学知识，例如茎和叶之间的拓扑连接以及器官的几何特征。两种方法的分割性能均低于HAIS模型（图9）。虽然DFSP和SS节省了从数据中学习的时间，但调整参数是一项耗时的任务，更重要的是，它们在点云分割方面的效率、准确性和稳定性无法与HAIS模型相比。

We used DFSP for post-processing the segmentation results of the
我们使用DFSP对分割结果进行后处理

Fig. 5. Simulation of corn plant morphologies: (A) lodging posture, (B) specially changing the inclination angle of one leaf, and (C) transforming a leaf into a broken one. Note: Within each plant, the points belonging to the same organ instance are depicted in identical colours, with different organs represented by different colours.
图5 玉米植株形态模拟：(A)倒伏姿势，(B)专门改变一片叶子的倾斜角度，(C)将一片叶子变成折断的叶子。注意：在每株植物中，属于同一器官实例的点用相同的颜色表示，不同的器官用不同的颜色表示。

Table 2 表2

Test results on PointNet ++ and HAIS.
PointNet++ 和 HAIS 上的测试结果。

Note: This study involved selecting 22 template data points from the original dataset, which were then expanded to 2200 and 22,000 data points, excluding the flipping enhancement, for training purposes. Both the semantic and instance segmentation models were trained using these enlarged datasets and evaluated against a set of 406 real-world data points. Additionally, a baseline model was trained using only the selected 22 template data points to assess the impact of various data enhancement methods.
注：本研究涉及从原始数据集中选择 22 个模板数据点，然后将其扩展到 2200 和 22,000 个数据点（不包括翻转增强）用于训练目的。语义分割模型和实例分割模型均使用这些扩大的数据集进行训练，并针对一组 406 个真实数据点进行评估。此外，仅使用选定的 22 个模板数据点来训练基线模型，以评估各种数据增强方法的影响。

Fig. 6. Visualisation of semantic segmentation results using data-augmentation methods on PointNet++: (A) ground truth, (B) flip, (C) rotation, (D) noise, (E) crop, (F) jitter, (G) elastic deformation, (H) leaf crossover, (I) LRDT, (J) our method, and (K) baseline. Note: The red and blud point clouds represent the stem and leaves, respectively.
图 6. 在 PointNet++ 上使用数据增强方法的语义分割结果可视化：(A) 地面实况，(B) 翻转，(C) 旋转，(D) 噪声，(E) 裁剪，(F) 抖动，(G) ）弹性变形，（H）叶交叉，（I）LRDT，（J）我们的方法，以及（K）基线。注：红色点云和蓝色点云分别代表茎和叶。

HAIS model to address the under-segmentation of leaves, which improved the leaf mAP by

. It is clear from Fig. 9 that the undersegmentation problems of the top and bottom leaves were essentially resolved. Thus, incorporating DFSP post-processing significantly enhances the segmentation capability of the HAIS model.
HAIS模型解决了叶子分割不足的问题，将叶子mAP提高了

。从图9可以清楚地看出，顶部和底部叶子的欠分割问题得到了根本解决。因此，结合DFSP后处理显着增强了HAIS模型的分割能力。

5. Discussion 5. 讨论

5.1. Plant organ segmentation dataset
5.1.植物器官分割数据集

It has been evident for a long time that the construction of a copiously labelled 3D dataset would facilitate the development of powerful 3D crop models for point-cloud feature segmentation. Consequently, the demand for such datasets containing organ-level labels has increased rapidly in recent years. Nevertheless, the efforts to produce such datasets have been quite forbidding owing to their complex structures and the
长期以来，显而易见的是，构建大量标记的 3D 数据集将有助于开发用于点云特征分割的强大 3D 裁剪模型。因此，近年来对此类包含器官级别标签的数据集的需求迅速增加。然而，由于其复杂的结构和数据，生成此类数据集的努力非常困难。

Fig. 7. Visualisation of instance segmentation results using data-augmentation methods on HAIS: (A) ground truth, (B) flip, (C) rotation, (D) noise, (E) crop, (F) jitter, (G) elastic deformation, (H) leaf crossover, (I) LRDT, (J) our method, and (K) baseline. Note: The different colours in the figure represent different instances, and black indicates unsegmented point clouds.
图 7. 在 HAIS 上使用数据增强方法的实例分割结果可视化：(A) 地面实况，(B) 翻转，(C) 旋转，(D) 噪声，(E) 裁剪，(F) 抖动，(G) ）弹性变形，（H）叶交叉，（I）LRDT，（J）我们的方法，以及（K）基线。注：图中不同颜色代表不同实例，黑色表示未分割的点云。

Table 3 表3

Semantic segmentation accuracies.
语义分割准确性。

Name	mIoU (%)	Stem mIoU (%) 投票率 (%)	Leaf mIoU (%) 叶面积 (%)
GrowSP	54.64	20.06
PointNet++

substantial labour resources needed for data labelling. Nevertheless, several useful annotation point-cloud databases have been provided, such as ROSE-X(Dutagaci et al., 2020), Rice Panicles(Gong et al., 2021), Cucumbers (Boogaard et al., 2021) and Soybean(Sun et al., 2023a). To replace the intensive labour requirements with a smart physics-based data augmentation method, this study constructed a maize annotation point-cloud database beginning with 428 data points and updated it with an innovative maize stem-leaf segmentation framework based on deformable point-cloud data to allow deep-learning model training with minimal annotated data items, which, to the best of our knowledge, is the most comprehensive open-source database currently available. We expect that this database and its augmentation technique can be used in many other agricultural study areas needing type segmentation.
数据标记需要大量劳动力资源。尽管如此，已经提供了一些有用的注释点云数据库，例如ROSE-X（Dutagaci et al., 2020）、Rice Panicles（Gong et al., 2021）、Cucumbers（Boogaard et al., 2021）和Soybean（ Sun 等人，2023a)。为了用基于智能物理的数据增强方法取代密集的劳动力需求，本研究构建了一个从 428 个数据点开始的玉米注释点云数据库，并使用基于可变形点云数据的创新玉米茎叶分割框架对其进行了更新允许使用最少的注释数据项进行深度学习模型训练，据我们所知，这是目前可用的最全面的开源数据库。我们期望该数据库及其增强技术可用于许多其他需要类型分割的农业研究领域。

5.2. The diversity of our deformation method
5.2.我们变形方法的多样性

The methodology introduced in this study relies on a physics-based deformation framework to deform plant point clouds by applying external forces to organs. This approach allows for precise control over the forces, thereby enabling the setting of different application positions, directions, and magnitudes to facilitate deformation control of the
本研究中引入的方法依赖于基于物理的变形框架，通过对器官施加外力来使植物点云变形。这种方法可以精确控制力，从而能够设置不同的应用位置、方向和大小，以促进变形控制

Table 4 表4

Instance segmentation accuracy and efficiency.
实例分割的准确性和效率。

Name	mAP (%)	Stem mAP (%)	Leaf mAP (%)	Time per plant (s) 每株植物的时间（秒）
SS	33.04	13.65	52.43	143.4
DFSP	56.61	39.96	73.25	3.2
HAIS
HAIS_PP

Note: HAIS_PP represents the result of optimizing the segmentation of the HAIS model using the post-processing method described in 3.2.3.
注：HAIS_PP表示使用3.2.3中描述的后处理方法对HAIS模型的分割进行优化的结果。

Fig. 8. Visualisation of semantic segmentation results using different methods: (A) ground truth, (B) segmentation results of GrowSP, and (C) segmentation results of PointNet++. Note: The red and blude point clouds represent the stem and leaves, respectively.
图 8. 使用不同方法的语义分割结果的可视化：（A）groundtruth，（B）GrowSP 的分割结果，（C）PointNet++ 的分割结果。注：红色和蓝色点云分别代表茎和叶。

Table 5 表5

Changes to plant point clouds with different augmentation methods.
使用不同的增强方法更改植物点云。

Method

Leaf

Stem

Number of point 点数

Coordinates

azimuth

Inclination

angle

Position

on the stem

bending

size

Inclination

blending

distort

flip

rotation

noise

crop

jitter

elastic

deformation

leaf crossover 叶交叉

LRBT

Ours

Fig. 9. Visualisation of instance segmentation results using different methods: (A) ground truth, (B) segmentation results of SS, (C) segmentation results of DFSP, (D) segmentation results of HAIS, and (E) segmentation results of HAIS_PP. Note: Different colours in the figure represent different instances, and black indicates unsegmented point clouds.
图 9. 使用不同方法的实例分割结果的可视化：（A）groundtruth，（B）SS 的分割结果，（C）DFSP 的分割结果，（D）HAIS 的分割结果，（E）HAIS 的分割结果HAIS_PP。注：图中不同颜色代表不同实例，黑色表示未分割的点云。

organs. In addition, through internal force transmission, applying force to an organ result in the overall deformation of the plant. Our deformation strategy can effect changes in the plant morphology (such as overall bending and tilting), relative positions of organs (such as changes in the leaf inclination and azimuth angles), and the shapes of the organs themselves (such as bending of the stems and leaves), thereby significantly increasing the data diversity and enhancing the generalisation ability of segmentation models.
器官。另外，通过内力传递，对器官施力，导致植物整体变形。我们的变形策略可以影响植物形态的变化（例如整体弯曲和倾斜）、器官相对位置的变化（例如叶子倾斜度和方位角的变化）以及器官本身的形状（例如茎的弯曲）和叶子），从而显着增加数据多样性并增强分割模型的泛化能力。

The experimental results in Table 2 show that all augmentation methods improved the model accuracy in the stem-leaf semantic segmentation tasks, but the organ-level enhancement methods outperformed the plant-level methods, with the stem mIOU averaging

higher. The visualisation results demonstrate that the organ-level enhancement models yielded better segmentation at the stem-leaf junction than the plant-level methods. This is likely because organ-level methods enhance the feature diversity of the point cloud at the leaf-stem junction through leaf substitution, rigid body transformations, or elastic deformation. Plant-level methods are weaker in this aspect as they enhance stem diversity through changes in the characteristics of the stem (e.g., tilt and twist). The PointNet ++ model trained with our deformation method achieved the best segmentation accuracy, possibly because our method not only changes the stem characteristics but also enhances the diversity of the point cloud features at the leaf-stem junction.
表2的实验结果表明，所有增强方法都提高了茎叶语义分割任务中的模型精度，但器官级增强方法优于植物级方法，茎mIOU平均

更高。可视化结果表明，器官级增强模型在茎叶交界处比植物级方法产生更好的分割。这可能是因为器官级方法通过叶子替换、刚体变换或弹性变形增强了叶-茎交界处点云的特征多样性。植物水平的方法在这方面较弱，因为它们通过改变茎的特征（例如倾斜和扭曲）来增强茎的多样性。用我们的变形方法训练的PointNet ++模型取得了最好的分割精度，可能是因为我们的方法不仅改变了茎特征，而且增强了叶茎交界处点云特征的多样性。

All enhancement methods significantly improved the model accuracy in the stem-leaf instance segmentation tasks, with our method yielding the best results. In particular, the models trained on 2200 data points outperformed those expanded to 22,000 data points with other methods. The model trained with our method exhibited a significant advantage in the leaf mAP (

, which was

higher than the next-highest leaf mAP (leaf crossover). The results indicate that increasing the distribution of leaf azimuth angles significantly boosts the leaf mAP. The plant directions in our dataset are random, causing the leaf azimuth angles at different leaf positions to be scattered in different directions in 3D space. The main operation of the rotation enhancement method is random angle rotation around the Z-axis, which achieved a leaf mAP of

, indicating that enhancing the diversity of the leaf azimuth angle distribution can improve the rotation invariance of the instance segmentation model, thereby significantly increasing the leaf segmentation accuracy. Noise and crop enhancements did not improve the distribution of the leaf azimuth angles. Flipping, which performs a symmetrical transformation of each leaf instance relative to the X/Y axis, had a limited effect on the azimuth distribution, and hence, a lower leaf mAP. Leaf crossover and LRDF can also increase the diversity of leaf azimuth angle distribution through organ substitution or rigid body transformations, and could improve the leaf segmentation accuracy in large datasets. Jitter, elastic deformation, and our method can increase the diversity of leaf azimuth angles through deformation, but the former two cannot precisely control the deformation of each leaf, and their modifications to azimuth angles are minor. Hence, their diversity was
所有增强方法都显着提高了茎叶实例分割任务中的模型准确性，其中我们的方法产生了最佳结果。特别是，在 2200 个数据点上训练的模型优于使用其他方法扩展到 22,000 个数据点的模型。用我们的方法训练的模型在叶子 mAP (

) 上表现出显着的优势，它比第二高的叶子 mAP (叶子交叉) 更高。结果表明，增加叶子方位角的分布显着提高了叶子mAP。我们的数据集中的植物方向是随机的，导致不同叶子位置的叶子方位角在3D空间中分散在不同的方向上。旋转增强方法的主要操作是随机的。绕Z轴的角度旋转，取得了

的叶子mAP，表明增强叶子方位角分布的多样性可以提高实例分割模型的旋转不变性，从而显着提高叶子分割率噪声和作物增强并没有改善叶子方位角的分布，翻转对每个叶子实例相对于 X/Y 轴进行对称变换，对方位角分布的影响有限，因此较低。叶地图。叶子交叉和LRDF还可以通过器官替换或刚体变换来增加叶子方位角分布的多样性，并且可以提高大型数据集中的叶子分割精度。抖动、弹性变形和我们的方法可以通过变形增加叶片方位角的多样性，但前两者不能精确控制每个叶片的变形，并且对方位角的修改较小。因此，他们的多样性是
still inferior to that of the rotation enhancement. Our method can easily change the azimuth angle of leaves by applying random direction and magnitude forces to each leaf, thereby increasing the overall azimuth angle distribution of the training dataset. In addition, the deformation of leaves in our method can change the leaf inclination angles, leaf curvature, and leaf size. External forces applied to stems can cause stem elongation or contraction, indirectly changing the attachment position of the leaf on the stem. These changes significantly increased the diversity of leaf instances in the training data, which is a fundamental reason for the higher leaf mAP of our method compared to other methods.
仍然不如旋转增强。我们的方法可以通过向每个叶子施加随机方向和大小的力来轻松改变叶子的方位角，从而增加训练数据集的整体方位角分布。此外，我们方法中叶片的变形可以改变叶片倾斜角度、叶片曲率和叶片尺寸。施加在茎上的外力会导致茎伸长或收缩，间接改变叶子在茎上的附着位置。这些变化显着增加了训练数据中叶子实例的多样性，这是我们的方法与其他方法相比具有更高叶子 mAP 的根本原因。

The experiments showed that the instance segmentation models trained with 22 data points could also segment the main body of the stem point clouds. Similar to semantic segmentation models, various enhancement methods primarily improve the segmentation accuracy at the stem-leaf junction. The optimal stem mAP of our deformation method was higher than that of the other enhancement methods, possibly also because the deformation of stems and leaves enhanced the point cloud feature diversity at the leaf-stem junction.
实验表明，用22个数据点训练的实例分割模型也可以分割出茎点云的主体。与语义分割模型类似，各种增强方法主要提高茎叶连接处的分割精度。我们的变形方法的最佳茎mAP高于其他增强方法，可能也是因为茎和叶的变形增强了叶茎交界处的点云特征多样性。

The changes in the plant point-cloud data with different enhancement methods are summarized in Table 5. It can be observed that our deformation method encompasses all other data change modes apart from not changing the number of point clouds, which is an important reason for the enhanced data diversity in our method. Jitter and elastic deformation are also deformation-based enhancement methods that can achieve data changes similar to those of our method, but the accuracy of the models trained with them was far lower than those with our method. There are two main reasons for this. First, our method allows for far greater control and precision in deformation than the other two. The physics-based approach enables fine control over the application of external forces in terms of the position, direction, magnitude, and areas of the point cloud that remain undeformed. As the deformation process is physics based, users can easily predict the effects of external forces on the plant point clouds and design deformation patterns in a targeted manner, achieving a high level of each type of change in the plant point clouds with our method. Although the other two deformation methods are capable of many types of data changes, they cannot ensure the level of each type owing to their lower controllability. Second, our method can preserve the local morphological characteristics of organs during deformation, while the other two methods may cause the organ point clouds to flatten (as shown in Fig. 10), thereby reducing the geometric feature differences between different instances. Therefore, the deformation data from jitter and elastic deformation were not as good as those of our method in terms of diversity and effectiveness.
表5总结了不同增强方法下植物点云数据的变化。可以看出，我们的变形方法除了不改变点云数量外，还涵盖了所有其他数据变化模式，这是造成这种情况的重要原因。增强了我们方法中的数据多样性。抖动和弹性变形也是基于变形的增强方法，可以实现与我们的方法类似的数据变化，但用它们训练的模型的准确性远远低于我们的方法。这有两个主要原因。首先，我们的方法比其他两种方法可以更好地控制和精确变形。基于物理的方法可以根据点云的位置、方向、大小和保持不变形的区域来精细控制外力的应用。由于变形过程是基于物理的，用户可以轻松预测外力对植物点云的影响，并有针对性地设计变形模式，用我们的方法实现植物点云中每种类型变化的高水平。另外两种变形方法虽然能够进行多种类型的数据变化，但由于可控性较低，无法保证每种类型的水平。其次，我们的方法可以在变形过程中保留器官的局部形态特征，而其他两种方法可能会导致器官点云变平（如图10所示），从而减少不同实例之间的几何特征差异。因此，抖动和弹性变形的变形数据在多样性和有效性方面不如我们的方法。

The PointNet++ and HAIS models could achieve excellent results with only 2200 training data using our enhancement method. As the training data increased to 22000 , the models trained with our enhancement method still exhibited significant improvements in all segmentation metrics, whereas the improvements with other methods were limited or even negative. This also indicates the high diversity of our deformation data and its effectiveness for maize stem-leaf segmentation tasks to a certain extent.
使用我们的增强方法，PointNet++ 和 HAIS 模型只需 2200 个训练数据即可取得优异的结果。当训练数据增加到 22000 时，使用我们的增强方法训练的模型在所有分割指标上仍然表现出显着的改进，而使用其他方法的改进是有限的甚至是负面的。这也表明我们的变形数据的高度多样性及其在一定程度上对玉米茎叶分割任务的有效性。

5.3. Stem-leaf segmentation
5.3.茎叶分割

In recent years, numerous unsupervised and supervised deep learning approaches have been developed for maize stem-leaf segmentation tasks to facilitate organ-level 3D phenotypic trait measurements. Unsupervised methods integrate the botanical characteristics of maize plants as prior knowledge into algorithms, which eliminates the need for data annotation and model training. However, the segmentation results are highly sensitive to the parameters and the models are often not efficient, making them unsuitable for high-throughput 3D phenotyping tasks of maize. In addition, although supervised methods achieve higher segmentation efficiency and stability, they require extensive annotation of

data.
近年来，针对玉米茎叶分割任务开发了许多无监督和监督深度学习方法，以促进器官水平的 3D 表型性状测量。无监督方法将玉米植物的植物学特征作为先验知识集成到算法中，从而消除了数据注释和模型训练的需要。然而，分割结果对参数高度敏感，模型往往效率不高，不适合玉米的高通量3D表型分析任务。此外，尽管有监督方法实现了更高的分割效率和稳定性，但它们需要对

数据进行大量注释。

When training deep-learning models for plant point clouds, researchers typically use more labelled data for model training to achieve better generalisation and segmentation accuracy, and the models are tested with considerably less data. With the aim of reducing the data annotation workload, this study adopted a model training approach with fewer data and tested the model with nearly 20 times the amount of data. Our work demonstrates the potential of using a small amount of labelled data to construct supervised deep-learning models for plant organ segmentation. The results indicate that, for relatively simple tasks such as the semantic segmentation of maize stems and leaves at the elongation stage, constructing a PointNet++ model trained with 22 labelled data points (without data augmentation) could segment the main stem point clouds. For more complex instance segmentation tasks, the introduction of suitable augmentation algorithms to expand the data allowed the trained HAIS model to segment most organ instances correctly for phenotypic parameter extraction. This reminds us that if the goal is to extract phenotypic traits from large-scale plant point clouds more quickly and stably, rather than pursuing perfect point-cloud segmentation accuracy, extensive data annotation may not be required to train a usable segmentation model rapidly.
在训练植物点云深度学习模型时，研究人员通常使用更多标记数据进行模型训练，以实现更好的泛化和分割精度，并且使用相当少的数据来测试模型。为了减少数据标注工作量，本研究采用了较少数据的模型训练方式，并用近20倍的数据量对模型进行了测试。我们的工作展示了使用少量标记数据构建用于植物器官分割的监督深度学习模型的潜力。结果表明，对于相对简单的任务，例如玉米茎叶伸长阶段的语义分割，构建用22个标记数据点训练的PointNet++模型（没有数据增强）可以分割主茎点云。对于更复杂的实例分割任务，引入合适的增强算法来扩展数据，使经过训练的 HAIS 模型能够正确分割大多数器官实例，以进行表型参数提取。这提醒我们，如果目标是更快、更稳定地从大规模植物点云中提取表型性状，而不是追求完美的点云分割精度，那么可能不需要大量的数据注释来快速训练可用的分割模型。

Increasing the quantity of labelled data is a common approach for segmentation models if higher segmentation accuracy for plant point clouds is required. However, the proposed segmentation framework that combines the HAIS model with DFSP also demonstrates another workflow for improving the segmentation accuracy; that is, optimizing the results of deep-learning models with established unsupervised segmentation methods. This is a relatively old-fashioned strategy that does not appear as appealing as an end-to-end strategy. However, we believe it is a very suitable workflow for plant point-cloud processing. As the morphology of the same species of plants at the same growth stage is very similar across different individuals, the segmentation results of deep-learning models are relatively stable and the patterns of missegmentation are also relatively fixed (such as the under-segmentation of the top and bottom leaves observed in this study). These missegmented point clouds, with fixed patterns and outcomes, have a significantly reduced quantity compared to the entire plant point cloud, thereby allowing unsupervised segmentation methods to segment them quickly under fixed algorithm parameters.
如果需要更高的植物点云分割精度，则增加标记数据的数量是分割模型的常用方法。然而，所提出的将 HAIS 模型与 DFSP 相结合的分割框架还展示了另一种提高分割精度的工作流程；也就是说，利用已建立的无监督分割方法来优化深度学习模型的结果。这是一种相对老式的策略，看起来不如端到端策略那么有吸引力。然而，我们相信这是一个非常适合植物点云处理的工作流程。由于同种植物在同一生长阶段的形态在不同个体之间非常相似，因此深度学习模型的分割结果相对稳定，误分割的模式也相对固定（例如植物的欠分割）。本研究中观察到的顶部和底部叶子）。这些错误分割的点云具有固定的模式和结果，与整个植物点云相比，其数量显着减少，从而允许无监督分割方法在固定的算法参数下快速分割它们。

Using the workflow in this study, we could annotate a small amount of point-cloud data, expand the data through physical deformation, and train an instance segmentation model with certain generalisation capabilities (the HAIS model was trained in this study). Subsequently, we
利用本研究的工作流程，我们可以对少量的点云数据进行注释，通过物理变形扩展数据，并训练具有一定泛化能力的实例分割模型（本研究训练的是HAIS模型）。随后，我们

Fig. 10. Deformation effects on organs by different methods: (A) original point cloud, (B) deformation results using jitter, (C) deformation results using elastic deformation, and (D) deformation results using our method.
图10.不同方法对器官的变形影响：（A）原始点云，（B）使用抖动的变形结果，（C）使用弹性变形的变形结果，（D）使用我们的方法的变形结果。
used this model for quick, stable segmentation of plant point clouds, and finally, we applied a specific unsupervised instance segmentation method (DFSP) for rapid optimization of mis-segmented point clouds with fixed patterns (the under-segmented bottom and top point clouds in this study). This workflow reduces the demand for labelled data by deeplearning models, highlighting their efficient and stable segmentation advantages. In addition, it mitigates the sensitivity to parameters and inefficiency of unsupervised methods, emphasizing their strong capability to address specific issues. We believe that this workflow will enhance the efficiency of using point cloud data for 3D phenotypic analysis.
使用该模型对植物点云进行快速、稳定的分割，最后，我们应用特定的无监督实例分割方法（DFSP）来快速优化具有固定模式的误分割点云（分割不足的底部和顶部点云）这项研究）。该工作流程减少了深度学习模型对标记数据的需求，凸显了其高效稳定的分割优势。此外，它还减轻了无监督方法对参数的敏感性和低效率，强调了它们解决特定问题的强大能力。我们相信该工作流程将提高使用点云数据进行 3D 表型分析的效率。

5.4. The scalability of our deformation method
5.4.我们的变形方法的可扩展性

Although this study focused on applying the physical deformation framework to the stem-leaf segmentation task of maize plants, its potential for broader application is significant. It is possible to forecast future maize population point clouds by employing virtual plant deformations at specific intervals and/or through random rotations and translations. This predictive capability lays the groundwork for developing large-scale population segmentation models that are informed by species evolution. We showcased this potential by synthesising two distinct population densities at both the seedling and elongation stages of maize, using three template datasets for each stage, as illustrated in Fig. 11(A) and (B) (for the seedling stage) and 11(C) and (D) (for the elongation stage). Thus, we exemplified the capacity of our deformation method to create diverse and realistic maize population models, which are essential for deciphering growth patterns and responses to environmental changes. Our methodology offers a comprehensive framework for the segmentation and analysis of plant structures across various growth stages and densities, thereby playing a crucial role in advancing plant phenotyping measurement technologies.
尽管这项研究的重点是将物理变形框架应用于玉米植物的茎叶分割任务，但其更广泛应用的潜力是巨大的。通过以特定间隔使用虚拟植物变形和/或通过随机旋转和平移，可以预测未来的玉米种群点云。这种预测能力为开发基于物种进化的大规模种群分割模型奠定了基础。我们通过综合玉米幼苗和伸长阶段的两种不同种群密度来展示这一潜力，每个阶段使用三个模板数据集，如图 11(A) 和 (B)（幼苗阶段）和 11( C) 和 (D)（用于伸长阶段）。因此，我们举例说明了变形方法创建多样化且现实的玉米种群模型的能力，这对于破译生长模式和对环境变化的响应至关重要。我们的方法为跨不同生长阶段和密度的植物结构的分割和分析提供了一个全面的框架，从而在推进植物表型测量技术方面发挥着至关重要的作用。

Moreover, the application of our approach is not limited to maize, but can also be extended to the morphological transformation of other species, including to tomatoes(Schunck et al., 2021), wheat, and Mimosa(Huang et al., 2013). The specific deformation processes tailored to these plants and the ensuing outcomes are detailed in Fig. 12. Our method, which is adept at producing realistic and varied plant models, advances plant segmentation technology and serves as a valuable resource for researchers and practitioners in plant science. This encompasses the study and measurement of plant phenotypes, thereby highlighting the adaptability and efficacy of our physics-based deformation method in contributing to the field of plant science. These applications further underscore the versatility and practicality of our deformation approach.
此外，我们的方法的应用不仅限于玉米，还可以扩展到其他物种的形态转化，包括番茄（Schunck et al., 2021）、小麦和含羞草（Huang et al., 2013）。针对这些植物量身定制的具体变形过程以及随后的结果如图 12 所示。我们的方法擅长生成逼真且多样化的植物模型，推进了植物分割技术，并为植物科学研究人员和从业者提供了宝贵的资源。这包括植物表型的研究和测量，从而突出了我们基于物理的变形方法在植物科学领域做出贡献的适应性和有效性。这些应用进一步强调了我们的变形方法的多功能性和实用性。

5.5. Limitations 5.5.局限性

The HAIS model was shown to have limitations, especially with clustering tasks that rely excessively on centroid offset predictions. In instances of dense or partially loose connections, this may lead to poor segmentation or the formation of unrealistic objects, thereby reducing
HAIS 模型被证明存在局限性，特别是对于过度依赖质心偏移预测的聚类任务。在连接密集或部分松散的情况下，这可能会导致分割不良或形成不切实际的对象，从而减少

Fig. 11. Simulated point-cloud distributions for maize populations using physics-based deformation: (A) seedling stage population point-cloud simulation with

plant spacing and

row spacing, (B) seedling stage population point-cloud simulation with

plant spacing and

row spacing, (C) elongation stage population point cloud simulation with

plant spacing and

row spacing, and (D) elongation stage population point-cloud simulation with

plant spacing and

row spacing.
图 11. 使用基于物理的变形模拟玉米种群的点云分布：(A) 苗期种群点云模拟，采用

株距和

行距，( B) 幼苗期种群点云模拟，采用

株距和

行距，(C)伸长期种群点云模拟，采用

株距和

行距，以及 (D) 使用

株距和

行距进行伸长阶段种群点云模拟。

Fig. 12. Application of data augmentation methods based on physical deformation on various plants: (A) top to bottom: annotated instance point clouds of tomato, wheat, and Mimosa pudica, (B) voxelisation of the original point clouds, (C) instance labels of the volumetric mesh, (D) morphological changes in the plants following the application of external forces, and (E) reconstruction of point cloud data through interpolation from the voxels.
图 12. 基于物理变形的数据增强方法在各种植物上的应用：(A) 从上到下：番茄、小麦和含羞草的带注释实例点云，(B) 原始点云的体素化，(C)体积网格的实例标签，(D) 施加外力后植物的形态变化，以及 (E) 通过体素插值重建点云数据。

prediction quality. Meanwhile, the efficiency of the deformation method in generating data in this article still needs to be improved. At present, we have adopted a two-stage deformation strategy (changing stems first and then leaves). In the future, we will optimize the deformation strategy and merge it into one stage to reduce deformation time. Meanwhile, currently we have only selected the point cloud at the last moment of the physical deformation process as the deformation data, which results in only one data being generated for each deformation operation; In the future, we will increase the time points for saving data during each deformation operation, so that multiple data can be generated for each deformation operation, thereby improving the efficiency of data generation. Although the method based on physical deformation in this study requires more time to simulate plant movements, the improvement in the overall leaf segmentation accuracy is significant in a way that far outweighs the new inconvenience. However, our method does not solve the problem of point-cloud quality differences, such as when complete portions are missing. Therefore, to achieve better segmentation accuracy, it is necessary to generate data by deforming point clouds of different qualities and using those of multiple qualities to fill-in the blanks.
预测质量。同时，本文变形方法生成数据的效率仍有待提高。目前我们采用的是两阶段变形策略（先换茎，后换叶）。未来我们会优化变形策略，合并为一个阶段，减少变形时间。同时，目前我们只选择了物理变形过程最后时刻的点云作为变形数据，导致每次变形操作只生成一个数据；未来我们将增加每次变形操作时保存数据的时间点，使得每次变形操作都可以生成多个数据，从而提高数据生成的效率。尽管本研究中基于物理变形的方法需要更多的时间来模拟植物运动，但整体叶子分割精度的改进是显着的，远远超过了新的不便。然而，我们的方法并没有解决点云质量差异的问题，例如当完整部分丢失时。因此，为了获得更好的分割精度，需要通过对不同质量的点云进行变形并利用多种质量的点云来填充空白来生成数据。

6. Conclusion 六，结论

This study developed a maize stem and leaf segmentation framework based on deformable point cloud data that allows the training of deeplearning models with fewer labelled data, achieving precise semantic and instance segmentations. We created a new maize-plant point-cloud database containing 428 annotated data points, allowing for sufficient data augmentation geared towards the valid and effective application of deep-learning models. We also developed a physics-based point-cloud deformation method that utilizes highly controllable deformations to significantly improve the morphological diversity of the training set while preserving the local geometric features of organs. Through a series of comparative experiments, our deformation data strategy was validated. Additionally, two labelled datasets were built from the plant data for every leaf-number sample to produce a wide variety of deformed point clouds for more thoroughly training the PointNet ++ segment and HAIS instance segmentation models. These models were then tested on 406 real data points, where the PointNet++ model secured a

mIoU in semantic segmentation, and the HAIS model obtained an 89.57

in instance segmentation. After post-processing, an instance segmentation result of

was achieved. The results clearly indicate that our methodology enables the efficient training of organ segmentation models, necessitating minimal labelled data within a shorter duration. Furthermore, it presents a robust tool for parsing point clouds, which is advantageous for maize phenotyping.
本研究开发了一种基于可变形点云数据的玉米茎叶分割框架，允许用更少的标记数据训练深度学习模型，实现精确的语义和实例分割。我们创建了一个新的玉米植物点云数据库，其中包含 428 个带注释的数据点，可以进行足够的数据增强，以实现深度学习模型的有效和有效应用。我们还开发了一种基于物理的点云变形方法，利用高度可控的变形来显着提高训练集的形态多样性，同时保留器官的局部几何特征。通过一系列的对比实验，我们的变形数据策略得到了验证。此外，根据每个叶数样本的植物数据构建了两个标记数据集，以生成各种变形点云，以便更彻底地训练 PointNet ++ 分段和 HAIS 实例分割模型。然后，这些模型在 406 个真实数据点上进行了测试，其中 PointNet++ 模型在语义分割中获得了

mIoU，HAIS 模型在实例分割中获得了 89.57

。经过后处理，得到

的实例分割结果。结果清楚地表明，我们的方法能够有效训练器官分割模型，需要在更短的时间内使用最少的标记数据。此外，它提供了一个用于解析点云的强大工具，这对于玉米表型分析是有利的。

CRediT authorship contribution statement
CRediT 作者贡献声明

Xin Yang: Data curation, Methodology, Software, Validation, Visualization, Writing - original draft. Teng Miao: Methodology, Project administration, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing, Resources, Conceptualization, Data curation, Funding acquisition, Investigation. Xueying Tian: Data curation, Investigation. Dabao Wang: Data curation, Investigation. Jianxiang Zhao: Data curation, Investigation. Lili Lin: Data curation, Investigation. Chao Zhu: Data curation, Investigation. Tao Yang: Data curation, Funding acquisition, Investigation. Tongyu Xu: Funding acquisition, Project administration, Supervision.
Xin Yang：数据管理、方法论、软件、验证、可视化、写作 - 初稿。腾苗：方法论、项目管理、软件、监督、验证、可视化、写作 - 初稿、写作 - 审查和编辑、资源、概念化、数据管理、资金获取、调查。田雪英：数据整理、调查。王大宝：数据整理、调查。赵建祥：数据整理、调查。林丽丽：数据整理、调查。朱超：数据整理、调查。杨涛：数据整理、资金获取、调查。徐同宇：资金获取、项目管理、监督。

Declaration of competing interest
竞争利益声明

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
作者声明，他们没有已知的可能影响本文报告工作的相互竞争的经济利益或个人关系。

Acknowledgments 致谢

This work was supported by Joint Funds from the National key research and development program[2022YFD2002303-01] and the
该工作得到国家重点研发计划联合基金项目[2022YFD2002303-01]和

Appendix 附录

Implementation of enhanced algorithms
增强算法的实现

The comparative augmentation methods included flipping, rotation, noise, cropping, jittering, elastic deformation, leaf crossover, and leaf rigid body transformation (LRDT). The first six methods do not consider the instance label information of the point cloud and apply enhancement algorithms to the plant point cloud directly; these are referred to as plant-level enhancement algorithms in this study. The final two methods and our method necessitate the use of label information to apply enhancement algorithms to organ point clouds, and we refer to these as organ-level enhancement algorithms. We compared the degree of alteration to plant data using our method with these point-cloud enhancement techniques (Figure A1).
比较增强方法包括翻转、旋转、噪声、裁剪、抖动、弹性变形、叶子交叉和叶子刚体变换（LRDT）。前六种方法没有考虑点云的实例标签信息，直接对植物点云应用增强算法；这些在本研究中被称为工厂级增强算法。最后两种方法和我们的方法需要使用标签信息将增强算法应用于器官点云，我们将这些称为器官级增强算法。我们将使用我们的方法与这些点云增强技术对植物数据的改变程度进行了比较（图 A1）。

The implementation details of these eight enhancement algorithms are are as follows:
这八种增强算法的实现细节如下：

(1) Flipping enhancement.
(1)翻转增强。

When enhancing the data through flipping, the plant point cloud was flipped along the X- and Y-axes. This procedure generated a total of 66 augmented training datasets.
通过翻转增强数据时，植物点云沿 X 轴和 Y 轴翻转。此过程总共生成了 66 个增强训练数据集。

(2) Rotation enhancement.
(2)旋转增强。

Prior to training the stem-leaf segmentation network using VCNN, Jin et al.(2020) implemented a data augmentation strategy that involved rotating the data. Following their methodology, we applied random rotations to maize point clouds: 0 to 360 degrees along the Z-axis, and 0 to 30 degrees along the X- and Y-axes.
在使用 VCNN 训练茎叶分割网络之前，Jin 等人（2020）实施了一种涉及旋转数据的数据增强策略。按照他们的方法，我们对玉米点云应用随机旋转：沿 Z 轴 0 到 360 度，沿 X 和 Y 轴 0 到 30 度。

(3) Noise and crop.
(3)噪声和作物。

We adopted the noise and crop augmentation techniques with optimal parameters, as applied by Xin et al. (2023) in their segmentation enhancement experiments on tomato plants. These optimal parameters were used to enhance the segmentation performance in our experiments.
我们采用了 Xin 等人应用的具有最佳参数的噪声和作物增强技术。（2023）在他们对番茄植物的分割增强实验中。这些最佳参数用于增强我们实验中的分割性能。

(4) Jitter and elastic deformation
(4)抖动和弹性变形

The HAIS model (Chen et al., 2021) for instance segmentation offers jitter and elastic deformation techniques. We applied these methods with default parameters to maize plants to evaluate their impact on the model performance.
实例分割的 HAIS 模型（Chen et al., 2021）提供了抖动和弹性变形技术。我们将这些具有默认参数的方法应用于玉米植株，以评估它们对模型性能的影响。

(5) Leaf crossover. (5)叶交叉。

Ghiasi et al. (2021) demonstrated that copy-pasting operations could significantly improve the performance of image instance segmentation tasks. Building on this, Xin et al. (2023) developed a 'leaf crossover' method for enhancing tomato plant point clouds, which achieved favourable results in PointNet++ semantic segmentation tasks. Inspired by their work, we developed a 'maize version' of the leaf crossover augmentation algorithm. This strategy enhances the diversity and structural complexity of the sample set by deleting, replacing, and adding leaf organ instances in maize plants. It involves randomly removing certain leaf instances from the labelled data, selecting appropriate leaf instances from different labelled samples for replacement, and randomly introducing new organ instances to increase the overall organ variability within the dataset.
吉亚西等人。（2021）证明复制粘贴操作可以显着提高图像实例分割任务的性能。在此基础上，Xin 等人。 (2023)开发了一种增强番茄植株点云的“叶子交叉”方法，在PointNet++语义分割任务中取得了良好的结果。受他们工作的启发，我们开发了叶子交叉增强算法的“玉米版本”。该策略通过删除、替换和添加玉米植物中的叶器官实例来增强样本集的多样性和结构复杂性。它涉及从标记数据中随机删除某些叶子实例，从不同标记样本中选择适当的叶子实例进行替换，以及随机引入新的器官实例以增加数据集中的整体器官变异性。

(6) Leaf rigid body transformation
(6)叶子刚体变换

Xin et al. (2023) found that enhancing tomato plant point clouds with rigid body transformations could improve the accuracy of PointNet++ semantic segmentation. Drawing on their work, we developed a rigid body transformation enhancement method for maize point clouds. For each leaf instance, we used the closest point to the stem within the leaf as the pivot point. Subsequently, using this pivot as the rotation centre, we performed random rotations around the Z-axis within the range of [-180, 180] degrees and around the principal axis of the leaf within the range of [-50, 50] degrees. Finally, we translated the leaf instance along the Z-axis within the range of

, where

is the length of the stem and

is the number of leaves.
辛等人。 (2023) 发现通过刚体变换增强番茄植株点云可以提高 PointNet++ 语义分割的准确性。借鉴他们的工作，我们开发了一种玉米点云的刚体变换增强方法。对于每个叶子实例，我们使用叶子内最接近茎的点作为枢轴点。随后，使用该枢轴作为旋转中心，我们在[-180, 180]度范围内围绕Z轴进行随机旋转，并在[-50, 50]度范围内围绕叶子主轴进行随机旋转。最后，我们沿着 Z 轴在

范围内平移叶子实例，其中

是茎的长度，

是茎的数量。树叶。

Fig. A1. Results of various 3D point-cloud deformation methods: (A) original maize point cloud, (B) horizontal flip, (C) rotation, (D) random noise, (E) random crop, (F) jitter, (G) elastic deformation, (H) random copy-paste, (I) LRDT, and (J) our method.
图A1。各种3D点云变形方法的结果：(A)原始玉米点云，(B)水平翻转，(C)旋转，(D)随机噪声，(E)随机裁剪，(F)抖动，(G)弹性变形、(H) 随机复制粘贴、(I) LRDT 和 (J) 我们的方法。

Note: Within each plant, points belonging to the same organ instance are depicted in identical colours, with different organs represented by different colours.
注意：在每株植物中，属于同一器官实例的点用相同的颜色表示，不同的器官用不同的颜色表示。

A. 2 Model training
A.2 模型训练

We tested the above methods using a workstation equipped with a 12th Gen Intel(R) Core(TM) i9-12900 K processor, 128-GB RAM and an RTX A6000 GPU. Approximately

were required for our physics-based deformation framework to deform a plant point cloud.
我们使用配备第 12 代 Intel(R) Core(TM) i9-12900 K 处理器、128 GB RAM 和 RTX A6000 GPU 的工作站测试了上述方法。我们基于物理的变形框架大约需要

来使植物点云变形。

We trained the PointNet++ model with 100 epochs, each encompassing 198 batches with sizes of 100, and downsampled the plant point-cloud data to 4096 points specifically for this model. We adopted the default hyperparameters proposed by Qi et al.(2017) and assessed semantic segmentation precision using the mean intersection-over-union (mIoU) metric, as recommended in their research. In contrast, the HAIS model was trained on a solitary GPU configuring a batch size of 100 over

iterations. We utilised the default hyperparameters of Chen et al.(2021) and evaluated the model's performance using the mean average precision (mAP) metric per their methodology. We normalised our data and adjusted the voxel size to 0.003 before introducing it into the model and maintained all other parameters at the default HAIS configuration.
我们用 100 个时期训练了 PointNet++ 模型，每个时期包含 198 个批次，大小为 100，并专门针对该模型将植物点云数据下采样到 4096 个点。我们采用了 Qi 等人 (2017) 提出的默认超参数，并按照他们的研究中的建议，使用平均交并集 (mIoU) 指标评估语义分割精度。相比之下，HAIS 模型是在单个 GPU 上进行训练的，该 GPU 配置了批量大小为 100 的

次迭代。我们利用 Chen 等人 (2021) 的默认超参数，并根据他们的方法使用平均精度 (mAP) 指标评估模型的性能。在将数据引入模型之前，我们对数据进行归一化并将体素大小调整为 0.003，并将所有其他参数维持在默认的 HAIS 配置。

A. 3 Strategy for generating deformation data
A.3 变形数据生成策略

The label data can be expanded to any number of instances with a varying degree of deformation for augmented deep-learning training using the deformation framework. To assess the effects of deformation data on model training, a series of comparative experiments were carried out.
标签数据可以扩展到任意数量的具有不同变形程度的实例，以使用变形框架进行增强深度学习训练。为了评估变形数据对模型训练的影响，进行了一系列对比实验。

Experiment 1 (E1): 实验1（E1）：

Objective: Identify the best way to select real label data for point-cloud points of different qualities.
目标：确定为不同质量的点云点选择真实标签数据的最佳方法。

Experimental scheme: We randomly selected one stem-missing point cloud and one stem-complete point cloud selected from leaf count data. We then generated 1000 deformed data points for each real dataset. The model was trained using stem-missing deformed data ( 11000 total), stemcomplete deformed data (11 000 total) and combined deformed data ( 5500 stem-missing deformed data and 5500 stem-complete data), as shown in E1_1-E1_3 in Table A1. Finally, the accuracy of the model was validated using the remaining 406 data points.
实验方案：我们从叶子计数数据中随机选择一个茎缺失点云和一个茎完整点云。然后我们为每个真实数据集生成 1000 个变形数据点。使用茎缺失变形数据（总共11000个）、茎完整变形数据（总共11000个）和组合变形数据（5500个茎缺失变形数据和5500个茎完整数据）来训练模型，如表E1_1-E1_3所示A1。最后，使用剩余的 406 个数据点验证了模型的准确性。

Experimental results: The deformation method is effective for data of different qualities, and combining the deformed data of both qualities can improve the segmentation accuracy of the model (E1_1-E1_3 in Table A2). However, if the goal is to process missing point-cloud data by training the model with enhanced complete data, relying solely on the deformation method in this study would be insufficient and vice versa. This indicates that, although the deformation method can augment the morphological features of plant point clouds, its ability to improve data quality diversity is limited. Therefore, when selecting real label data, it is necessary to consider the quality and sources of the data. For different batches of data (e.g. data obtained from different sensors or scenes), it is best to select real label data from each batch for deformation assuming that the data quality and features are similar within the same batch. In subsequent experiments, both stem-missing and -complete point clouds were selected as ground-truth label data for plants with different leaf counts.
实验结果：变形方法对于不同质量的数据都是有效的，将两种质量的变形数据结合起来可以提高模型的分割精度（表A2中的E1_1-E1_3）。然而，如果目标是通过使用增强的完整数据训练模型来处理丢失的点云数据，则仅依靠本研究中的变形方法是不够的，反之亦然。这表明，虽然变形方法可以增强植物点云的形态特征，但其提高数据质量多样性的能力有限。因此，在选择真实标签数据时，需要考虑数据的质量和来源。对于不同批次的数据（例如从不同传感器或场景获得的数据），假设同一批次内的数据质量和特征相似，最好从每批次中选择真实的标签数据进行变形。在随后的实验中，选择茎缺失和完整的点云作为不同叶数植物的地面实况标签数据。

Table A1 表A1

Data selection strategy of Experiment 1.
实验1的数据选择策略。

Experiment

name

Number of stem-missing label data
词干缺失标签数据的数量

selected in each leaf number plant
在每个叶号植物中选择

Number of stem-complete label data
词干完整标签数据的数量

selected in each leaf number plant
在每个叶号植物中选择

Number of Deformed Data
变形数据数

Generated per Label Data
根据标签数据生成

Total Number of Deformed Data
变形数据总数

Used for Training 用于训练

E1_1

1000

11,000

E1_2

1000

11,000

E1_3

500

11,000

Table A2 表A2

Test results of Experiment 1 .
实验1的测试结果。

Experiment name 实验名称	PointNet++ /mIoU			HAIS /mAP
Experiment name 实验名称	Average	Stem	Leaf	Average	Stem	Leaf
E1_1 (test on stem-missing) E1_1（茎缺失测试）	82.68	67.72	97.64	84.70	85.95
E1_1 (test on stem-complete) E1_1（词干完整测试）	82.72	67.77	99.84	85.38	87.77	82.99
E1_1 (test on all) E1_1（全部测试）	81.95		97.56	85.21	87.01	83.42
E1_2 (test on stem-missing) E1_2（茎缺失测试）	91.48	84.17	98.80	69.70	57.84	80.31
E1_2 (test on stem-complete) E1_2（词干完整测试）	91.46	84.14	99.42	87.35	93.67	81.40
E1_2(test on all) E1_2（全部测试）	91.22	83.71	98.73	83.11	85.47	80.74
E1_3 (test on stem-missing) E1_3（茎缺失测试）	91.77	84.74	98.80	82.50	84.56	80.43
E1_3 (test on stem-complete) E1_3（词干完整测试）	91.72	84.67	98.78	87.79	94.83	81.20
E1_3(test on all) E1_3（全部测试）	91.70	84.64	98.76	86.48	92.01	80.95

Experiment 2 (E2): 实验2（E2）：

Objectives: 1) To clarify the qualitative relationship between the number of deformed data points and the model's segmentation accuracy and 2) to clarify the qualitative relationship between the degree of point-cloud deformation and the segmentation accuracy of the model.
目标：1）阐明变形数据点的数量与模型分割精度之间的定性关系；2）阐明点云变形程度与模型分割精度之间的定性关系。

Experimental scheme: Different degrees of point cloud deformation can be achieved by adjusting the parameters

, as described in Section 3.2.1. The parameters were set with the aim of ensuring both realism and diversity of the deformed point clouds. Parameters

were set to

, 5,20 , and 0 for stem deformation and

, and 15 for leaf deformation, respectively, to achieve realistically deformed plant point clouds. This force application strategy (denoted as F0) results in the stem deformation being more elongated with slight bending, while the leaf deformation reflects changes in the leaf angle, including minor changes in the leaf position and curvature, similar to the natural variation observed in real maize plants. To increase the point cloud diversity under this strategy, parameters

were doubled (denoted as F1), resulting in values of

, 40 , and 0 for stem deformation and

, and 30 for leaf deformation. All parameters

were simply set to 15 for both stems and leaves (denoted as F2) for more diverse point-cloud deformations. This strategy encompasses deformations produced by F0, thereby inducing more random deformations in the plant stems and leaves, and hence, some point clouds may not resemble normal maize plant morphology. In addition, to enhance diversity under the F2 strategy, parameters

were doubled to 30 for both the stems and leaves (denoted as F3), resulting in more
实验方案：通过调整参数

至

可以实现不同程度的点云变形，如3.2.1节所述。设置参数的目的是确保变形点云的真实性和多样性。茎变形参数

分别设置为

、 5,20 和 0 ，叶子变形参数

和 15 ，以实现逼真的变形植物点云。这种施力策略（表示为 F0）导致茎变形更加拉长并轻微弯曲，而叶子变形反映了叶子角度的变化，包括叶子位置和曲率的微小变化，类似于真实中观察到的自然变化玉米植物。为了增加该策略下的点云多样性，参数

加倍（表示为 F1），导致茎变形值为

、 40 和 0 ，以及

均简单设置为 15，以实现更多样化的点云变形。该策略包含 F0 产生的变形，从而在植物茎和叶中引起更多随机变形，因此，某些点云可能与正常的玉米植物形态不同。此外，为了增强F2策略下的多样性，茎和叶的参数

均加倍至30（记为F3），从而产生更多
'distorted' maize plant morphologies compared to the other strategies. Two real labels (one with a complete stem and one with a missing stem) were randomly selected for each leaf number, and different amounts

, and 1000) of deformed data were generated for each label for training (Table A3).
与其他策略相比，玉米植株形态“扭曲”。为每个叶号随机选择两个真实标签（一个有完整茎，一个有茎缺失），并为每个标签生成不同数量的变形数据进行训练（表A3）。

Experimental results: The results of Experiment 2 (Table A4) revealed that for the first objective, the accuracy improved with an increase in the amount of deformed data across all force application strategies, and optimal results were achieved when 1000 deformed data points were generated for each real data point. This suggests that once a method for applying external forces is established, the feature representation and model generalisability improve with more deformation data. The upper limit on the number of data points that can enhance the segmentation results was not explored further, as it would likely be specific to the dataset used in this study and not be generally applicable to other datasets.
实验结果：实验2（表A4）的结果表明，对于第一个目标，精度随着所有施力策略中变形数据量的增加而提高，并且当为1000个变形数据点生成时获得最佳结果。每个真实的数据点。这表明，一旦建立了施加外力的方法，特征表示和模型的通用性就会随着更多的变形数据而提高。没有进一步探讨可以增强分割结果的数据点数量的上限，因为它可能特定于本研究中使用的数据集，并且不适用于其他数据集。

An analysis of the impact of different force application strategies on the model accuracy indicated that the accuracy differences among F1

F3 were minimal but higher than F0 for the semantic segmentation task. This suggests that more random directions and greater magnitudes of applied force enhanced the model generalisability for this dataset. For the more complex task of instance segmentation, the F2 strategy outperformed the others (especially with 2200 data points), whereas the F3 strategy resulted in significantly poorer outcomes owing to the high degree of distortion leading to a greater proportion of ineffective data. This implies that random force directions and moderate force magnitudes can increase the diversity and quantity of 'effective' data, thereby improving the model performance. Determining the appropriate range of external forces involves ensuring data diversity while striving to ensure that the data authenticity is not too poor. However, achieving the optimal balance between diversity and 'good realism' remains a complex problem to be explored through experiments on more different datasets. In the context of this study, employing random force directions with uniformly set parameters

at a balanced magnitude of

for external forces is recommended for optimal deformation as described.
分析不同的施力策略对模型精度的影响表明，对于语义分割任务，F1

F3 之间的精度差异很小，但高于 F0。这表明更多的随机方向和更大的施加力增强了该数据集的模型通用性。对于更复杂的实例分割任务，F2 策略优于其他策略（尤其是在 2200 个数据点的情况下），而 F3 策略的结果明显较差，因为高度失真导致无效数据比例更大。这意味着随机的力方向和适度的力大小可以增加“有效”数据的多样性和数量，从而提高模型性能。确定合适的外力范围，涉及到保证数据的多样性，同时努力保证数据的真实性不会太差。然而，在多样性和“良好的现实性”之间实现最佳平衡仍然是一个复杂的问题，需要通过在更多不同的数据集上进行实验来探索。在本研究的背景下，建议采用随机力方向和均匀设置的参数

，外力的平衡大小为

，以获得所描述的最佳变形。

Table A3 表A3

Implementation plan for Experiment 3.
实验3的实施计划。

Experiment

Name

The strategy of 的策略

applying external 应用外部

forces

Number of stem-missing label
缺茎标签数量

data selected in each leaf number
每个叶子编号中选择的数据

plant

Number of stem-complete label
茎完整标签数量

data selected in each leaf number
每个叶子编号中选择的数据

plant

Number of Deformed Data
变形数据数

Generated per Label Data
根据标签数据生成

Total Number of Deformed
变形总数

Data Used for Training
用于训练的数据

E2_1

100

2200

E2_2

500

11,000

E2_3

1000

22,000

E2_4

100

2200

E2_5

100

11,000

E2_6

100

22,000

E2_7

500

2200

E2_8

1000

11,000

E2_9

100

22,000

E2_10

500

2200

E2_11

1000

11,000

E2_12

22,000

Table A4 表A4

Test results of Experiment 2.
实验2的测试结果。

Experiment Name 实验名称	PointNet++ / mIoU			HAIS / mAP
Experiment Name 实验名称	mIoU (%)	Stem mIoU (%) 投票率 (%)	Leaf mIoU (%) 叶面积 (%)	mAP (%)	Stem mAP (%)	Leaf mAP (%)
E2_1	81.40	75.80	87.00	83.43	90.99	75.87
E2_2	84.66	80.03	89.28	87.77	92.64	82.91
E2_3	87.06	83.17	90.94	88.60	92.14	85.05
E2_4	90.76	82.92	98.59	80.90	87.39	74.71
E2_5	92.06	85.29	98.82	87.82	91.82	83.82
E2_6	92.23	85.61	98.86	89.02	92.22	85.81
E2_7	90.54	82.55	98.54	85.89	91.89	79.88
E2_8	91.70	84.64	98.76	86.48	92.01	80.95
E2_9	91.93	85.04	98.82	89.57	93.32	85.83
E2_10	90.01	81.56	98.45	70.73	76.85	64.61
E2_11	91.15	83.62	98.69	82.08	83.40	80.76
E2_12	91.23	83.76	98.70	83.94	84.58	83.31

Experiment 3 (E3). 实验3（E3）。

Objective: To elucidate the qualitative relationship between the quantity of real labelled point clouds and the model's segmentation accuracy.
目的：阐明真实标记点云数量与模型分割精度之间的定性关系。

Experimental scheme: The study randomly selected varying numbers of labeled real point clouds from 5-leaf and 6-leaf plants (198 samples total) to generate deformed point clouds for training, ensuring a consistent quantity of training data as detailed in E3_1 E3_4 in Table A5. To evaluate the impact of real versus deformed point clouds on the generalisation ability of the model, the F2 force application strategy from Experiment 2 was used to deform the four real data selections in E3_1, thereby generating additional training data (E3_5-E3_7) to enhance the diversity of the deformed data. The model's accuracy was tested on the remaining 5-leaf and 6-leaf samples. It's important to note the specific focus on 5-leaf and 6-leaf plant data due to the structured comparison of different quantities of label data, with a cap at 30, as data from other leaf numbers were insufficient for comparable experiments. Nonetheless, the findings from these two leaf numbers are considered representative.
实验方案：本研究随机选取5叶和6叶植物中不同数量的带标签真实点云（共198个样本）生成变形点云进行训练，保证训练数据数量一致，详见表E3_1 E3_4 A5。为了评估真实点云与变形点云对模型泛化能力的影响，使用实验 2 中的 F2 力应用策略对 E3_1 中的四个真实数据选择进行变形，从而生成额外的训练数据 (E3_5-E3_7) 以增强变形数据的多样性。在剩余的 5 叶和 6 叶样本上测试了模型的准确性。值得注意的是，由于对不同数量的标签数据进行结构化比较，因此特别关注 5 叶和 6 叶植物数据，上限为 30，因为来自其他叶数的数据不足以进行可比实验。尽管如此，这两个叶子数量的结果被认为具有代表性。

Experimental results: The results of E3_1 E3_4 in Table A5 indicate that, with a constant total training data volume, augmenting the number of real label data enhances model accuracy, with E3_1 showing notably lower accuracy compared to E3_2 E3_4. However, beyond a certain threshold, the model's performance plateaus, as evidenced by the minimal accuracy variance among E3_2 E3_4. The outcomes of E3_5 E3_7 demonstrate that enhancing the total training data volume-without increasing the real label data count-can still improve accuracy due to the greater diversity of
实验结果：表A5中E3_1 E3_4的结果表明，在总训练数据量恒定的情况下，增加真实标签数据的数量可以提高模型精度，其中E3_1的精度明显低于E3_2 E3_4。然而，超过某个阈值后，模型的性能就会趋于稳定，E3_2 E3_4 之间的精确度差异最小就证明了这一点。 E3_5 E3_7 的结果表明，在不增加真实标签数据数量的情况下，增加总训练数据量仍然可以提高准确性，因为
deformed data. E3_7 was lower than E3_5, indicating that as the amount of data increases, 'bad' data may also increase, leading to a decrease in the model accuracy. In the case of a larger amount of training data, the performance of E3_5 E3_7 was still not as good as that of E3_2 E3_4, suggesting that the morphological distinctions between two real maize plants can better boost the generalisation ability of the model compared to the variations among deformed data derived from the same label. However, this disparity can be partially mitigated by expanding the volume of deformed data.
变形的数据。 E3_7低于E3_5，表明随着数据量的增加，“坏”数据也可能增加，导致模型精度下降。在训练数据量较大的情况下，E3_5 E3_7的性能仍然不如E3_2 E3_4，这表明与变异相比，两种真实玉米植株之间的形态差异可以更好地提升模型的泛化能力从同一标签派生的变形数据之间。然而，这种差异可以通过扩大变形数据量来部分缓解。

Table A5 表A5

Data selection strategy and test results of Experiment 3.
实验3的数据选择策略和测试结果。

Experiment

Name

Number of stem-missing label
缺茎标签数量

data selected in each leaf
每个叶子中选择的数据

number plant

Number of stem-complete label
茎完整标签数量

data selected in each leaf
每个叶子中选择的数据

number plant

Number of Deformed 变形数

Data Generated per Label
每个标签生成的数据

Data

Total Number of Deformed
变形总数

Point Clouds Used for
点云用于

Training

HAIS test result 海斯测试结果

Stem

mAP

Leaf

mAP

E3_2

180

3600

90.88

94.44

87.32

E3_3

3600

90.31

92.74

87.89

E3_4

3600

90.70

94.46

86.94

E3_5

1 (same as E3_1)
1（与E3_1相同）

1800

7200

85.80

84.82

86.78

Summary of strategies for generating deformation data.
生成变形数据的策略摘要。

In the subsequent model validation phase, we opted for a minimal approach by selecting only two labelled datasets for our experiments with the aim of assessing the efficacy of the deformation techniques further and minimising the sample labelling workload. One dataset was randomly selected from those with complete stems and the other from datasets lacking stems to ensure a broad representation of plant conditions. We standardized the setting of external forces by fixing all parameters

to 15 to maximise the diversity of the generated deformation data. We generated 1000 point clouds for each labelled point cloud, forming a training dataset of 22,000 point clouds. Thereafter, we randomly selected 100 out of the 1000 point clouds obtained from each annotated point cloud deformation, forming a training dataset of 2200 point clouds. We trained the PointNet++ and HAIS models with 2200 and 22,000 data points, respectively, to test the diversity provided by deformable data at different data volumes.
在随后的模型验证阶段，我们选择了一种最小方法，仅选择两个标记数据集进行实验，目的是进一步评估变形技术的有效性并最大限度地减少样本标记工作量。一个数据集是从具有完整茎的数据集中随机选择的，另一个数据集是从缺乏茎的数据集中随机选择的，以确保广泛代表植物状况。我们通过将所有参数

固定为 15 来标准化外力的设置，以最大限度地提高生成的变形数据的多样性。我们为每个标记点云生成 1000 个点云，形成 22,000 个点云的训练数据集。此后，我们从每个带注释的点云变形获得的1000个点云中随机选择100个，形成2200个点云的训练数据集。我们分别用 2200 个和 22,000 个数据点训练 PointNet++ 和 HAIS 模型，以测试不同数据量下可变形数据提供的多样性。

The model training procedure adhered to the protocols outlined in A.1. As part of the data preparation, all plant point clouds were translated to align their lowest point with the coordinate origin, thereby ensuring a uniform starting point for each dataset. Furthermore, the

-, and

ordinates of each point cloud were normalised independently to standardise the data input to the model. The effectiveness and accuracy of the trained model were subsequently evaluated using a substantial test set comprising 406 actual data points, which allowed for a comprehensive assessment of the model performance under varying conditions and data diversity levels.
模型训练程序遵循 A.1 中概述的协议。作为数据准备的一部分，所有植物点云都被平移，以将其最低点与坐标原点对齐，从而确保每个数据集都有统一的起点。此外，每个点云的

-、

- 和

坐标均独立标准化，以标准化模型的数据输入。随后使用包含 406 个实际数据点的大量测试集评估训练模型的有效性和准确性，从而可以在不同条件和数据多样性水平下全面评估模型性能。

References 参考

Ao, Z., Wu, F., Hu, S., Sun, Y., Su, Y., Guo, Q., Xin, Q., 2022. Automatic segmentation of stem and leaf components and individual maize plants in field terrestrial LiDAR data using convolutional neural networks. The Crop Journal, Crop Phenotyping Studies with Application to Crop Monitoring 10, 1239-1250. https://doi.org/10.1016/j . cj.2021.10.010

Boogaard, F.P., van Henten, E.J., Kootstra, G., 2021. Boosting plant-part segmentation of cucumber plants by enriching incomplete 3D point clouds with spectral data. Biosystems Engineering 211, 167-182. https://doi.org/10.1016/j . biosystemseng. 2021.09.004.

Charles, R.Q., Su, H., Kaichun, M., Guibas, L.J., 2017. PointNet: deep Learning on point sets for 3D classification and segmentation. In: In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85. https:// doi.org/10.1109/CVPR.2017.16 .

Chaudhury, A., Godin, C., 2020. Skeletonization of plant point cloud data using stochastic optimization framework. Front. Plant Sci. 11 https://doi.org/10.3389/ fpls. 2020.00773.
Chaudhury, A., Godin, C.，2020。使用随机优化框架对植物点云数据进行骨架化。正面。植物科学。 11 https://doi.org/10.3389/fpls。 2020.00773。

Chen, S., Fang, J., Zhang, Q., Liu, W., Wang, X., 2021. In: Hierarchical Aggregation for 3D Instance Segmentation. IEEE, Montreal, QC, Canada, pp. 15447-15456. https:// doi.org/10.1109/ICCV48922.2021.01518 .

de Gélis, I., Lefèvre, S., Corpetti, T., 2023. DC3DCD: unsupervised learning for multiclass 3D point cloud change detection. ISPRS Journal of Photogrammetry and Remote Sensing 206, 168-183. https://doi.org/10.1016/j.isprsjprs.2023.10.022 .

Du, R., Ma, Z., Xie, P., He, Y., Cen, H., 2023. PST: plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage. ISPRS Journal of Photogrammetry and Remote Sensing 195, 380-392. https://doi.org/10.1016/j . isprsjprs.2022.11.022.

Dutagaci, H., Rasti, P., Galopin, G., Rousseau, D., 2020. ROSE-X: an annotated data set for evaluation of 3D plant organ segmentation methods. Plant Methods 16, 28. https://doi.org/10.1186/s13007-020-00573-w .

Elnashef, B., Filin, S., Lati, R.N., 2019. Tensor-based classification and segmentation of three-dimensional point clouds for organ-level plant phenotyping and growth analysis. Computers and Electronics in Agriculture 156, 51-61. https://doi.org/

j.compag. 2018.10 .036).
Elnashef, B.、Filin, S.、Lati, R.N.，2019。基于张量的三维点云分类和分割，用于器官级植物表型分析和生长分析。农业中的计算机和电子产品 156, 51-61。 https://doi.org/

j.compag。 2018.10.036）。

Gabrani, M., Tretiak, O.J., 1996. Elastic transformations, in: Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers. Presented at the Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers, pp. 501-505 vol.1. DOI: 10.1109/ACSSC.1996.600972.
Gabrani, M., Tretiak, O.J.，1996。弹性变换，见：第 30 届阿西洛玛信号、系统和计算机会议的会议记录。发表于第 30 届阿西洛玛信号、系统和计算机会议的会议记录，第 501-505 页，第 1 卷。 DOI：10.1109/ACSSC.1996.600972。

Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.-Y., Cubuk, E.D., Le, Q.V., Zoph, B., 2021. Simple copy-paste is a strong data augmentation method for instance segmentation.
Ghiasi, G.、Cui, Y.、Srinivas, A.、Qian, R.、Lin, T.-Y.、Cubuk, E.D.、Le, Q.V.、Zoph, B., 2021。简单的复制粘贴是一种强大的方法用于实例分割的数据增强方法。
In: In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2917-2927. https://doi.org/10.1109/ CVPR46437.2021.00294.
见：见：2021 年 IEEE/CVF 计算机视觉和模式识别会议 (CVPR)。在 2021 年 IEEE/CVF 计算机视觉和模式识别会议 (CVPR) 上发表，第 2917-2927 页。 https://doi.org/10.1109/CVPR46437.2021.00294。

Gong, L., Du, X., Zhu, K., Lin, K., Lou, Q., Yuan, Z., Huang, G., Liu, C., 2021. Panicle-3D: efficient phenotyping tool for Precise semantic segmentation of Rice panicle point cloud. Plant Phenomics 2021. https://doi.org/10.34133/2021/9838929 .

Guo, X., Sun, Y., Yang, H., 2023. FF-net: feature-fusion-based network for semantic segmentation of 3D plant point cloud. Plants 12, 1867. https://doi.org/10.3390/ plants12091867.
郭X.，孙Y.，杨H.，2023。FF-net：用于3D植物点云语义分割的基于特征融合的网络。植物 12，1867 年。https://doi.org/10.3390/plants12091867。

Huang, H., Wu, S., Cohen-Or, D., Gong, M., Zhang, H., Li, G., Chen, B., 2013. L1-medial skeleton of point cloud. ACM Trans. Graph. 32, 1-8. https://doi.org/10.1145/ 2461912.2461913.
Huang, H.、Wu, S.、Cohen-Or, D.、Gong, M.、Zhang, H.、Li, G.、Chen, B., 2013。L1-点云内侧骨架。 ACM 翻译。图形。 32、1-8。 https://doi.org/10.1145/2461912.2461913。

Jiang, L., Zhao, H., Shi, S., Liu, S., Fu, C.-W., Jia, J., 2020. PointGroup: dual-set point grouping for 3D instance segmentation. In: Presented at the Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition, pp. 4867-4876.
Jiang, L.、Zhao, H.、Shi, S.、Liu, S.、Fu, C.-W.、Jia, J., 2020。PointGroup：用于 3D 实例分割的双集点分组。见：发表于 IEEE/CVF 计算机视觉和模式识别会议论文集，第 4867-4876 页。

Jin, S., Su, Y., Wu, F., Pang, S., Gao, S., Hu, T., Liu, J., Guo, Q., 2019. Stem-leaf segmentation and phenotypic trait Extraction of individual maize using terrestrial LiDAR data. IEEE T Geosci Remote 57, 1336-1346. https://doi.org/10.1109/ TGRS.2018.2866056.
金生，苏云，吴凤，庞生，高生，胡天，刘静，郭清，2019。茎叶分割与表型性状提取使用地面激光雷达数据对单个玉米进行分析。 IEEE T Geosci Remote 57, 1336-1346。 https://doi.org/10.1109/TGRS.2018.2866056。

Jin, S., Su, Y., Gao, S., Wu, F., Ma, Q., Xu, K., Ma, Q., Hu, T., Liu, J., Pang, S., Guan, H., Zhang, J., Guo, Q., 2020. Separating the structural components of maize for field phenotyping using terrestrial LiDAR data and deep convolutional neural networks. IEEE T Geosci Remote. https://doi.org/10.1109/TGRS.2019.2953092 .

Li, D., Cao, Y., Tang, X., Yan, S., Cai, X., 2018a. Leaf segmentation on dense plant point clouds with facet region growing. Sensors 18, 3625. https://doi.org/10.3390/ s18113625.
李丹，曹勇，唐新，严胜，蔡新，2018a。具有小面区域生长的密集植物点云上的叶子分割。传感器 18、3625。https://doi.org/10.3390/s18113625。

Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B., 2018. PointCNN: Convolution On XTransformed Points, in: Advances in Neural Information Processing Systems. Curran Associates, Inc.
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B., 2018。PointCNN：XTransformed Points 的卷积，见：神经信息处理系统的进展。柯伦联合公司

Li, D., Li, J., Xiang, S., Pan, A., 2022a. PSegNet: simultaneous semantic and instance segmentation for point clouds of plants. Plant Phenomics 2022. https://doi.org/

李达、李建、向胜、潘安，2022a。 PSegNet：植物点云的同步语义和实例分割。植物表型组学 2022。https://doi.org/

Li, D., Shi, G., Li, J., Chen, Y., Zhang, S., Xiang, S., Jin, S., Li, D., Shi, G., Li, J., Chen, Y., Zhang, S., Xiang, S., Jin, S., 2022b. PlantNet: a dual-function point cloud segmentation network for multiple plant species. ISPRS Journal of Photogrammetry and Remote Sensing 184, 243-263. https://doi.org/10.1016/j . isprsjprs. 2022.01.007.

Liu, Z., Tang, H., Lin, Y., Han, S., 2019. Point-voxel CNN for efficient 3D deep Learning. Advances in Neural Information Processing Systems. Curran Associates Inc.
Liu, Z., Tang, H., Lin, Y., Han, S., 2019。用于高效 3D 深度学习的点体素 CNN。神经信息处理系统的进展。柯伦联合公司

Luo, L., Jiang, X., Yang, Y., Samy, E.R.A., Lefsrud, M., Hoyos-Villegas, V., Sun, S., Luo, L., Jiang, X., Yang, Y., Samy, E.R.A., Lefsrud, M., Hoyos-Villegas, V., Sun, S., Luo, L.,
罗L.，江X.，杨Y.，萨米，E.R.A.，Lefsrud，M.，Hoyos-Villegas，V.，孙S.，罗L.，江X.，杨Y.。 , Samy, E.R.A., Lefsrud, M., Hoyos-Villegas, V., Sun, S., Luo, L.,

Jiang, X., Yang, Y., Samy, E.R.A., Lefsrud, M., Hoyos-Villegas, V., Sun, S., 2023. Eff3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient deep Learning. Plant Phenomics 5, 0080. https://doi.org/10.34133/ plantphenomics.0080.
Jiang, X., Yang, Y., Samy, E.R.A., Lefsrud, M., Hoyos-Villegas, V., Sun, S., 2023。 Eff3DPSeg：使用注释高效深度学习进行 3D 器官级植物芽分割。植物表型学 5, 0080。https://doi.org/10.34133/plantphenomics.0080。

Miao, T., Wen, W., Li, Y., Wu, S., Zhu, C., Guo, X., 2021a. Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots. GigaScience. https://doi.org/10.1093/ gigascience/giab031.
苗涛、文伟、李勇、吴胜、朱成、郭X.，2021a。 Label3DMaize：用于玉米芽 3D 点云数据注释的工具包。千兆科学。 https://doi.org/10.1093/gigascience/giab031。

Miao, T., Zhu, C., Xu, T., Yang, T., Li, N., Zhou, Y., Deng, H., 2021b. Automatic stem-leaf segmentation of maize shoots using three-dimensional point cloud. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2021.106310 .

Ngo, T.D., Hua, B.-S., Nguyen, K., 2023. ISBNet: a 3D point cloud instance segmentation network with instance-aware sampling and box-aware dynamic convolution. In: In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13550-13559. https://doi.org/10.1109/ CVPR52729.2023.01302.
Ngo, T.D.、Hua, B.-S.、Nguyen, K.，2023。ISBNet：具有实例感知采样和框感知动态卷积的 3D 点云实例分割网络。见：见：2023 年 IEEE/CVF 计算机视觉和模式识别会议 (CVPR)。在 2023 年 IEEE/CVF 计算机视觉和模式识别会议 (CVPR) 上发表，第 13550-13559 页。 https://doi.org/10.1109/CVPR52729.2023.01302。

Qi, C.R., Yi, L., Su, H., Guibas, L.J., 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, in: Advances in Neural Information Processing Systems. Curran Associates, Inc.
Qi, C.R.、Yi, L.、Su, H.、Guibas, L.J.，2017。PointNet++：度量空间中点集的深度层次特征学习，见：神经信息处理系统的进展。柯伦联合公司

Schunck, D., Magistri, F., Rosu, R.A., Cornelißen, A., Chebrolu, N., Paulus, S., Léon, J., Behnke, S., Stachniss, C., Kuhlmann, H., Klingbeil, L., 2021. Pheno4D: a spatiotemporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis. PLoS ONE 16, e0256340.
Schunck, D.、Magistri, F.、Rosu, R.A.、Cornelißen, A.、Chebrolu, N.、Paulus, S.、Léon, J.、Behnke, S.、Stachniss, C.、Kuhlmann, H.、Klingbeil , L., 2021。Pheno4D：玉米和番茄植物点云的时空数据集，用于表型分析和高级植物分析。 PLoS ONE 16，e0256340。

Shi, S., Wang, Z., Shi, J., Wang, X., Li, H., 2020. From points to Parts: 3D object detection from point cloud with Part-aware and Part-aggregation network. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1. https://doi.org/10.1109/TPAMI.2020.2977026 .

Sin, F.S., Schroeder, D., Barbič, J., 2013. Vega: non-Linear FEM deformable object simulator. Computer Graphics Forum 32, 36-48. https://doi.org/10.1111/j.14678659.2012.03230.x .

Su, W., Zhu, D., Huang, J., Guo, H., 2018. Estimation of the vertical leaf area profile of corn (Zea mays) plants using terrestrial laser scanning (TLS). COMPUTERS AND ELECTRONICS IN AGRICULTURE. https://doi.org/10.1016/j.compag.2018.03.037 .

Sun, Y., Zhang, Z., Sun, K., Li, S., Yu, J., Miao, L., Zhang, Z., Li, Y., Zhao, H., Hu, Z., Xin, D., Chen, Q., Zhu, R., 2023a. Soybean-MVS: annotated three-dimensional model dataset of whole growth period soybeans for 3D plant organ segmentation. Agriculture 13, 1321. https://doi.org/10.3390/agriculture13071321 .

Sun, Y., Zhang, Z., Sun, K., Li, S., Yu, J., Miao, L., Zhang, Z., Li, Y., Zhao, H., Hu, Z., Xin, D., Chen, Q., Zhu, R., Sun, Y., Zhang, Z., Sun, K., Li, S., Yu, J., Miao, L., Zhang, Z., Li, Y., Zhao, H., Hu, Z., Xin, D., Chen, Q., Zhu, R., Saeed, F., Sun, S.,
Rodriguez-Sanchez, J., Snider, J., Liu, T., Li, C., 2023b. Cotton plant part 3D segmentation and architectural trait extraction using point voxel convolutional neural networks. Plant Methods 19, 33. https://doi.org/10.1186/s13007-02300996-1 .

Turgut, K., Dutagaci, H., Galopin, G., Rousseau, D., 2022. Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods. Plant Methods 18, 20. https://doi.org/10.1186/s13007-022-00857-3 .

Wang, D., Song, Z., Miao, T., Zhu, C., Yang, X., Yang, T., Zhou, Y., Den, H., Xu, T., 2023. DFSP: a fast and automatic distance field-based stem-leaf segmentation pipeline for point cloud of maize shoot. Front. Plant Sci. 14, 1109314. https://doi.org/10.3389/ fpls.2023.1109314.
王丹、宋志、苗涛、朱成、杨晓、杨涛、周勇、登辉、徐涛，2023。DFSP：a基于距离场的快速自动玉米芽点云茎叶分割流程。正面。植物科学。 14、1109314。https://doi.org/10.3389/fpls.2023.1109314。

Wang, D., Wang, D., 2020. Unsupervised semantic and instance segmentation of forest point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 165, 86-97. https://doi.org/10.1016/j.isprsjprs.2020.04.020 .

Wu, S., Wen, W., Xiao, B., Guo, X., Du, J., Wang, C., Wang, Y., 2019. An Accurate Skeleton Extraction Approach From 3D Point Clouds of Maize Plants. Front. Plant Sci. DOI:

fpls. 2019.00248.
吴 S.、文 W.、肖 B.、郭 X.、杜 J.、王 C.、王 Y.，2019。玉米植株 3D 点云的精确骨架提取方法。正面。植物科学。 DOI：

fpls。 2019.00248。

Wu, S., Wen, W., Wang, Y., Fan, J., Wang, C., Gou, W., Guo, X., 2020. MVS-pheno: a portable and low-cost phenotyping platform for maize shoots using multiview stereo 3D reconstruction. Plant Phenomics. https://doi.org/10.34133/2020/1848437 .

Xiang, L., Bao, Y., Tang, L., Ortiz, D., Salas-Fernandez, M.G., Xiang, L., Bao, Y., Tang, L., Ortiz, D., Salas-Fernandez, M.G., 2019. Automated morphological traits extraction for sorghum plants via 3D point cloud data analysis. Computers and Electronics in Agriculture 162, 951-961. https://doi.org/10.1016/j.compag.2019.05.043 .

Xin, B., Sun, J., Bartholomeus, H., Kootstra, G., 2023. 3D data-augmentation methods for semantic segmentation of tomato plant parts. Front. Plant Sci. 14 https://doi.org/

fpls. 2023.1045545.
Xin, B.、Sun, J.、Bartholomeus, H.、Kootstra, G., 2023。番茄植株部位语义分割的 3D 数据增强方法。正面。植物科学。 14 https://doi.org/

fpls。 2023.1045545。

Yu, A., Grauman, K., 2017. Semantic jitter: dense supervision for visual Comparisons via synthetic images. In: Presented at the Proceedings of the IEEE International Conference on Computer Vision, pp. 5570-5579.
Yu, A., Grauman, K.，2017。语义抖动：通过合成图像进行视觉比较的密集监督。见：发表于 IEEE 国际计算机视觉会议论文集，第 5570-5579 页。

Zermas, D., Morellas, V., Mulla, D., Papanikolopoulos, N., 2020. 3D model processing for high throughput phenotype extraction - the case of corn. Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2019.105047 .

Zhang, Z., Yang, B., Wang, B., Li, B., 2023. GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds. DOI: 10.48550/arXiv.2305.16404.
张Z.，杨B.，王B.，李B.，2023。GrowSP：3D点云的无监督语义分割。 DOI：10.48550/arXiv.2305.16404。

Zhang, W., Wu, S., Wen, W., Lu, X., Wang, C., Gou, W., Li, Y., Guo, X., Zhao, C., 2023a. Three-dimensional branch segmentation and phenotype extraction of maize tassel based on deep learning. Plant Methods 19, 76. https://doi.org/10.1186/s13007 . 023-01051-9.

- Corresponding authors. 通讯作者。
E-mail addresses: miaoteng@syau.edu.cn (T. Miao), xutongyu@syau.edu.cn (T. Xu).
电子邮箱地址：miaoteng@syau.edu.cn (T. Miao), xutongyu@syau.edu.cn (T. Xu)。

Maize stem-leaf segmentation framework based on deformable point clouds 基于可变形点云的玉米茎叶分割框架

A R T I C L E I N F O文章信息

Keywords: 关键词：

Abstract 抽象的

1. Introduction 一、简介

2. Related works 2. 相关作品

2.1. Traditional plant point-cloud segmentation 2.1. 传统植物点云分割

2.2. Plant point-cloud segmentation via deep learning2.2.通过深度学习进行植物点云分割

2.3. Point-cloud data augmentation2.3.点云数据增强

3. Materials and methods3 材料与方法

3.1. Dataset construction3.1.数据集构建

3.2. The segmentation pipeline3.2.分割管道

3.2.1. Plant point-cloud deformation3.2.1.植物点云变形

3.2.2. Point-cloud segmentation model3.2.2.点云分割模型

3.2.3. Post-processing of instance segmentation results3.2.3.实例分割结果的后处理

4. Results 4. 结果

4.1. Point-cloud deformation effect4.1.点云变形效果

deformation 形变

4.2. Data augmentation effects4.2.数据增强效应

4.3. Segmentation results4.3.分割结果

5. Discussion 5. 讨论

5.1. Plant organ segmentation dataset5.1.植物器官分割数据集

5.2. The diversity of our deformation method5.2.我们变形方法的多样性

5.3. Stem-leaf segmentation5.3.茎叶分割

5.4. The scalability of our deformation method5.4.我们的变形方法的可扩展性

5.5. Limitations 5.5.局限性

6. Conclusion 六，结论

CRediT authorship contribution statementCRediT 作者贡献声明

Declaration of competing interest竞争利益声明

Acknowledgments 致谢

Appendix 附录

Implementation of enhanced algorithms增强算法的实现

A. 2 Model trainingA.2 模型训练

A. 3 Strategy for generating deformation dataA.3 变形数据生成策略

References 参考

Maize stem-leaf segmentation framework based on deformable point clouds
基于可变形点云的玉米茎叶分割框架

A R T I C L E I N F O
文章信息

2.1. Traditional plant point-cloud segmentation
2.1. 传统植物点云分割

2.2. Plant point-cloud segmentation via deep learning
2.2.通过深度学习进行植物点云分割

2.3. Point-cloud data augmentation
2.3.点云数据增强

3. Materials and methods
3 材料与方法

3.1. Dataset construction
3.1.数据集构建

3.2. The segmentation pipeline
3.2.分割管道

3.2.1. Plant point-cloud deformation
3.2.1.植物点云变形

3.2.2. Point-cloud segmentation model
3.2.2.点云分割模型

3.2.3. Post-processing of instance segmentation results
3.2.3.实例分割结果的后处理

4.1. Point-cloud deformation effect
4.1.点云变形效果

4.2. Data augmentation effects
4.2.数据增强效应

4.3. Segmentation results
4.3.分割结果

5.1. Plant organ segmentation dataset
5.1.植物器官分割数据集

5.2. The diversity of our deformation method
5.2.我们变形方法的多样性

5.3. Stem-leaf segmentation
5.3.茎叶分割

5.4. The scalability of our deformation method
5.4.我们的变形方法的可扩展性

CRediT authorship contribution statement
CRediT 作者贡献声明

Declaration of competing interest
竞争利益声明

Implementation of enhanced algorithms
增强算法的实现

A. 2 Model training
A.2 模型训练

A. 3 Strategy for generating deformation data
A.3 变形数据生成策略