留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

3DPhenoFish: Application for two- and three-dimensional fish morphological phenotype extraction from point cloud analysis

Yu-Hang Liao Chao-Wei Zhou Wei-Zhen Liu Jing-Yi Jin Dong-Ye Li Fei Liu Ding-Ding Fan Yu Zou Zen-Bo Mu Jian Shen Chun-Na Liu Shi-Jun Xiao Xiao-Hui Yuan Hai-Ping Liu

Yu-Hang Liao, Chao-Wei Zhou, Wei-Zhen Liu, Jing-Yi Jin, Dong-Ye Li, Fei Liu, Ding-Ding Fan, Yu Zou, Zen-Bo Mu, Jian Shen, Chun-Na Liu, Shi-Jun Xiao, Xiao-Hui Yuan, Hai-Ping Liu. 3DPhenoFish: Application for two- and three-dimensional fish morphological phenotype extraction from point cloud analysis. Zoological Research, 2021, 42(4): 492-502. doi: 10.24272/j.issn.2095-8137.2021.141
Citation: Yu-Hang Liao, Chao-Wei Zhou, Wei-Zhen Liu, Jing-Yi Jin, Dong-Ye Li, Fei Liu, Ding-Ding Fan, Yu Zou, Zen-Bo Mu, Jian Shen, Chun-Na Liu, Shi-Jun Xiao, Xiao-Hui Yuan, Hai-Ping Liu. 3DPhenoFish: Application for two- and three-dimensional fish morphological phenotype extraction from point cloud analysis. Zoological Research, 2021, 42(4): 492-502. doi: 10.24272/j.issn.2095-8137.2021.141

3DPhenoFish: 基于点云分析的二维和三维鱼类形态表型分析和应用

doi: 10.24272/j.issn.2095-8137.2021.141

3DPhenoFish: Application for two- and three-dimensional fish morphological phenotype extraction from point cloud analysis

Funds: This work was supported by the National Natural Science Foundation of China (32072980) and Key Research and Development Projects in Tibet (XZ202001ZY0016N, XZ201902NB02, XZNKY-2019-C-053)
More Information
  • 摘要: 鱼类形态表型是水产养殖和生态学研究中人工育种、功能基因定位和群体遗传分析的重要资源。传统的形态学表型测量需要耗费大量的时间与劳动力,更重要的是人工测量高度依赖于操作经验,导致表型测量的结果具有一定的主观性。因此,我们开发了一个可以从三维点云数据中提取鱼类形态表型的软件3DPhenoFish。该软件提供了一个直观的用户界面,将背景剔除、坐标归一化、三维分割、关键点识别和表型提取的功能进行了整合。用户可以自动获取鱼体上18个关键形态点,基于关键点的二维表型,以及鱼体表面积和体积的三维表型。同时,3DPhenoFish还允许用户为自动识别的关键点进行微调,并自定义个性化的表型。基于3DPhenoFish,我们对四种高原特有的裂腹鱼亚科鱼类进行了高通量表型分析,包括拉萨裸裂尻、拉萨河尖裸鲤、双须叶须鱼和异齿裂腹鱼。结果表明,使用3DPhenoFish高通量提取的形态表型与人工测量结果表现出高度的线性相关性(>0.94)。基于高通量提取的形态学表型,我们可以很好地将不同物种进行区分,甚至可以区分同一物种的不同种群。综上所述,我们开发了高效、准确和可定制的鱼类表型分析工具3DPhenoFish,用于从三维点云数据中批量提取形态学表型,有助于克服人工测量中的低通量和高成本的一些传统挑战,因此3DPhenoFish可用于功能基因定位、新品种培育和资源保护研究中的表型分析。3DPhenoFish是一个开源软件,并可以在https://github.com/lyh24k/3DPhenoFish/tree/master免费下载。
    #Authors contributed equally to this work
  • Figure  1.  Workflow scheme in 3DPhenoFish for point cloud analysis and morphological phenotype extraction

    Whole pipeline includes data acquisition, data pre-processing, semantic segmentation, phenotype extraction, and data management.

    Figure  2.  Point cloud filtering for fish point clouds

    Process includes point cloud down-sampling (A), background filtering (B), outlier filtering (C), and final fish point cloud (D).

    Figure  3.  Coordinate normalization for fish point cloud

    Normalization process involves two steps: transferring point cloud of fish head from positive X-axis (A and B) to negative X-axis (C and D), and then transferring point cloud of fish dorsal fin to positive Y-axis (D).

    Figure  4.  Semantic segmentation for fish point cloud

    A: Pre-segment point cloud using super voxel method. B: Fin segmentation is performed using adaptive weighted region growth segmentation. C: Head, eye, body, and fins are segmented from point cloud, then used for following key point recognition and morphological phenotype extraction.

    Figure  5.  Key point recognition and phenotype extraction

    A: Main 2D phenotypes determined from distances among key points estimated directly on background plane of fish point cloud. B: Main 3D phenotypes estimated from point cloud conformation of fish, including arc length, surface, and volume. C: 3D phenotypes for head. Key points recognized in point cloud include snout point (A), front point of eye (B), back point of eye (C), external point of opercular (D), starting point of pectoral fin (E), end point of pectoral fin base (F), lowest point of ventral margin (G), starting point of ventral fin (H), end point of ventral fin base (I), starting point of anal fin (J), end point of anal fin (K), lower point of caudal peduncle (L), end point of coccyx (M), end point of tail fin (N), upper point of caudal peduncle (O), end point of dorsal fin (P), starting point of dorsal fin (Q), and highest point of dorsal margin (R).

    Figure  6.  Main interface of 3DPhenoFish

    A: Main point cloud image viewer. B: List of point clouds that need to be processed. C: Properties of current point cloud. D: List of key points for fish point cloud. E: List of morphological phenotypes for fish point cloud. F: List of operation records. G: Toolbar used to open and save files, adjust visual interface of point cloud, and automatically segment fish point cloud. Morphological phenotype extraction in the software must be executed strictly by down-sampling (Ⅰ), background removal (Ⅱ), and key point recognition (Ⅲ).

    Figure  7.  Linear correlation analysis of morphological phenotypes from 3DPhenoFish and manual measurement

    2D phenotypes were extracted from 3DPhenoFish and manual measurements were collected and compared for 30 randomly selected fish samples. Correlation coefficients of 17 morphological phenotypes were calculated (Supplementary Table S1), including full length (A), body length (B), dorsal snout distance (C), body height (D), caudal peduncle height (E), and head length (F).

    Figure  8.  Phenotype-based clustering of sample classifications of species and populations using linear discriminant analysis

    Samples from Schizopygopsis younghusbandi, Oxygymnocypris stewartii, Ptychobarbus dipogon, and Schizothorax oconnori were used for analysis. Clustering of samples using traditional 2D morphological phenotypes (A) and 2D and 3D morphological phenotypes (B). Clustering of S. younghusbandi samples using traditional 2D morphological phenotypes (C) and 2D and 3D morphological phenotypes (D).

    Figure  9.  Morphological phenotypes exhibited significant differences among species and Schizopygopsis younghusbandi populations

    Distribution of head height/head length (A) and dorsal snout distance/body length (B) for Schizothoracinae species and dorsal arc width/caudal arc width (C) and head volume/head length (D) for S. younghusbandi populations. Significant differences are shown by labels above bars, samples sharing no label letter indicate significant difference between two groups (P≤0.05).

  • [1] Aliyu I, Gana KJ, Musa AA, Agajo J, Orire AM, Abiodun FT, et al. 2017. A proposed fish counting algorithm using digital image processing technique. ATBU, Journal of Science, Technology & Education, 5(1): 1−11.
    [2] Balakrishnama S, Ganapathiraju A. 1998. Linear discriminant analysis-a brief tutorial. Institute for Signal and information Processing, 18(1998): 1−8.
    [3] Balta H, Velagic J, Bosschaerts W, De Cubber G, Siciliano B. 2018. Fast statistical outlier removal based method for large 3D point clouds of outdoor environments. IFAC-PapersOnLine, 51(22): 348−353. doi: 10.1016/j.ifacol.2018.11.566
    [4] Bär T, Reuter JF, Zöllner JM. 2012. Driver head pose and gaze estimation based on multi-template ICP 3-D point cloud alignment. In: Proceedings of 2012 15th International IEEE Conference on Intelligent Transportation Systems. Anchorage: IEEE, 1797–1802.
    [5] Batanov S D, Starostina O S, Baranova I A. 2019. Non-contact methods of cattle conformation assessment using mobile measuring systems. IOP Conference Series: Earth and Environmental Science, 315(3): 032006.
    [6] Besl PJ, McKay ND. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2): 239−256. doi: 10.1109/34.121791
    [7] Bradski G, Kaehler A. 2008. Learning OpenCV: Computer Vision with the OpenCV library. O'Reilly Media, Inc.
    [8] Chen XW, Zhu YL, Wu T, Wang ZQ. 2017. The point cloud registration technology based on SAC-IA and improved ICP. J Xi’an Polytech Univ, 31(3): 395−401. (in Chinese)
    [9] Comba L, Biglia A, Aimonino DR, Gay P. 2018. Unsupervised detection of vineyards by 3D point-cloud UAV photogrammetry for precision agriculture. Computers and Electronics in Agriculture, 155: 84−95. doi: 10.1016/j.compag.2018.10.005
    [10] Fernandes AFA, Turra EM, de Alvarenga ÉR, Passafaro TL, Lopes FB, Alves GFO, et al. 2020. Deep Learning image segmentation for extraction of fish body measurements and prediction of body weight and carcass traits in Nile tilapia. Computers and Electronics in Agriculture, 170: 105274. doi: 10.1016/j.compag.2020.105274
    [11] Gongal A, Karkee M, Amatya S. 2018. Apple fruit size estimation using a 3D machine vision system. Information Processing in Agriculture, 5(4): 498−503. doi: 10.1016/j.inpa.2018.06.002
    [12] Hao M M, Yu H L, Li D L. 2015. The measurement of fish size by machine vision-a review. In: Proceedings of 9th IFIP WG 5.14 International Conference on Computer and Computing Technologies in Agriculture. Cham: Springer, 15–32.
    [13] Hartmann A, Czauderna T, Hoffmann R, Stein N, Schreiber F. 2011. HTPheno: an image analysis pipeline for high-throughput plant phenotyping. BMC Bioinformatics, 12(1): 148. doi: 10.1186/1471-2105-12-148
    [14] Huang LW, Li SQ, Zhu AQ, Fan XY, Zhang CY, Wang HY. 2018. Non-contact body measurement for qinchuan cattle with LiDAR sensor. Sensors, 18(9): 3014. doi: 10.3390/s18093014
    [15] Le Cozler Y, Allain C, Caillot A, Delouard JM, Delattre L, Luginbuhl T, et al. 2019. High-precision scanning system for complete 3D cow body shape imaging and analysis of morphological traits. Computers and Electronics in Agriculture, 157: 447−453. doi: 10.1016/j.compag.2019.01.019
    [16] Li ML, Sun CM. 2018. Refinement of LiDAR point clouds using a super voxel based approach. ISPRS Journal of Photogrammetry and Remote Sensing, 143: 213−221. doi: 10.1016/j.isprsjprs.2018.03.010
    [17] Lin GC, Tang YC, Zou XJ, Xiong JT, Li JH. 2019. Guava detection and pose estimation using a low-cost RGB-D sensor in the field. Sensors, 19(2): 428. doi: 10.3390/s19020428
    [18] López-Fanjul C, Toro M Á. 2007. Fundamentos de la Mejora Genética en Acuicultura. Madrid: Genética y Genómica en Acuicultura, 155–182.
    [19] Navarro A, Lee-Montero I, Santana D, Henríquez P, Ferrer MA, Morales A, et al. 2016. IMAFISH_ML: a fully-automated image analysis software for assessing fish morphometric traits on gilthead seabream (Sparus aurata L.), meagre (Argyrosomus regius) and red porgy (Pagrus pagrus). Computers and Electronics in Agriculture, 121: 66−73. doi: 10.1016/j.compag.2015.11.015
    [20] Oliphant TE. 2007. Python for scientific computing. Computing in Science & Engineering, 9(3): 10−20.
    [21] Orts-Escolano S, Morell V, García-Rodríguez J, Cazorla M. 2013. Point cloud data filtering and downsampling using growing neural gas. In: Proceedings of the 2013 International Joint Conference on Neural Networks. Dallas: IEEE, 1–8.
    [22] Pérez-Ruiz M, Tarrat-Martín D, Sánchez-Guerrero MJ, Valera M. 2020. Advances in horse morphometric measurements using LiDAR. Computers and Electronics in Agriculture, 174: 105510. doi: 10.1016/j.compag.2020.105510
    [23] Pezzuolo A, Guarino M, Sartori L, Marinello F. 2018. A feasibility study on the use of a structured light depth-camera for three-dimensional body measurements of dairy cows in free-stall barns. Sensors, 18(2): 673.
    [24] Rusu RB, Cousins S. 2011. 3D is here: point cloud library (PCL). In: Proceedings of 2011 IEEE International Conference on Robotics and Automation. Shanghai: IEEE, 1–4.
    [25] Schnabel R, Wahl R, Klein R. 2007. Efficient RANSAC for point‐cloud shape detection. Computer Graphics Forum, 26(2): 214−226. doi: 10.1111/j.1467-8659.2007.01016.x
    [26] Schroeder WJ, Avila LS, Hoffman W. 2000. Visualizing with VTK: a tutorial. IEEE Computer Graphics and Applications, 20(5): 20−27. doi: 10.1109/38.865875
    [27] Shah SZH, Rauf HT, IkramUllah M, Khalid MS, Farooq M, Fatima M, et al. 2019. Fish-pak: fish species dataset from pakistan for visual features based classification. Data in Brief, 27: 104565. doi: 10.1016/j.dib.2019.104565
    [28] Spampinato C, Chen-Burger YH, Nadarajan G, Fisher RB. 2008. Detecting, tracking and counting fish in low quality unconstrained underwater videos. In: Proceedings of the 3rd International Conference on Computer Vision Theory and Applications. Madeira: DBLP, 514–519.
    [29] Spampinato C, Giordano D, Di Salvo R, Chen-Burger YHJ, Fisher RB, Nadarajan G. 2010. Automatic fish classification for underwater species behavior understanding. In: Proceedings of the 1st ACM International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams. Firenze: ACM, 45–50.
    [30] Vo AV, Truong-Hong L, Laefer DF, Bertolotto M. 2015. Octree-based region growing for point cloud segmentation. ISPRS Journal of Photogrammetry and Remote Sensing, 104: 88−100. doi: 10.1016/j.isprsjprs.2015.01.011
    [31] Wang G A, Hwang JN, Wallace F, Rose C. 2019. Multi-scale fish segmentation refinement and missing shape recovery. IEEE Access, 7: 52836−52845. doi: 10.1109/ACCESS.2019.2912612
    [32] Wang ZL, Walsh KB, Verma B. 2017. On-tree mango fruit size estimation using RGB-D images. Sensors, 17(12): 2738.
    [33] Wu YX, Li F, Liu FF, Cheng LN, Guo LL. 2016. Point cloud segmentation using Euclidean cluster extraction algorithm with the Smoothness. Meas Control Technol, 35(3): 36−38. (in Chinese)
    [34] Zermas D, Morellas V, Mulla D, Papanikolopoulos N. 2020. 3D model processing for high throughput phenotype extraction–the case of corn. Computers and Electronics in Agriculture, 172: 105047. doi: 10.1016/j.compag.2019.105047
  • ZR-2021-141 Supplementary Tables and Figures.pdf
  • 加载中
图(9)
计量
  • 文章访问数:  820
  • HTML全文浏览量:  422
  • PDF下载量:  189
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-04-24
  • 录用日期:  2021-07-05
  • 网络出版日期:  2021-07-06
  • 刊出日期:  2021-07-18

目录

    /

    返回文章
    返回