Biodiv Sci ›› 2024, Vol. 32 ›› Issue (4): 23400.  DOI: 10.17520/biods.2023400  cstr: 32101.14.biods.2023400

• Reviews • Previous Articles     Next Articles

Advances in lineage sorting theories and their detection methods

Yanwen Lv1,2(), Ziyun Wang1,2(), Yu Xiao1,2(), Zihan He1,2(), Chao Wu1,2(), Xinsheng Hu1,2,*()()   

  1. 1 College of Forestry and Landscape Architecture, South China Agricultural University, Guangzhou 510642
    2 Guangdong Key Laboratory for Innovative Development and Utilization of Forest Plant Germplasm, Guangzhou 510642
  • Received:2023-10-22 Accepted:2024-02-20 Online:2024-04-20 Published:2024-03-28
  • Contact: * E-mail: xinsheng@scau.edu.cn

Abstract:

Background & Aims: Lineage sorting (LS) refers to the process in which multiple populations are descended from a common ancestral population or species and are now reproductively isolated from one another. It provides an approach to gain insights into speciation, and is often classified into the phases of polyphyly, paraphyly, and monophyly. The first two phases are in the state of incomplete lineage sorting (ILS) where gene trees do not correctly reflect species trees. The third phase is in the state of complete LS where gene trees are concordant with species trees. Here, we reviewed relevant theories and summarized recent progresses in method for LS detection.

Progress: We first systematically discussed the coalescent theories of how genome sites with distinct evolutionary properties (neutral or selective) in an ancestral population were transmitted to progeny populations. We discussed the potential relationships between gene trees and species trees for neutral and selective genes, respectively. Secondly, we delved into LS analyses based on the neutral DNA sequences, including construction of phylogeny under ILS and the network-based phylogenetic analysis. We then discussed the impacts of selection on LS analysis and methods for detecting both directional and balancing selection based on gene trees and species trees. Finally, we discussed a few open questions about the effects of mating system on LS, the detection of ILS, and the effects of pollen and seed flow on LS.

Prospect: New theories are needed to explore how mating system shapes the LS process for both selective and neutral genes. To appropriately assess ILS for individual genes based on species trees, it is crucial to improve the method estimating species trees and to fully utilize the potential of genome sequence data in future study. Given a high frequency of natural hybridization in plant species, a phylogenetic network method is needed to simultaneously examine pollen and seed flow together with ILS. Answers to these questions could help us to understand in-depth the LS process in plant species.

Key words: phylogeny, gene tree, species tree, gene flow, selection