In this paper, a novel speech source localization (SSL) algorithm by the use of the central and lateral microphone arrays is proposed with low complexity and high precision in undesirable acoustical environments. Firstly, a proposed star-shaped microphone array (SSMA) at the room’s center is considered in combination with the improved 2D steered response power (2D-SRP) algorithm for determining the speakers’ directions, where the PENS algorithm is considered for speakers counting. Then, the two closest cross-shaped arrays (CSMA) to each speaker on the wall are selected, where these arrays prepare the proper information with less microphones. Each CSMA is considered in combination with subband generalized eigenvalue decomposition (SB-GEVD) method on various time frames for estimating the two others speaker’s direction. Therefore, the 3D speakers’ positions are calculated by the crossing of the estimated directions from SSMA and two CSMAs. The proposed lateral-central microphone array-multiple sound source localization method (LCMA-MSSL) is compared with other research, which shows the superiority of the proposed method in undesirable noisy and reverberant scenarios.