Extreme Learning Machine (ELM) is a neural network training paradigm that is characterized by simplicity, speed and high level of accuracy. The tuning of the network parameters is normally carried out with non-linear optimization algorithms that break this principle of simplicity and reduced execution time. This article shows that ELM network tuning can be performed efficiently by simple optimization algorithms, consistent with its basic philosophy. Experiments with 8 optimization algorithms are shown, considering 6 widely used databases in training algorithm benchmarks. The numerical results show that the Golden Section Algorithm dramatically reduces the network hyperparameter search time compared to the search while maintaining a high level of accuracy.