As a new means of earth resource survey, land use change and coverage (LUCC) and ecological environment monitoring, remote sensing technology has a great advantage. The automatic classification for remote sensing image is the key technology to extract rich ground-object information and monitor the dynamic change of LUCC. Machine learning can flexibly build a model portrayed by parameters, and automatically extract information, which has been widely used in image classification because of its good robustness and convergence, and easy to be combined with other methods. Based on the study of traditional extreme learning machine (ELM) theory, the optimal selection of kernel function parameters and regularizing parameters were performed by using artificial fish swarm algorithm (AF) and the optimal ELM image classification model (AF-ELM) was constructed. The classification model used AF to optimize the wavelet kernel parameters and regularizing parameters of ELM to improve the classification accuracy. After that the classification for multi-spectral remote sensing image was implemented by using the parameter-optimized ELM classifier, meanwhile, compared with some standard classifier such as artificial neural networks(ANM), support vector machine (SVM) and extreme learning machine (ELM), and it was comparatively analyzed with the ELM polynomial kernel and RBF kernel classification algorithm. The experiments proved that optimal AF-ELM classifier was more faster and accurate, which was superior to those before-mentioned classifiers. It can be used for the automatic extraction of various elements from remote sensing image.