A tracking system based on Eight-Neighborhood Searching (ENS) has been developed to track moving target in complex background visible-band or infrared image sequences. The ENS is used for target localization. The tracking system can be controlled according to the tracking state parameter, which is evaluated by the target region features. The target feature used in the algorithm is the boundary of the object. Both global and local features are computed based on the boundary. Also a method based on motionn estimation is compared with ENS. Our system has been tested on several image ssequences in visible-band, which include cars and ships. Although the camera moves freely in the sequences, accurate tracking results were obtained. The motion of the camera makes it impossible to estimate the motion of the object. Processing speed on a Pentium II 400 MHz PC was approximately 30 frames/second. The tracking system works very well on a 400-frames sequence of moving cars, in which the template is automatically updated at a step of 20 frames.