This paper describes a method of tracking moving objects in a cluttered scene. If the motion of an object is similar to that of the background, the contour is not determined only by optical flow. It is also difficult to determine the contour only from edges because many edges may be extracted in the background and no edges may be extracted on some parts of the contour. But the contour is determined by using optical flow and edges in a long sequence. When an object motion is different from the neighborhood, the motion boundary is extracted as the contour. Some edges near the motion boundary are extracted and replace corresponding parts of the motion boundary. While the object is moving, more edges are added to the contour. When two objects with similar motion overlap, the contours are predicted by the stored contours and the current optical flow. The object boundary is determined by the edges near the predicted contours. Experimental results for synthetic and real scenes show the usefulness of the method.