Video events recognition is a challenging task for high-level understanding of video se- quence. At present, there are two major limitations in existing methods for events recognition. One is that no algorithms are available to recognize events which happen alternately. The other is that the temporal relationship between atomic actions is not fully utilized. Aiming at these problems, an algo- rithm based on an extended stochastic context-free grammar (SCFG) representation is proposed for events recognition. Events are modeled by a series of atomic actions and represented by an extended SCFG. The extended SCFG can express the hierarchical structure of the events and the temporal re- lationship between the atomic actions. In comparison with previous work, the main contributions of this paper are as follows: ① Events (include alternating events) can be recognized by an improved stochastic parsing and shortest path finding algorithm. ② The algorithm can disambiguate the detec- tion results of atomic actions by event context. Experimental results show that the proposed algo- rithm can recognize events accurately and most atomic action detection errors can be corrected sim- ultaneously.