We present an efficient entanglement purification protocol (EPP) with controlled-not (CNOT) gates and linear optics. With the CNOT gates, our EPP can reach a higher fidelity than the conventional one. Moreover, it does not require the fidelity of the initial mixed state to satisfy 1 2. If the initial state is not entangled, it still can be purified. With the linear optics, this protocol can get pure maximally entangled pairs with some probabilities. Meanwhile, it can be used to purify the entanglement between the atomic ensembles in distant locations. This protocol may be useful in long-distance quantum communication.