An exact sequence $0\rightarrow A\rightarrow B\rightarrow C\rightarrow 0$ of left $R$-modules is called cyclically pure if for every right ideal $I$ of $R$, the sequence $0\rightarrow (R/I)\otimes A \rightarrow (R/I)\otimes B \rightarrow (R/I)\otimes C \rightarrow 0$ is exact. In this paper, we study some special modules with respect to cyclic purity, such as $CP$-projective, $CP$-injective and $CP$-flat modules.