The angle of progression (AoP) is a crucial parameter for labor progress assessment and delivery decision guidance. Accurate AoP measurement requires automatically segmenting the pubic symphysis (PS) and fetal head (FH) in intrapartum ultrasound videos. Most established methods for AoP automated measurement are based on static ultrasound images, ignoring the temporal correlation between video frames. Together with limitations such as noise and limited training data, these methods cannot be applied well to ultrasound videos. To address these issues, this paper proposes an ultrasound video segmentation network (UVSN) for AoP measurement. First, an atrous convolutional block is proposed to extract semantic features at various scales in order to better capture semantic information from regions of different sizes and reduce semantic discrepancies between PS and FH. Additionally, a bi-directional spatial-temporal fusion module is introduced to utilize the contextual information of the current frame, enhancing feature representation for better. Finally, coordinate attention is used to aggregate and decode feature layers from different granularities to focus on the target region and enhance target localization. Extensive experiments on our private ultrasound video dataset have been carried out to compare our proposed method with current state-of-the-art image-based and video-based methods. The results demonstrate that our proposed method outperformed them with better segmentation performance and minimal AoP measurement error, which is expected to be helpful for obstetricians. Our code is available at https://github.com/Teksab/UVSN.