Deblocking is a step in video coding for removing distortions that may result from dividing a video frame into blocks, and encoding the video frame based on the blocks. Techniques described herein can include determining the activity in neighboring blocks along the boundary of the blocks, where the activity measures smoothness or complexity of pixels in the boundary area. An average of the activity can then be determined, as well a difference in the activity between the left block and the right block. The average activity and the difference in activity can then be used to determine a classification for the boundary area. The classification can further be used to select a filter to apply to the pixels in the boundary area. Once the filter have been applied, the blocks can be added to a reconstructed video frame.