The present disclosure relates to a signal processing device, a signal processing method, a program, and a signal transmission system, capable of enabling video switching in frame units of an original video signal. A pixel thinning control unit thin out every other pair of two neighboring samples of each line from N consecutive frames of a video signal including an m*n(48P-60P)*N/r:g:b/10-bit or 12-bit signal in which the number of pixels per frame exceeds the number of pixels specified in an HD format at intervals of 2N lines, map the thinned every other pair of two neighboring pixel samples to first to 4N-th sub images including an m′*n′/48P-60P/r′:g′:b′/10-bit or 12-bit signal, and arrange a vertical blanking region between two of N regions in which pixel samples of respective frames of the first video signal are mapped in each sub image.