Invention Grant
- Patent Title: Differencing based self-supervised scene change detection (D-SSCD) with temporal consistency
-
Application No.: US17691397Application Date: 2022-03-10
-
Publication No.: US12062188B2Publication Date: 2024-08-13
- Inventor: Vijaya Raghavan Thiruvengadathan Ramkumar , Bahram Zonooz , Elahe Arani
- Applicant: NavInfo Europe B.V.
- Applicant Address: NL Eindhoven
- Assignee: NavInfo Europe B.V.
- Current Assignee: NavInfo Europe B.V.
- Current Assignee Address: NL Eindhoven
- Agency: Peacock Law P.C.
- Agent Justin R. Muehlmeyer
- Main IPC: G06T7/20
- IPC: G06T7/20 ; G06N3/088

Abstract:
A computer implemented network for executing a self-supervised scene change detection method in which image pairs (T0, T1) from different time instances are subjected to random photometric transformations to obtain two pairs of augmented images (T0→T0′, T0″; T1→T1′, T1″), which augmented images are passed into an encoder (fθ) and a projection head (gϕ) to provide corresponding feature representations. Absolute feature differencing is applied over the outputs of the projection head (gϕ) to obtain difference representations (d1, d2) of changed features between the pair of images, and a self-supervised objective function (LSSL) is applied on the difference representations d1 and d2 to maximize a cross-correlation of the changed features, wherein d1 and d2 are defined as
d
1
=
❘
"\[LeftBracketingBar]"
g
(
f
(
T
0
′
)
)
-
g
(
f
(
T
0
′
)
)
❘
"\[RightBracketingBar]"
d
2
=
❘
"\[LeftBracketingBar]"
g
(
f
(
T
0
″
)
)
-
g
(
f
(
T
1
″
)
)
❘
"\[RightBracketingBar]"
(
1
)
Furthermore, an invariant prediction and change consistency loss is applied in the D-SSCD Network to reduce the effects of differences in the lighting conditions or camera viewpoints by enhancing the image alignment between the temporal images in the decision and feature space.
d
1
=
❘
"\[LeftBracketingBar]"
g
(
f
(
T
0
′
)
)
-
g
(
f
(
T
0
′
)
)
❘
"\[RightBracketingBar]"
d
2
=
❘
"\[LeftBracketingBar]"
g
(
f
(
T
0
″
)
)
-
g
(
f
(
T
1
″
)
)
❘
"\[RightBracketingBar]"
(
1
)
Furthermore, an invariant prediction and change consistency loss is applied in the D-SSCD Network to reduce the effects of differences in the lighting conditions or camera viewpoints by enhancing the image alignment between the temporal images in the decision and feature space.
Public/Granted literature
- US20230289977A1 Differencing Based Self-Supervised Scene Change Detection (D-SSCD) with Temporal Consistency Public/Granted day:2023-09-14
Information query