Un problema reale: Shot detection detection

Transcript

http://imagelab ing unimo it
http://imagelab.ing.unimo.it
Dispense del corso di Elaborazione di Immagini e Audio Digitali
Un problema reale:
Shot detection
Prof. Roberto Vezzani
Roberto Vezzani - Imagelab – Università di Modena e Reggio Emilia
Un esempio di elaborazione di video
• In questa lezione affrontiamo un problema reale di
elaborazione
l b
i
di un video
id
• Dato un problema da risolvere, vedremo diverse strategie
per affrontarlo
ff t l e i metodi
t di per valutarne
l t
le
l prestazioni
t i i
Cosa è la “Shot Detection”?
• Problem definition
– shot detection: given a video V consisting of n
shots, find the beginning and end of each shot.
• Also known as shot boundary detection or
transition detection.
• It is fundamental to any kind of video analysis
and video application since it enables
segmentation of a video into its basic components:
the shots.
shots
Struttura di un video
Classificazione delle scene (Pattern recognition)
recognition)
Esempio: classificazione di shot in un video preso da telegiornale
Classificazione delle transizioni
• Hard cuts: A cut is an instantaneous transition
from one scene to the next.
next
• Fades: A fade is a gradual transition between a
scene and a constant image (fade-out)
(fade out) or between
a constant image and a scene (fade-in).
• Dissolves: A dissolve is a ggradual transition from
one scene to another, in which the first scene
fades out and the second scene fades in.
• Wipe
Wi
Wipe:
another
h common scene break
b k is
i a wipe,
i
in
i
which a line moves across the screen, with the
new scene appearing behind the line.
line
Fades
• During a fade, images have their intensities multiplied
b some value
by
l α. During
D i a ffade-in,
d i α increases
i
ffrom 0
to 1, while during a fade-out α decreases from 1 to 0.
Dissolvenza
• Combinazione di fade-in e fade-out.
Cut Detection
• Normalmente i frame di un video derivano dal
campionamento
i
temporale
l di una funzione
f
i
continua.
i
Se
S il
framerate è sufficientemente alto, le variazioni interframe
devono essere “piccole”
piccole
• Serie temporale delle dissimilarità: f(n).
• n = numero del frame,
frame N numero totale di frames.
frames
• Suppose we use function d(x,y) to measure the
dissimilarity between frame x and y.
y The discontinuity
feature value for frame n is f(n)=d(n-1,n).
• Pick the cuts position from f(n) based on some threshold
techniques.
Esempio
Misure di dissimilarità
• Intensity/color histogram
d H ( f ),
) H (g) 
255
2


H
(
f
)(
i
)

H
(
g
)(
i
)

i 0
The Edge Change Ratio ECR(n, k)
imd1 = rgb2gray(im1);
Imd2 = rgb2gray(im2);
% black background image
bw1 = edge(imd1,
g (
, 'sobel');
);
bw2 = edge(imd2, 'sobel');
% invert image to white background
ibw2 = 1-bw2;
ibw1 = 1-bw1;
1 bw1;
s1 = size(find(bw1),1);
s2 = size(find(bw1),1);
% dilate
se = strel('square',3);
dbw1 = imdilate(bw1, se);
dbw2 = imdilate(bw2, se);
imIn = dbw1 & ibw2;
imOut = dbw2 & ibw1;
ECRIn = size(find(imIn),1)/s2;
(
(
), ) ;
ECROut = size(find(imOut),1)/s1;
ECR = max(ECRIn, ECROut);
Thresholding – come calcolare la soglia
• Global threshold
A hard
h d cut is
i declared
d l d each
h time
i
the
h discontinuity
di
i i value
l
f(n) surpasses a global thresholds.
• Adaptive threshold
A hard cut is detected based on the difference of the
current feature values f(n)
( ) from its local neighborhood.
g
Generally this kind of method has 2 criteria for a hard cut
declaration:
- F(n)
F( ) ttakes
k the
th maximum
i
value
l inside
i id the
th neighborhood.
i hb h d
- The difference between f(n) and its neighbors’ feature
values is bigger than a given threshold.
threshold
•
•
•
•
Metodi empirici
Metodi manuali
Modellazione matematica del sistema
Modellazione probabilistica del sistema
Metodi probabilistici: formule base


Use a slide window with size 2w+1.
The middle frame in the window is detected as a cut if:
- Its feature value is the maximum in the window.
- Its feature value is greater than
max( left  Td  left ,  right  Td  right )
where Td is a p
parameter g
given a value of 5 in this
experiment.

The statistics model is based on following
g assumption:
p
The dissimilarity feature values f(n) for a frame comes from two distributions:
one for shot boundaries(S) and one for “not“not-a-shotshot-boundary”(N). In general,
S has
as a co
considerably
s de ab y larger
a ge mean
ea a
and
d sta
standard
da d de
deviation
at o tthan
a N.
Experiments
• Input: a lot of frames…
• Dissimilarity function
- Intensity histogram
- Edge change ratio (ECR)
• Thresholding
- Adaptive threshold based on statistics model.
Results
• Intensity histogram dissimilarity + adaptive thresholding
Results(cont.)
• ECR dissimilarity + adaptive thresholding
Analisi delle prestazioni
• Precision and Recall
Pr ecision 
Re call 
correttame nte _ identifica ti
correttame nte _ identifica ti  falsi _ allarmi
correttame nte _ identifica ti
correttame nte _ identifica ti  identifica zioni _ perse
F 
2 * Pr ecision * Re call
Pr ecision  Re call
• Curve ROC: coppie (Precision-Recall) per diversi valori di
eventuali soglie o parametri
Curva ROC - Receiver operating characteristic
(da wikipedia)
Problemi correlati
• Estrazione dei keyframes
• Uso dei keyframes nel video editing

Un problema reale: Shot detection detection

Transcript

Documenti analoghi

Laboratorio Laboratorio – Elaborazione Elaborazione di Video

curriculum VERONICA GABRIELLI short

eia_VideoProcessing

Presentazione M2 Magotti Paolo 30092009RE

Convenzione - calzificio Mauriziano

Industria Chimica Reggiana ICRSpA Sede legale: Via M. Gasparini

CampusOne- ICT Gennaio 2004 Responsabile Letizia Leonardi ICT

Ognibene Power (Reggio Emilia

Tombstone pubblicazione bilancio e relazione CG

Verbale di nomina dei rappresentanti dei genitori per la primaria