Statistical Theory of Signal Detection

3 Statistical Theory of Signal Detection

The gravitational-wave signal will be buried in the noise of the detector and the data from the detector will be a random process. Consequently the problem of extracting the signal from the noise is a statistical one. The basic idea behind the signal detection is that the presence of the signal changes the statistical characteristics of the data $x$ , in particular its probability distribution. When the signal is absent the data have probability density function (pdf) $p0(x )$ , and when the signal is present the pdf is $p1(x)$ .

A full exposition of the statistical theory of signal detection that is outlined here can be found in the monographs [102 , 56 , 98 , 96 , 66 , 44 , 77 ]. A general introduction to stochastic processes is given in [100 ]. Advanced treatment of the subject can be found in [64 , 101 ].

The problem of detecting the signal in noise can be posed as a statistical hypothesis testing problem. The null hypothesis $H0$ is that the signal is absent from the data and the alternative hypothesis $H1$ is that the signal is present. A hypothesis test (or decision rule) $δ$ is a partition of the observation set into two sets, $ℛ$ and its complement $ℛ ′$ . If data are in $ℛ$ we accept the null hypothesis, otherwise we reject it. There are two kinds of errors that we can make. A type I error is choosing hypothesis $H 1$ when $H0$ is true and a type II error is choosing $H0$ when $H1$ is true. In signal detection theory the probability of a type I error is called the false alarm probability, whereas the probability of a type II error is called the false dismissal probability. $1 − (false dismissal probability)$ is the probability of detection of the signal. In hypothesis testing the probability of a type I error is called the significance of the test, whereas $1 − (probability of type II error)$ is called the power of the test.

The problem is to find a test that is in some way optimal. There are several approaches to find such a test. The subject is covered in detail in many books on statistics, for example see references [54 , 41 , 62 ].

3.1 Bayesian approach
3.2 Minimax approach
3.3 Neyman–Pearson approach
3.4 Likelihood ratio test
3.4.1 Gaussian case – The matched filter