Whitening long range dependence in large sample covariance matrices of multivariate stationary processes

06/03/2020

∙

by   Peng Tian, et al.

∙

0

∙

share

Let 𝐗 be an N× T data matrix which can be represented as 𝐗=𝐂_N^1/2𝐙𝐑_T^1/2 with 𝐙 an N× T random matrix whose rows are spherically symmetric, 𝐑_T a deterministic T× T positive definite Toeplitz matrix, and 𝐂_N a deterministic N× N nonnegative definite matrix. In particular, 𝐙 can have i.i.d standard Gaussian entries. We prove the weak consistency of an unbiased estimator 𝐑_T=(r̂_i-j) of ξ_N𝐑_T where ξ_N=N^-1tr 𝐂_N, r̂_k is the average of the entries on the kth diagonal of T^-1𝐗^*𝐗. When each row of 𝐗 are long range dependent, i.e. the spectral density of Toeplitz matrix 𝐑_T is regularly varying at 0 with exponent a∈ (-1,0), we prove that although 𝐑̂_T may not be consistent in spectral norm, a weaker consistency of the form 𝐑_T^-1/2𝐑̂_T 𝐑_T^-1/2 - ξ_N 𝐈a.s.0 still holds when N,T→∞ with N≫log^3/2 T. We also establish useful probability bounds for deviations of the above convergence. It is shown next that this is strong enough for the implementation of a whitening procedure. We then apply the above result to a complex Gaussian signal detection problem where 𝐂_N is a finite rank perturbation of the identity.