Diagonal Loading, Robust Signal Processing under Snapshot Constraints and Stochastic Matrices
Arthur Baggeroer
Massachusetts Institute of Technology
Email:
The concept of diagonal loading for may purposes including inversion, desensitization, stability and rank deficiencies has been used in the engineering and science literature for many. It goes under many names beyond diagonal loading, regularization, ridge regression, white noise gain constraint, plus many others. The general issue is to keep projections with low signal to noise ratios from dominating solutions and then lead to unstable and physically meaningless results. This often involves both the low eigenvalues and their associated eigenvectors of a sample covariance matrix as typically
used in most adaptive systems. This presentation will provide a brief overview of the engineering literature starting with the MVDR (Minimum Variance Distortionless
Response) for frequency wavenumber estimation of Capon and the closely related Yule-Walker and Wiener-Levinson algorithms associated with deconvolution, whitening filters, and "maximum entropy spectral" estimation. Almost all these problems involve the statistics of the sample covariance matrix which are often estimated with sample support L less than or near N, the dimension of the sensors or filter lengths involved. One can be lead to the odd result that more sensors lead to poorer performance in adaptive array processing because of these issues.
Advances in Stochastic Eigen-Analysis
Alan Edelman
Massachusetts Institute of Technology
Email:
Stochastic Eigen-Analysis (random matrix theory) is now a big subject with applications in many fields of science, finance and engineering. This talk surveys some of the important mathematics that is a very modern development as well as the computational software that is transforming theory into useful practice. An application to rank detection will be discussed.
Asymptotic Mean Squared Error Performance of
Diagonally Loaded Capon-MVDR Processor
Christ D. Richmond*, N. Raj Rao**, and Alan Edelman**
*MIT Lincoln Laboratory
Email:
**Massachusetts Institute of Technology
Email: [raj, edelman] @mit.edu
The asymptotic local error mean squared error (MSE) performance of the Capon algorithm, a.k.a the minimum variance distortionless response (MVDR) spectral estimator, has been studied extensively by several authors. Stoica et al. [1], Vaidyanathan and Buckley [2], and Hawkes and Nehorai [3] have exploited Taylor's theorem and complex gradient methods to provide accurate prediction of the MSE performance of these signal parameter estimates that is valid (i) above the estimation threshold signal-to-noise ratio (SNR) and (ii) provided a sufficient number of training samples is available for covariance estimation. The goal of this present analysis is to extend these results to the case in which the sample covariance matrix is diagonally loaded, as is often done in practice for regularization, stabilizing matrix inversion, and white noise gain control [4]. Recent advances in random matrix theory facilitate calculation of the required moments of the inverse of a diagonally loaded complex Wishart matrix [5], allowing for the first time accurate MSE performance predictions for the snapshot deficient case often encountered in non-stationary environments. This initial work focuses on the MSE prediction of angle estimates derived for the canonical case of single and multiple planewave signals in white noise. Consideration is also given to the impact of array response mismatch.
* This work was sponsored by the NAVSEA PEO-IWSS under Air Force contract FA8721-05-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Government.
[1] P. Stoica, P. Handel, T. Soderstrom, “Study of Capon Method for Array Signal Processing,” Circuits Syst. Signal Processing, Vol. 14, No. 6, pp. 749--770, 1995.
[2] C. Vaidyanathan, K. M. Buckley, “Performance Analysis of the MVDR Spatial Spectral Estimator,” IEEE Transactions on Signal Processing, Vol. 43, No. 6, pp. 1427--1437, June 1995.
[3] M. Hawkes, A. Nehorai, “Acoustic Vector-Sensor Beamforming and Capon Direction Estimation,” IEEE Transactions on Signal Processing, Vol. 46, No. 9, 2291--2304, 1998.
[4] H. Cox, R. Zeskind, M. Owen, “Robust Adaptive Beamforming,” IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 35, No. 10, pp. 1365--1376, October 1987.
[5] R. Rao Nadakuditi, A. Edelman, “The Polynomial Method for Random Matrices,” In preparation.
Panel Discussion: The Importance of Stochastic Eigen-Analysis (SEA)
Henry Cox, Joseph Guerci, Arogyaswami Paulraj, Donald Tufts
Moderator: Steven Thomas Smith
This panel discussion explores the contributions and utility of SEA to both engineering applications and mathematical knowledge. Relevance to the fields of array processing, communications, parameter estimation, statistical analysis, and random matrix theory will be covered, as well as the interdisciplinary aspects that SEA has within these areas. This panel is intended to provide both historical and contemporary context for the subjects presented at the Workshop, as well as hinting at future developments, which will be explored more fully by the second day's panel.
Biographies of Panelists
Henry Cox received the Doctor of Science degree form MIT in 1963. He spent the 1972-1973 academic year as a research associate at the Marine Physical laboratory of Scripps Institution of Oceanography. He is currently a senior principal research scientist at Lockheed Martin Orincon. Prior to its acquisition in 1993, he was the senior vice present and chief technology officer of the Orincon Corporation He has over 40 years of experience in the application of advanced signal processing technology to problems of national defense. He is an authority on antisubmarine warfare, and a recognized technical expert in signal processing, antenna arrays, underwater acoustics, and sonar system design and analysis. He currently serves on several of senior technical advisory panels for the Navy and other agencies. His recent technical contributions include development of new off-board sensor concepts, real time processing systems for very large arrays, algorithms for robust adaptive beamforming, development of simplified techniques for matched field processing, pioneering work in bi-static active sonar, new processing methods for Doppler exploitation in active sonar, and for passive ranging, and development of system approaches for passive distributed systems and shallow-water active sonar. From 1981 to 1991, he was a Vice President of BBN. During his distinguished carrier as a naval officer, he held a number of important R&D positions, retiring as Captain USN in 1981. He was the Project Manager for the Undersea Surveillance Project, Division Director at DARPA, and Officer in Charge of the New London Laboratory of the Naval Underwater Systems Center. Dr. Cox is the author of more than 50 technical papers. He is a fellow of both the Institute of Electrical and Electronics Engineering and the Acoustical Society of America. He was awarded the Gold Medal of the American Society of Naval Engineers. In 1991 he received the Distinguished Technical Achievement Award of the Ocean Engineering Society. He was elected to the National Academy of Engineering in 2002.
Dr. Joseph Guerci is currently the Director of the Special Projects Office, of the Defense Advanced Research Projects Agency (or DARPA). In this capacity, he is responsible for leading the development of some of the nation’s most advanced technologies and systems aimed at meeting emerging national defense needs from underground to outerspace—literally. The Special Projects Office is home to some of DARPA’s most advanced generation after next sensor and signal processing systems from airship and spaceborne electronically scanned antennas in the 1000 m^2—plus class, to entirely new methods of chem/bio detection and advanced multidimensional real-time knowledge-aided adaptive signal processing. Many of these systems have been or are in the process of being successfully transitioned to a multitude of real-world end-users.
An alumnus of Polytechnic University of New York with a PhD in Systems Engineering, Dr. Guerci has over 60 peer-reviewed publications, eight US patents, and is the author of the recent book titled Space-Time Adaptive Processing for Radar. He is also a member of the IEEE Radar Systems Panel and a Fellow of the IEEE.
Professor Arogyaswami Paulraj supervises the Smart Antenna Research Group in the Dept. of Electrical Engineering, Stanford University and is a pioneer in space-time (MIMO) wireless systems. He has helped shape research and development worldwide in this new technology. He is the author of over 300 research papers and holds twenty three patents. He is a fellow of the IEEE and the Indian National Academy of Engineering. He founded Iospan Wireless in late 90s to develop fixed wireless solutions. He is a co-founder of Beceem Communications Inc. currently developing semiconductors for broadband wireless interne
Don Tufts received three of his four academic degrees from the Massachusetts Institute of Technology, S.B. ’57, S.M. ’58, and Sc.D. ’60, all in Electrical Engineering and graduated from Williams College in 1955. He was a Bell Laboratories Fellow at M.I.T. From 1960 to 1967 he was at Harvard University, first as Research Fellow and Lecturer and then as Assistant Professor of Applied Mathematics. Since 1967 he has been Professor of Electrical and Computer Engineering at the University of Rhode Island, where his research and teaching interests are in the areas of signal processing, communications, and computer systems. .He has been a consultant for Bell Laboratories, the National Academy of Sciences, the Institute for Defense Analyses, the Office of Naval Research, and for several companies.
He is an IEEE Fellow (1982) "for contributions to digital communications and signal processing". In 2000 he was awarded the IEEE Milenium Medal and the Technical Achievement Award of the IEEE Signal Processing Society. At URI he received the College of Engineering Research Excellence Award in 1987 and the URI University Scholarly Achievement Award in 1998.
Steven Thomas Smith was born in La Jolla, CA in 1963. He received the B.A.Sc. degree in electrical engineering and mathematics from the University of British Columbia, Vancouver, BC in 1986 and the Ph.D. degree in applied mathematics from Harvard University, Cambridge, MA in 1993. From 1986 to 1988 he was a research engineer at ERIM, Ann Arbor, MI, where he developed morphological image processing algorithms. He is currently a senior member of the technical staff at MIT Lincoln Laboratory, which he joined in 1993. His research involves algorithms for adaptive signal processing, detection, and tracking to enhance radar and sonar systems performance. He has taught signal processing courses at Harvard and for the IEEE. His most recent work addresses intrinsic estimation and superresolution bounds, mean and variance CFAR, advanced tracking methods, and space-time adaptive processing algorithms. He was an associate editor of the IEEE Transactions on Signal Processing (2000–2002), and received the SIAM outstanding paper award in 2001.
Several Problems from Adaptive Signal Processing where Complicated Functions of Random Matrices Arise, Naturally
Louis Scharf
Colorado State University
Email:
In this talk we review several problems that arise in the detection or estimation of subspace signals in noise and interference of unknown covariance. Fairly rigorous reasoning from principles of invariance, from generalized likelihoods, or from conjugate gradient searches produce statistics that are complicated functions of sample covariances. We pose several questions about these functions that may be illuminated by the theory of large random matrices.
Bayesian Bounds
Harry Van Trees
George Mason University
Email:
Two problems of widespread interest are the estimation of parameters and the estimation of random waveforms which are observed via a nonlinear transformation in the presence of noise. In most of these cases, it is difficult to find the optimum solution. A widely-used technique is to find a bound on the performance of any estimator, or some class of estimators, and compare the performance of various sub-optimal estimators to the bound.
The talk will discuss Bayesian bounds with emphasis on Bayesian Cramer-Rao bounds. Early work on nonlinear parameter estimation will reviewed and recent work on nonlinear filtering and tracking will be described. Open issues in the area will be discussed.
Reduced Rank Filtering for Random Covariance Matrices
Ralf Müller
Norwegian University of Science and Technology (NTNU)
Email:
We consider linear multistage detectors with universal (large system) weighting for synchronous code division multiple access (CDMA) in multipath fading channels with many users. A convenient choice of the basis of the projection subspace allows a joint projection of all users. Taking advantage of this property, the complexity per bit of multistage detectors with universal weights scales linearly with the number of users on the uplink CDMA channel, while other known multistage detectors with universal weights and different bases of the projection subspace keep the same quadratic complexity order per bit as the linear minimum mean square error (LMMSE) detector.
We focus on the design of two kinds of detectors with linear complexity. The detector of Type I is obtained as asymptotic approximation of the polynomial expansion detector proposed by Moshavi et al. The detector of Type II has the same performance as the multistage Wiener filter (MSWF) in large systems.
Issues for Statistical Eigen-Analysis from Adaptive Beamforming
Henry Cox
Lockheed Martin Orincon
Email:
Matrix inversions, eigen-decompositions and singular value decompositions of the sample covariance matrix or data matrix play an important role in both the implementation and analysis of adaptive beamforming algorithms. This presentation reviews a number of adaptive beamforming problems from the point of view of identifying issues for statistical eigen-analysis. The important problem of MVDR-like beamforming for a large array in a dynamic environment is used to bring out specific problems and illustrate corresponding engineering approaches, for which analytical results have been unavailable. The non-stationarity of the background forces a compromise between a few nearly stationary snapshots and many snapshots during which the eigen-structure has evolved. This frequently leads to a snapshot starved situation. Topics discussed include: rules of thumb for sample support, rank deficient and ill-conditioned matrices, isotropic noise rank, diagonal loading, sensitivity, white noise gain constraint, eigen-vector base projections, inbred vs. non-inbred processing, and transformations to reduce degrees of freedom and rank. The goal is to provide background and motivation for future research in statistical eigen-analysis.
The Probability Distribution of the MVDR Beamformer Outputs under Diagonal Loading
N.Raj Rao and Alan Edelman
Massachusetts Institute of Technology
Email: [raj, edelman]@mit.edu
The MVDR beamformer is the most extensively used array processing algorithm and involves inverting the sample covariance matrix. In the snapshot deficient scenario, when the number of sensors is greater than or approximately equal to the number of snapshots, the eigenvalues of the resulting sample covariance matrix are poorly conditioned. Diagonal loading is then applied to the sample covariance matrix. Very little is known [1] about the performance of the MVDR beamformer under diagonal loading mainly because the probability distribution of the outputs is unknown.