Librería Portfolio

A realistic and comprehensive review of joint approaches to machine learning and signal processing algorithms, with application to communications, multimedia, and biomedical engineering systems

Digital Signal Processing with Kernel Methods reviews the milestones in the mixing of classical digital signal processing models and advanced kernel machines statistical learning tools. It explains the fundamental concepts from both fields of machine learning and signal processing so that readers can quickly get up to speed in order to begin developing the concepts and application software in their own research.

Digital Signal Processing with Kernel Methods provides a comprehensive overview of kernel methods in signal processing, without restriction to any application field. It also offers example applications and detailed benchmarking experiments with real and synthetic datasets throughout. Readers can find further worked examples with Matlab source code on a website developed by the authors.

Presents the necessary basic ideas from both digital signal processing and machine learning concepts
Reviews the state-of-the-art in SVM algorithms for classification and detection problems in the context of signal processing
Surveys advances in kernel signal processing beyond SVM algorithms to present other highly relevant kernel methods for digital signal processing
An excellent book for signal processing researchers and practitioners, Digital Signal Processing with Kernel Methods will also appeal to those involved in machine learning and pattern recognition.



About the Authors xiii

Preface xvii

Acknowledgements xxi

List of Abbreviations xxiii

Part I Fundamentals and Basic Elements 1

1 From Signal Processing to Machine Learning 3

1.1 A New Science is Born: Signal Processing 3

1.1.1 Signal Processing Before Being Coined 3

1.1.2 1948: Birth of the Information Age 4

1.1.3 1950s: Audio Engineering Catalyzes Signal Processing 4

1.2 From Analog to Digital Signal Processing 5

1.2.1 1960s: Digital Signal Processing Begins 5

1.2.2 1970s: Digital Signal Processing Becomes Popular 6

1.2.3 1980s: Silicon Meets Digital Signal Processing 6

1.3 Digital Signal Processing Meets Machine Learning 7

1.3.1 1990s: New Application Areas 7

1.3.2 1990s: Neural Networks, Fuzzy Logic, and Genetic Optimization 7

1.4 Recent Machine Learning in Digital Signal Processing 8

1.4.1 Traditional Signal Assumptions Are No Longer Valid 8

1.4.2 Encoding Prior Knowledge 8

1.4.3 Learning and Knowledge from Data 9

1.4.4 From Machine Learning to Digital Signal Processing 9

1.4.5 From Digital Signal Processing to Machine Learning 10

2 Introduction to Digital Signal Processing 13

2.1 Outline of the Signal Processing Field 13

2.1.1 Fundamentals on Signals and Systems 14

2.1.2 Digital Filtering 21

2.1.3 Spectral Analysis 24

2.1.4 Deconvolution 28

2.1.5 Interpolation 30

2.1.6 System Identi?cation 31

2.1.7 Blind Source Separation 36

2.2.3 Sparsity, Compressed Sensing, and Dictionary Learning 44

2.3 Multidimensional Signals and Systems 48

2.3.1 Multidimensional Signals 49

2.3.2 Multidimensional Systems 51

2.4 Spectral Analysis on Manifolds 52

2.4.1 Theoretical Fundamentals 52

2.4.2 Laplacian Matrices 54

2.5 Tutorials and Application Examples 57

2.5.1 Real and Complex Signal Processing and Representations 57

2.5.2 Convolution, Fourier Transform, and Spectrum 63

2.5.3 Continuous-Time Signals and Systems 67

2.5.4 Filtering Cardiac Signals 70

2.5.5 Nonparametric Spectrum Estimation 74

2.5.6 Parametric Spectrum Estimation 77

2.5.7 Source Separation 81

2.5.8 Time-Frequency Representations and Wavelets 84

2.5.9 Examples for Spectral Analysis on Manifolds 87

2.6 Questions and Problems 94

3 Signal Processing Models 97

3.1 Introduction 97

3.2 Vector Spaces, Basis, and Signal Models 98

3.2.1 Basic Operations for Vectors 98

3.2.2 Vector Spaces 100

3.2.3 Hilbert Spaces 101

3.2.4 Signal Models 102

3.2.5 Complex Signal Models 104

3.2.6 Standard Noise Models in Digital Signal Processing 105

3.2.7 The Role of the Cost Function 107

3.2.8 The Role of the Regularizer 109

3.3 Digital Signal Processing Models 111

3.3.1 Sinusoidal Signal Models 112

3.3.2 System Identi?cation Signal Models 113

3.3.3 Sinc Interpolation Models 116

3.3.4 Sparse Deconvolution 120

3.3.5 Array Processing 121

3.4 Tutorials and Application Examples 122

3.4.1 Examples of Noise Models 123

3.4.2 Autoregressive Exogenous System Identi?cation Models 132

3.4.3 Nonlinear System Identi?cation Using Volterra Models 138

3.4.4 Sinusoidal Signal Models 140

3.4.5 Sinc-based Interpolation 144

3.4.6 Sparse Deconvolution 152

3.4.7 Array Processing 157

3.5 Questions and Problems 160

3.A MATLABsimpleInterp Toolbox Structure 161

4 Kernel Functions and Reproducing Kernel Hilbert Spaces 165

4.1 Introduction 165

4.2 Kernel Functions and Mappings 169

4.2.1 Measuring Similarity with Kernels 169

4.2.2 Positive-De?nite Kernels 169

4.2.3 Reproducing Kernel in Hilbert Space and Reproducing Property 170

4.2.4 Mercer's Theorem 173

4.3 Kernel Properties 174

4.3.1 Tikhonov's Regularization 175

4.3.2 Representer Theorem and Regularization Properties 176

4.3.3 Basic Operations with Kernels 178

4.4 Constructing Kernel Functions 179

4.4.1 Standard Kernels 179

4.4.2 Properties of Kernels 180

4.4.3 Engineering Signal Processing Kernels 181

4.5 Complex Reproducing Kernel in Hilbert Spaces 184

4.6 Support Vector Machine Elements for Regression and Estimation 186

4.6.1 Support Vector Regression Signal Model and Cost Function 186

4.6.2 Minimizing Functional 187

4.7 Tutorials and Application Examples 191

4.7.1 Kernel Calculations and Kernel Matrices 191

4.7.2 Basic Operations with Kernels 194

4.7.3 Constructing Kernels 197

4.7.4 Complex Kernels 199

4.7.5 Application Example for Support Vector Regression Elements 202

4.8 Concluding Remarks 205

4.9 Questions and Problems 205

Part II Function Approximation and Adaptive Filtering 209

5 A Support Vector Machine Signal Estimation Framework 211

5.1 Introduction 211

5.2 A Framework for Support Vector Machine Signal Estimation 213

5.3 Primal Signal Models for Support Vector Machine Signal Processing 216

5.3.1 Nonparametric Spectrum and System Identi?cation 218

5.3.2 Orthogonal Frequency Division Multiplexing Digital Communications 220

5.3.3 Convolutional Signal Models 222

5.3.4 Array Processing 225

5.4 Tutorials and Application Examples 227

5.4.1 Nonparametric Spectral Analysis with Primal Signal Models 227

5.4.2 System Identi?cation with Primal Signal Model ??-?lter 228

5.4.3 Parametric Spectral Density Estimation with Primal Signal Models 230

5.4.4 Temporal Reference Array Processing with Primal Signal Models 231

5.