DSP alum **Christoph Studer** (postdoc 2010-12) has been awarded tenure at Cornell University. Christoph is an expert in signal processing, communications, machine learning, and their implementation in VLSI circuits. He has received a Swiss National Science Foundation postdoc fellowship, a US NSF CAREER Award, and numerous best paper awards. He is still is not a fan of Blender.

# Author Archives: jkh6

# The Implicit Regularization of Ordinary Least Squares Ensembles

D. LeJeune, H. Javadi, R. G. Baraniuk, "**The Implicit Regularization of Ordinary Least Squares Ensembles**," arxiv.org/abs/1910.04743, 10 October 2019.

** Ensemble methods** that average over a collection of independent predictors that are each limited to a subsampling of both the examples and features of the training data command a significant presence in machine learning, such as the ever-popular random forest, yet the

nature of the **subsampling effect**, particularly of the features, is not well understood. We study the case of an ensemble of linear predictors, where each individual predictor is fit using ordinary least squares on a random submatrix of the data matrix. We show that, under standard Gaussianity assumptions, when the number of features selected for each predictor is optimally tuned, **the asymptotic risk of ****a large ensemble is equal to the asymptotic ridge regression ****risk**, which is known to be optimal among linear predictors in this setting. In addition to eliciting this **implicit regularization **that results from subsampling, we also connect this ensemble to the **dropout** technique used in training deep (neural) networks, another strategy that has been shown to have a ridge-like regularizing effect.

Above: Example (rows) and feature (columns) subsampling of the training data **X** used in the ordinary least squares fit for one member of the ensemble. The *i*-th member of the ensemble is only allowed to predict using its subset of the features (green). It must learn its parameters by performing ordinary least squares using the subsampled examples of (red) and the subsampled examples (rows) and features (columns) of **X** (blue, crosshatched).

# DSP Alum Mark Davenport Awarded PECASE

Mark Davenport, Rice DSP alum and Associate Professor at Georgia Tech, has been awarded a **Presidential Early Career Award for Scientists and Engineers**. The PECASE is the highest honor bestowed by the United States Government to outstanding scientists and engineers who are beginning their independent research careers and who show exceptional promise for leadership in science and technology. Additional awards include a Sloan Research Fellowship in Mathematics and the Class of 1940 W. Roane Beard Outstanding Teacher Award from Georgia Tech. Mark's Georgia Tech colleague and Rice DSP Alum Justin Romberg was awarded the PECASE in 2009.

# DSP50 – 26 April 2019

Rice University

26 April 2019

Rice University has been a major force in DSP research and education since two young faculty members launched their first DSP course some 50 years ago. Sidney Burrus and Tom Parks, together with their colleagues, joined over the years by Rui deFigueredo, Don Johnson and waves of additional faculty, built an internationally recognized program that spawned key theory and algorithms for digital filter design, fast Fourier transforms, array processing, wavelet transforms, compressive sensing, and deep learning. Rice's many outstanding DSP alumni now hold leadership positions in academics, industry, and government.

DSP50 will commemorate and celebrate the past, present, and future of one of the longest-running successful research and education programs at Rice with a variety of speakers, panel sessions, and discussions.

**WHEN:** Friday 26 April 2019

**WHERE:** BioScience Research Collaborative (BRC), Rice University, Houston, Texas

**MORE INFO:** dsp.rice.edu/DSP50

# DSP Alum Eva Dyer Awarded Sloan Fellowship

Rice PhD alum Eva Dyer has been awarded a prestigious Sloan Fellowship to support her research at the Georgia Institute of Technology on new computational methods for discovering principles that govern the organization and structure of the brain. Eva's broader research interests lie at the intersection of machine learning, optimization and neuroscience.

# Four Papers at ICLR 2019

DSP group members will be traveling en masse to New Orleans in May 2019 to present four regular papers at the **International Conference on Learning Representations**

- R. Balestriero and R. G. Baraniuk, “
**Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference**” - J. Wang, R. Balestriero, and R. G. Baraniuk, “
**A Max-Affine Perspective of Recurrent Neural Networks**” - A. Mousavi, G. Dasarathy, and R. G. Baraniuk, “
**A Data-Driven and Distributed Approach to Sparse Signal Representation and Recovery**” - J. J. Michalenko, A. Shah, A. Verma, R. G. Baraniuk, S. Chaudhuri, and A. B. Patel, “
**Representing Formal Languages: A Comparison between Finite Automata and Recurrent Neural Networks**”

# Pavan Kota Awarded NIH NLM Training Grant in Biomedical Informatics and Data Science

**Pavan Kota** has been awarded a prestigious **NIH NLM Training Grant in Biomedical Informatics and Data Science** that will support his PhD research on new kinds of microbial diagnostics. One particularly promising approach is based on designing coded strands of DNA that enable compressive sensing to be applied. The upshot is that only a few coded DNA probes are required to detect and classify a much larger number of potential microbes.

# Chris Metzler Defends PhD Thesis

Chris Metzler successfully defended his PhD thesis entitled *Data-Driven Computational Imaging: With Application to Imaging Through and Around Obstacles*. He's pictured here at Rice's Valhalla pub performing the ritual "trimming of the tie" ceremony for new PhDs with Prof. Richard Baraniuk. Chris' next stop is Stanford University for a postdoc with Prof. Gordon Wetzstein in the Stanford Computational Imaging Lab.

# DSP Alum John Treichler Receives the IEEE SPS Society Award

John Treichler, Rice DSP alum, distinguished TI visiting professor, and pioneer in the development of digital signal processing (DSP), has been awarded the *Society Award* by the IEEE Signal Processing Society. The Society Award honors outstanding technical contributions in a field within the scope of the Signal Processing Society and outstanding leadership within that field. John will present the Norbert Wiener Lecture at ICASSP 2019 in Brighton, UK.

Now the Chief Technology Officer and Senior Scientist of Raytheon Applied Signal Technology, John was cited for his “contributions and leadership in the practical use of adaptive digital signal processing and for sustained service to the Society.” Among his many creations, John is celebrated for inventing the “constant modulus” adaptive filtering algorithm, which is used to compensate for interference, such as multipath echoes, on communication signals.

# Compressive Sensing Resources are Back!

The Compressive Sensing Resources webpage, a collection of over 1000 links to papers and preprints on sub-Nyquist sampling, is available once again on the Rice DSP website. To contribute a new paper, update an existing link, or make a comment, please email CSresourcesRice at gmail dot com.