D. LeJeune, H. Javadi, R. G. Baraniuk, "The Implicit Regularization of Ordinary Least Squares Ensembles," arxiv.org/abs/1910.04743, 10 October 2019.

Ensemble methods that average over a collection of independent predictors that are each limited to a subsampling of both the examples and features of the training data command a significant presence in machine learning, such as the ever-popular random forest, yet the
nature of the subsampling effect, particularly of the features, is not well understood.  We study the case of an ensemble of linear predictors, where each individual predictor is fit using ordinary least squares on a random submatrix of the data matrix. We show that, under standard Gaussianity assumptions, when the number of features selected for each predictor is optimally tuned, the asymptotic risk of a large ensemble is equal to the asymptotic ridge regression risk, which is known to be optimal among linear predictors in this setting. In addition to eliciting this implicit regularization that results from subsampling, we also connect this ensemble to the dropout technique used in training deep (neural) networks, another strategy that has been shown to have a ridge-like regularizing effect.

Above: Example (rows) and feature (columns) subsampling of the training data X used in the ordinary least squares fit for one member of the ensemble. The i-th member of the ensemble is only allowed to predict using its subset of the features (green). It must learn its parameters by performing ordinary least squares using the subsampled examples of (red) and the subsampled examples (rows) and features (columns) of X (blue, crosshatched).

Mark Davenport, Rice DSP alum and Associate Professor at Georgia Tech, has been awarded a Presidential Early Career Award for Scientists and Engineers. The PECASE is the highest honor bestowed by the United States Government to outstanding scientists and engineers who are beginning their independent research careers and who show exceptional promise for leadership in science and technology. Additional awards include a Sloan Research Fellowship in Mathematics and the Class of 1940 W. Roane Beard Outstanding Teacher Award from Georgia Tech. Mark's Georgia Tech colleague and Rice DSP Alum Justin Romberg was awarded the PECASE in 2009.

The DSP50 event on April 26th was a great success and we genuinely thank all who made it possible as well as all who were able to join us. DSP50 commemorated and celebrated the past, present, and future of one of the longest-running successful research and education programs at Rice with a variety of speakers, panel sessions, and discussions. If you missed out, but are interested in the videos, they can now be found on the Rice University ECE Youtube Page at the links below.

We are also working on developing a DSP50 commemorative coffee table book, if you're interested in a copy upon completion please fill out this form here.

Welcome and Intro
Genesis and Beyond
Waves of progress
Selection of DSP Impacts
Presentations
The Future of DSP at Rice

Rice University
26 April 2019

Rice University has been a major force in DSP research and education since two young faculty members launched their first DSP course some 50 years ago. Sidney Burrus and Tom Parks, together with their colleagues, joined over the years by Rui deFigueredo, Don Johnson and waves of additional faculty, built an internationally recognized program that spawned key theory and algorithms for digital filter design, fast Fourier transforms, array processing, wavelet transforms, compressive sensing, and deep learning.  Rice's many outstanding DSP alumni now hold leadership positions in academics, industry, and government.

DSP50 will commemorate and celebrate the past, present, and future of one of the longest-running successful research and education programs at Rice with a variety of speakers, panel sessions, and discussions.

WHEN:  Friday 26 April 2019

WHERE:  BioScience Research Collaborative (BRC), Rice University, Houston, Texas

MORE INFO:  dsp.rice.edu/DSP50

Rice PhD alum Eva Dyer has been awarded a prestigious Sloan Fellowship to support her research at the Georgia Institute of Technology on new computational methods for discovering principles that govern the organization and structure of the brain.  Eva's broader research interests lie at the intersection of machine learning, optimization and neuroscience.

DSP group members will be traveling en masse to New Orleans in May 2019 to present four regular papers at the International Conference on Learning Representations

• R. Balestriero and R. G. Baraniuk, “Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference”
• J. Wang, R. Balestriero, and R. G. Baraniuk, “A Max-Affine Perspective of Recurrent Neural Networks”
• A. Mousavi, G. Dasarathy, and R. G. Baraniuk, “A Data-Driven and Distributed Approach to Sparse Signal Representation and Recovery”
• J. J. Michalenko, A. Shah, A. Verma, R. G. Baraniuk, S. Chaudhuri, and A. B. Patel, “Representing Formal Languages: A Comparison between Finite Automata and Recurrent Neural Networks”

Pavan Kota has been awarded a prestigious NIH NLM Training Grant in Biomedical Informatics and Data Science that will support his PhD research on new kinds of microbial diagnostics.  One particularly promising approach is based on designing coded strands of DNA that enable compressive sensing to be applied.  The upshot is that only a few coded DNA probes are required to detect and classify a much larger number of potential microbes.

Chris Metzler successfully defended his PhD thesis entitled Data-Driven Computational Imaging: With Application to Imaging Through and Around Obstacles.  He's pictured here at Rice's Valhalla pub performing the ritual "trimming of the tie" ceremony for new PhDs with Prof. Richard Baraniuk.  Chris' next stop is Stanford University for a postdoc with Prof. Gordon Wetzstein in the Stanford Computational Imaging Lab.

John Treichler, Rice DSP alum, distinguished TI visiting professor, and pioneer in the development of digital signal processing (DSP), has been awarded the Society Award by the IEEE Signal Processing Society.  The Society Award honors outstanding technical contributions in a field within the scope of the Signal Processing Society and outstanding leadership within that field.  John will present the Norbert Wiener Lecture at ICASSP 2019 in Brighton, UK.

Now the Chief Technology Officer and Senior Scientist of Raytheon Applied Signal Technology, John was cited for his “contributions and leadership in the practical use of adaptive digital signal processing and for sustained service to the Society.” Among his many creations, John is celebrated for inventing the “constant modulus” adaptive filtering algorithm, which is used to compensate for interference, such as multipath echoes, on communication signals.

The Compressive Sensing Resources webpage, a collection of over 1000 links to papers and preprints on sub-Nyquist sampling, is available once again on the Rice DSP website. To contribute a new paper, update an existing link, or make a comment, please email CSresourcesRice at gmail dot com.