Talk title: Is Model Compression Always Harmful to the Performance of Neural Networks?

Date & time: January 22^nd, 2022 Saturday, 4pm-5pm PST

Venue: Zoom (Eventbrite registration required)

Eventbrite link: tinyurl.com/cie-sf-0122

Host: CIE-SF

Speaker: Qing Jin

Abstract:

The great success of deep learning is accompanied by exploding model size and computational cost of deep models. As an example, GPT-3 requires 175 billion parameters. This makes model compression and acceleration critical for application of deep neural networks on resource-constrained and speed-demanding platforms, especially for those with limited energy and stringent memory budget, such as DSP or IoT devices. In this talk, we will talk about modern techniques to compress deep neural networks, including model quantization for image classification task and network pruning for generative models. We will talk about conventional methods of neural network quantization and their limitations, followed by more recent progress, including advanced methods of adaptive bit-widths and mixed-precision quantization. After that, we discuss network pruning for generative models, which are typically much larger and more difficult to compress in comparison with those for image classification. Finally, we will introduce knowledge distillation which is a powerful method to enhance the performance of small-size models. We hope this will give a basic understanding of the compression techniques for deep models. 

Speaker Bio:

Qing Jin is currently a PhD student in Northeastern University working on deep learning acceleration and software-hardware co-design for innovative computing systems. Qing obtained his Bachelor and Master degrees from Nankai University, both in Microelectronics, and Master degree in Computer Engineering from Texas A&M University. He has been working on RF integrated circuit design as well as low-power analog circuit design in the early years, and work on computer vision and deep learning algorithms in recent days. He has been a research intern in several companies including ByteDance, SenseBrain, Kwai and Snap, working on model compression, neural network quantization, and neural architecture search (NAS) for deep neural networks. His work has been published in top-tier conferences and journals in a broad area including computer vision, machine learning, as well as solid-state circuits, such as CVPR, ICML, NeurIPS, AAAI, BMVC, CICC, ISLPED, and JSSC. He is a co-recipient of Best Student Paper Award Finalist of IEEE Custom Integrated Circuits Conference (CICC). He is interested in both practical design and theoretical analysis related to deep learning.

美洲中國工程師學會高中生獎學金

Chinese Institute of Engineers, San Francisco Bay Area Chapter (CIE/USA-SF) is now accepting applications for the Scholarship Award. The scholarship award winners will be awarded US $200 dollars and a granted CIE scholarship certificate, presented at the CIE/USA-SF Annual Conference in 2022 in May 14 (Saturday), at Santa Clara Convention Center.

Being the oldest Chinese engineering and science society in North America, and following a long-standing tradition of pursuing excellence, CIE/USA-SF has established this award program to recognize and encourage the outstanding high school students of its members. The selection criteria will be based on an overall accomplishment of applicants' academic performance, extracurricular activities achievement and the participation in community services. Each award winner will receive a Certificate of Merit and a check of two hundred dollars and shall need to attend the CIE/SF Annual Conference to receive it.

All applicants must meet the following qualifications:

1.     A CIE/USA-SF member's children

2.     Currently a Junior or Senior (11th or 12th grade) in high school 

3.     Has never received the award from the CIE/SF Chapter before

4.     Must be able to attend the award ceremony to receive the award and certificate

All applicants must submit the following documents in soft copies:

1.     A completed application form (download template).

2.     A passport type photo (> 300kb), to be published in the CIE/USA-SF 2022 annual conference proceedings 

3.     A high school Transcript

4.     A personal essay (no more than 250 words) which includes information such as: honors and awards received, offices of student organizations held, extracurricular activities, community service involved, volunteer work experiences, etc.

5.     (For Non-CIE/SF parents only) Join membership online from HERE.    

The deadline for the application is March 15th, 2022.  

For 12th grade student, please send in your application ASAP for possible early decision. 

Contact Person: T.C. Yang

Application material please email to This email address is being protected from spambots. You need JavaScript enabled to view it.