Neural architecture search (NAS) methods have been developed to automate the design process of deep neural networks (DNN). In this tutorial, we survey the critical elements of NAS methods that – to various extents – consider hardware implementation of the resulting DNNs. We will classify these methods into three major classes: single-objective NAS (no hardware is considered), hardware-aware NAS (DNN is optimized for a particular hardware platform), and NAS with hardware co-optimization (hardware is directly co-optimized with DNN as a part of NAS). We emphasize the multi-objective design approach that must be adopted in NAS and focus on co-design algorithms developed for concurrent optimization of DNN architectures and hardware platforms. As most research in this area deals with NAS for convolutional neural network-based image classifiers, our case studies will be devoted to this application.
Prof. Lukas Sekanina received all his degrees from the Brno University of Technology, Czech Republic (Ing. in 1999, Ph.D. in 2002), where he is currently a full professor and Head of the Department of Computer Systems. In his research, he combines computational intelligence (evolutionary design, neural networks, cellular automata) and hardware design methods to automatically design, optimize, and approximate complex digital circuits and hardware accelerators.
He was awarded the Fulbright scholarship and worked on the evolutionary circuit design with NASA Jet Propulsion Laboratory (Caltech) in Pasadena in 2004. He was a visiting lecturer with Pennsylvania State University (2001), Universidad Politécnica de Madrid (2012), and a visiting researcher with the University of Oslo in 2001.
He has been an Associate Editor of IEEE Transactions on Evolutionary Computation (2011-2014), and Editorial board member of Genetic Programming and Evolvable Machines Journal and International Journal of Innovative Computing and Applications. He served as General Chair of DDECS 2013, Program Co-Chair of DDECS 2021, EuroGP 2018 - 2019, DTIS 2016, ICES 2008, and Topic chair of DATE 2020 - 2022 (Approximate computing). He is a Senior Member of IEEE.