Ian En-Hsu Yen

My name is Enxu Yan (or Ian En-Hsu Yen). I previously worked as Research Scientist at Microsoft, Google and Snap Research on AI acceleration and optimization, and co-founded an AI acceleration startup company. I got my PhD from the Machine Learning Department of Carnegie Mellon University.

Education

Carnegie Mellon University

• PhD , Machine Learning Department, 2016 - 2018.

University of Texas at Austin

• PhD student, Department of Computer Science, 2013 - 2016

National Taiwan University

• M.S., Department of Computer Science and Information Engineering, 2011 - 2012
• B.S. in Computer Science and B.B.A. in Information Management, 2007 - 2011

[Back to top]

Working Experience

• Chief Scientist and Co-Founder, 2019 - 2025.
  Developing algorithm-hardware co-design solution for AI acceleration.

• Research Scientist, 2018 - 2019.
  Develop model compression toolchains for real-time network inference on mobile platforms.

• Intern Research Scientist, 2017.
  Deep Learning with Large Output Domain (published in ICML 2018) (with Satyen Kale, Felix Yu and Sanjiv Kumar).

• Intern Research Scientist, 2016.
  Convex Relaxation for Deep Learning (with Pushmeet Kohli and Abdelrahman Mohamed).

• Intern Research Scientist, 2015.
  Large-scale Convex Optimization Method for Exemplar Clustering (published in AISTATS 2016) (with Dmitry Malioutov and Abhishek Kumar)

• Intern Data Scientist, 2014.
  Build Sponsored-Product Recommendation System based on Inductive Matrix Completion.

• Research Assistant, 2012 - 2013.
  Develop Multi-Party Distributed Learning algorithm via Indexed Learning thechnique (with Shou-De Lin).

• Intern Data Scientist, 2011 - 2012.
  Develop Recommendation System and Entity Extractors for exatracting organization name and job title from unstructured text.

[Back to top]

Publications

• MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding.
  Ranajoy Sadhukhan, Jian Chen, Zhuoming Chen, Vashisth Tiwari, Ruihang Lai, Jinyuan Shi, Ian E.H. Yen, Avner May, Tianqi Chen, Beidi Chen
  In International Conference on Learning Representations (ICLR), 2025.
• Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm.
  Shaoyi Huang, Dongkuan Xu, Ian E.H. Yen, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Hang Liu, Caiwen Ding.
  In Association for Computational Linguistics (ACL), 2022.
• Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm.
  Dongkuan Xu, Ian E.H. Yen, Jinxi Zhao, Zhibin Xiao.
  In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2021.
• Minimizing FLOPs to Learn Efficient Sparse Representations.
  Biswajit Paria, Chih-Kuan Yeh, Ian E.H. Yen, Ning Xu, Pradeep Ravikumar, Barnabas Poczos.
  In International Conference on Learning Representations (ICLR), 2020.
• Temporal Structure Mining for Weakly Supervised Action Detection.
  Tan Yu, Zhou Ren, Yuncheng Li, Ian E.H. Yen, Ning Xu, Junsong Yuan.
  In International Conference on Computer Vision (ICCV), 2019.
• Efficient Global String Kernel with Random Features: Beyond Counting Substructures.
  Lingfei Wu, Ian En-Hsu Yen, Siyu Huo, Liang Zhao, Kun Xu, Liang Ma, Shouling Ji and Charu Aggarwal.
  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2019.
• Scalable Global Alignment Graph Kernel Using Random Features: From Node Embedding to Graph Embedding.
  Lingfei Wu, Ian En-Hsu Yen, Zhen Zhang, Kun Xu, Liang Zhao, Xi Peng, Yinglong Xia and Charu Aggarwal.
  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2019.

• 2018

• Representer Point Selection for Explaining Deep Neural Networks.
  Chih-Kuan Yeh*, Joon Sik Kim*, Ian E.H. Yen, and Pradeep Ravikumar. (equally contributed)
  In Advances in Neural Information Processing Systems (NeurIPS), 2018.
• MixLasso: Generalized Mixed Regression via Convex Atomic-Norm Regularization.
  Ian E.H. Yen, Wei-Cheng Lee, Kai Zhong, Sung-En Chang, Pradeep Ravikumar and Shou-De Lin.
  In Advances in Neural Information Processing Systems (NeurIPS), 2018.
• Word Mover's Embedding: From Word2Vec to Document Embedding.
  Lingfei Wu, Ian E.H. Yen, Kun Xu, Fangli Xu, Avinash Balakrishnan, Pin-Yu Chen, Pradeep Ravikumar and Michael J. Witbrock.
  In Empirical Methods in Natural Language Processing (EMNLP), 2018.
• Loss Decomposition for Fast Learning in Large Output Spaces.
  Ian E.H. Yen, Satyen Kale, Felix Yu, Daniel Holtmann-Rice, Sanjiv Kumar, and Pradeep Ravikumar.
  In International Conference on Machine Learning (ICML), 2018.
• Scalable Spectral Clustering Using Random Binning Features.
  Lingfei Wu, Pin-Yu Chen, Ian E.H. Yen, Fangli Xu, Yinglong Xia, and Charu Aggarwal.
  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2018.
• Random Warping Series: A Random Features Method for Time-Series Embedding. [pdf]
  Lingfei Wu, Ian E.H. Yen, Jinfeng Yi, Fangli Xu, Qi Lei, and Michael J. Witbrock.
  In International Conference on Artificial Intelligence and Statistics (AISTATS), 2018.

• 2017

• PPDSparse: A Parallel Primal-Dual Sparse Method for Extreme Classification. [pdf] [slide] [poster]
  Ian E.H. Yen, Xiangru Huang, Wei Dai, Pradeep Ravikumar, Inderjit S. Dhillon and Eric P. Xing.
  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017.
• Latent Feature Lasso. [pdf] [slide] [poster] [talk] [code]
  Ian E.H. Yen, Wei-Cheng Li, Sung-En Chang, Arun S. Suggala, Shou-De Lin and Pradeep Ravikumar.
  In International Conference on Machine Learning (ICML), 2017.
• Doubly Greedy Primal-Dual Coordinate Methods for Sparse Empirical Risk Minimization. [pdf]
  Qi Lei, Ian E.H. Yen, Chao-Yuan Wu, Pradeep Ravikumar and Inderjit Dhillon.
  In International Conference on Machine Learning (ICML), 2017.
• Greedy Direction Method of Multiplier for MAP Inference of Large Output Domain. [pdf]
  Xiangru Huang, Ian E.H. Yen, Ruohan Zhang, Qixing Huang, Pradeep Ravikumar, and Inderjit Dhillon.
  In International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
• Scalable Convex Multiple Sequence Alignment via Entropy-Regularized Dual Decomposition. [pdf]
  Jiong Zhang, Ian E.H. Yen, Pradeep Ravikumar, and Inderjit Dhillon.
  In International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.

• 2016

• Dual Decomposed Learning with Factorwise Oracles for Structural SVMs of Large Output Domain. [pdf]
  Ian E.H. Yen, Xiangru Huang, Kai Zhong, Ruohan Zhang, Pradeep Ravikumar and Inderjit S. Dhillon.
  In Advances in Neural Information Processing Systems (NeurIPS), 2016.
• Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability. [pdf]
  Lingfei Wu^*, Ian E.H. Yen^*, Jie Chen and Rui Yan. (* equally contributed)
  In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2016.
• Large-scale Submodular Greedy Exemplar Selection with Structured Similarity Matrices. [pdf]
  Dmitry Malioutov, Abhishek Kumar and Ian E.H. Yen.
  In Conference on Uncertainty in Artificial Intelligence (UAI), 2016.
• PD-Sparse: A Primal and Dual Sparse Approach to Extreme Multiclass and Multilabel Classification. [pdf] [slide] [poster] [code]
  Ian E.H. Yen^*, Xiangru Huang^*, Kai Zhong, Pradeep Ravikumar and Inderjit S. Dhillon. (* equally contributed)
  In International Conference on Machine Learning (ICML), 2016.
• A Convex Atomic-Norm Approach to Multiple Sequence Alignment and Motif Discovery. [pdf] [slide] [poster]
  Ian E.H. Yen^*, Xin Lin^*, Jiong Zhang, Pradeep Ravikumar and Inderjit S. Dhillon. (* equally contributed)
  In International Conference on Machine Learning (ICML), 2016.
• Scalable Exemplar Clustering and Facility Location via Augmented Block Coordinate Descent with Column Generation. [pdf] [code]
  Ian E.H. Yen, Dmitry Malioutov, and Abhishek Kumar.
  In International Conference on Artificial Intelligence and Statistics (AISTATS), 2016.

• 2015

• Sparse Linear Programming via Primal and Dual Augmented Coordinate Descent. [pdf] [slide] [poster] [code]
  Ian E.H. Yen, Kai Zhong, Cho-Jui Hsieh, Pradeep Ravikumar and Inderjit S. Dhillon.
  In Advances in Neural Information Processing Systems (NeurIPS), 2015.
• A Dual-Augmented Block Minimization Framework for Learning with Limited Memory. [pdf] [poster]
  Ian E.H. Yen, Shan-Wei Lin, Shou-De Lin.
  In Advances in Neural Information Processing Systems (NeurIPS), 2015.
• A Convex Exemplar-based Approach to Dirichlet Process Mixture Models. [pdf] [slide]
  Ian E.H. Yen, Jimmy Lin, Kai Zhong, Pradeep Ravikumar and Inderjit S. Dhillon.
  In International Conference on Machine Learning (ICML), 2015.
• Tackling the Achilles Heel of Social Networks: Influence Propagation based Language Model Smoothing. [pdf]
  Rui Yan, Ian E.H. Yen, Cheng-Te Li, Shiqi Zhao and Hu Xiaohua.
  In International World Wide Web Conference (WWW), 2015.

• 2014

• Constant Nullspace Strong Convexity and Fast Convergence of Proximal Methods under High-Dimensional Settings. [pdf] [appendix]
  Ian E.H. Yen, Cho-Jui Hsieh, Pradeep Ravikumar and Inderjit S. Dhillon.
  In Advances in Neural Information Processing Systems (NeurIPS), 2014.
• Sparse Random Feature as Coordinate Descent in Hilbert Space. [pdf] [appendix]
  Ian E.H. Yen, Ting-Wei Lin, Shou-De Lin, Pradeep Ravikumar and Inderjit S. Dhillon.
  In Advances in Neural Information Processing Systems (NeurIPS), 2014.
• Proximal Quasi-Newton for Computationally Intensive L1-regularized M-estimators. [pdf] [appendix] [code]
  Kai Zhong, Ian E.H. Yen, Inderjit S. Dhillon and Pradeep Ravikumar.
  In Advances in Neural Information Processing Systems (NeurIPS), 2014.
• Optimal Tests of Treatment Effects for the Overall Population and Two Subpopulations in Randomized Trials, using Sparse Linear Programming. [pdf]
  Michael Rosenblum, Han Liu, Ian E.H. Yen
  Journal of American Statistical Association (JASA) (Theory and Methodology), 2014.

• Before 2014

• Indexed Block Coordinate Descent for Large-Scale Linear Classification with Limited Memory. [pdf] [slide] [code]
  Ian E.H. Yen, Chun-Fu Chang, Ting-Wei Lin, Shan-Wei Lin, Shou-De Lin.
  In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2013.
• On Convergence Rate of Concave-Convex Procedure. [pdf] [slide]
  Ian E.H. Yen, Nanyun Peng, Po-Wei Wang and Shou-de Lin
  In NIPS Optimization for Machine Learning Workshop, 2012.
• Presenting Time-Evolving Activities Using Communication Archive Data. [pdf]
  Sheng-Jie Luo, Shih-Wen Huang, En-Hsu Yen, Li-Ting Huang, Bing-Yu Chen, Kwan-Liu Ma.
  In ACM SIGGRAPH ASIA (poster), 2011.
• Feature engineering and classfier ensemble for KDD Cup 2010. [pdf]
  H.F. Yu, H.Y. Lo, H.P Hsieh, J.K. Lou, T.G. McKenzie, J.W. Chou, P.H. Chung, C.H. Ho, C.F. Chang, Y.H. Wei, J.Y. Weng, En-Hsu Yen, C.W. Chang, T.T. Kuo, Y.C. Lo, P.T. Chang, C. Po, C.Y. Wang, Y.H. Huang, C.W. Hung, Y.X. Ruan, Y.S. Lin, S.D. Lin, H.T Lin, C.J. Lin.
  In Proceedings of the KDD Cup 2010 Workshop, July 2010. First-place winner report of KDD Cup 2010..

[Back to top]

Softwares

• PD-Sparse
  Extreme Multiclass and Multilabel solver based on Dual Block-Coordinate Frank-Wolfe (Dual-BCFW) algorithm, which can solve problem of 10⁴ ~ 10⁶ classes/labels with single core in one day (published in ICML 2016).
• MixingSDPSolve
  A Boolean (0/1) Quadratic Programming solver employing the algorithm proposed in Mixing Method for solving large-scale SDP, which can handle problems of millions of variables.
• ConvexExemplarClustering
  A scalable solver for Exemplar Clustering via convex relaxation, which can scale to a million of samples with sparse or low-rank feature matrix. (published in AISTAT 2016).
• LPsparse
  General-purpose, large-scale Linear Programming solver based on Augmented Lagrangian and Primal/Dual Coordinate Descent, which can be orders-of-magnitude faster than conventional commercial solver for sparse constraint matrix and moderate precision. (published in NeurIPS 2015).
• CRFsparse
  A L1-regularized CRF (Conditional Random Field) solver based on Proximal Quasi-Newton Algorithm (published in NeurIPS 2014).
• IndexSVM
  A scalable solver for Truncated-loss Linear SVM, where pre-built Nearest Neighbor index is used to search coordinates with large gradient. (published in KDD 2013)
• RPGM
  An educational-purpose tool for learning/inference of relational Bayesian Network / Markov Random Field. It has been used in a Probabilistic Graphical Model course to realize Hierchical HMM and Texture-Synthesis MRF through schema.

[Back to top]

Awards

• Best Reviewer Award from NeurIPS 2017.
• First prize of $25k from CMU-Citadel Datathon, an one-day data analysis competition.
• First prize of dataHACK UT challenge, a 12 hour predictive modeling competition co-hosted by the MSBA Program and USAA.
• Second prize of Body Sensor Network analysis competition held in conjunction with BSN 2012, London, UK.
• First prize of KDD Cup 2011, as a member in NTU team. The challenge is about Recommendation of Music Items.
• First prize of KDD Cup 2010, as a member in NTU team. The challenge is about Educational Data Mining.
• Presidential Awards, National Taiwan University (2008-2010).

[Back to top]

Teaching/Tutorial Experience

Tutorial Talks

• Convex Optimization Algorithms for Machine Learning, Summer 2015 at IBM T.J.Watson Research Center. [Slide]
• Support Vector Machine and Convex Optimization, Summer 2012 at National Taiwan University. [Slide]
• Variational Approximate Inference, Summer 2012 at National Taiwan University. [Slide]
• Introduction to Probabilistic Graphical Model, Summer 2011 at National Taiwan University. [Slide]
• Exact Inference and Clique Tree Algorithm in Graphical Model, Fall 2010 at National Taiwan University. [Slide]
• Particle-based Approximate Inference in Graphical Model, Fall 2010 at National Taiwan University. [Slide]

Teaching Assistant

• Software Engineering, Fall 2015 and Summer 2016.
• Statistical Learning and Data Mining, Spring 2015.
• Machine Learning, Spring 2014.
• Graphical Model, Fall 2013.
• Probabilistic Graphical Model, Fall 2012.
• Statistical Methods for Intelligent Information Processing, Fall 2010.
• Game Theory with Application to Finance and Marketing, Fall 2010.

[Back to top]