Efficient Ensemble via Rotation-Based Multi-Input Multi-Output Network
효율적 앙상블을 위한 회전 기반 다중 입출력 신경망
- 주제(키워드) Efficient ensemble , CNN
- 발행기관 고려대학교 대학원
- 지도교수 박세준
- 발행년도 2024
- 학위수여년월 2024. 8
- 학위명 석사
- 학과 및 전공 대학원 인공지능학과
- 원문페이지 54 p
- 실제URI http://www.dcollection.net/handler/korea/000000287923
- UCI I804:11009-000000287923
- DOI 10.23186/korea.000000287923.11009.0001563
- 본문언어 영어
초록/요약
Multi-input multi-output structures are ensemble structures that train several ensemble members in a single network, enhancing performance at a small additional cost compared to a single network. There have been various attempts to develop multi-input multi-output structures; however, incorporating the benefits of self-supervised learning into a multi-input multi-output structure remains an area yet to be explored. In this work, we develop a multi-input multi-output structure by simultaneously learning original and self-supervised tasks, thereby utilizing the benefits of self-supervised learning. More precisely, for multiple inputs, we suggest a new mixing strategy and minibatch structure for a rotation-based self-supervised learning technique, and in terms of multiple outputs, we broaden the label space of multiple classifiers to predict both the original class and true rotation degree. We observe that our approach with wider networks on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets shows improved performance compared to previous works, even with nearly half the number of parameters.
more목차
1 Introduction 1
2 Related Works 4
2.1 Self-supervised learning and variants . . . . . . . . . . . . . . . . . . . . 4
2.2 Implicit deep ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Methodology 7
3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2 Multi-input multi-output . . . . . . . . . . . . . . . . . . . . . . 8
3.1.3 Self-supervised label augmentation . . . . . . . . . . . . . . . . 12
3.2 Proposed Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4 Experiments 22
4.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.2 Main experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.3 Impact of input repetition rate . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.1 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3.2 Diversity between members . . . . . . . . . . . . . . . . . . . . 30
4.4 Other number of multiple inputs/outputs and set of rotation degrees . . . 33
4.5 Other minibatch constructions and mixing strategies . . . . . . . . . . . . 34
4.5.1 Other pairs of rotation degrees . . . . . . . . . . . . . . . . . . . 35
4.5.2 Comparison among mixing strategies . . . . . . . . . . . . . . . 36
5 Conclusion
