Semester
Spring
Date of Graduation
2021
Document Type
Dissertation
Degree Type
PhD
College
Statler College of Engineering and Mineral Resources
Department
Lane Department of Computer Science and Electrical Engineering
Committee Chair
Nasser M. Nasrabadi
Committee Member
Jeremy Dawson
Committee Member
Powsiri Klinkhachorn
Committee Member
Xin Li
Committee Member
Keith Morris
Committee Member
Matthew C. Valenti
Abstract
Deep models have provided high accuracy for different applications such as person recognition, image segmentation, image captioning, scene description, and action recognition. In this dissertation, we study the deep learning models and their application in improving the performance and reliability of person recognition. This dissertation focuses on five aspects of person recognition: (1) multimodal person recognition, (2) quality-aware multi-sample person recognition, (3) text-independent speaker verification, (4) adversarial iris examples, and (5) morphed face images. First, we discuss the application of multimodal networks consisting of face, iris, fingerprint, and speech modalities in person recognition. We propose multi-stream convolutional neural network architectures to incorporate person recognition traits introducing three multimodal frameworks: multi-level abstraction, generalized compact bilinear pooling, and quality-aware multi-sample multimodal fusion. Then, a novel cross-device text-independent speaker verification architecture which consists of spectro-temporal and prosodic features is introduced. Through intensive experimental setups the performance of each proposed framework is studied.
Although biometric recognition systems are fast becoming part of security applications, these systems are still vulnerable to image manipulations. To study the reliability of deep models in person recognition, we focus on adversarial examples and morphed images. We introduce adversarial examples for iris recognition framework with non-targeted and targeted attacks and study the possibility of fooling an iris recognition system in white-box and black-box frameworks.Then, we present defense strategies to detect adversarial iris examples. These defense strategies are based on wavelet domain denoising of the input examples by investigating each wavelet sub-band. Finally, we study the morphed face images in which a facial reference image can be verified as two or more separate identities. Here, a novel differential morph attack detection framework using a deep Siamese network is proposed. Then, we improve the performance utilizing landmark and appearance disentanglement through contrastive representations.
Recommended Citation
Soleymani, Sobhan, "Deep Models for Improving the Performance and Reliability of Person Recognition" (2021). Graduate Theses, Dissertations, and Problem Reports. 8028.
https://researchrepository.wvu.edu/etd/8028
Embargo Reason
Publication Pending
Included in
Electrical and Electronics Commons, Signal Processing Commons, Systems and Communications Commons