Semester

Spring

Date of Graduation

2021

Document Type

Dissertation

Degree Type

PhD

College

Statler College of Engineering and Mineral Resources

Department

Lane Department of Computer Science and Electrical Engineering

Committee Chair

Nasser M. Nasrabadi

Committee Member

Jeremy Dawson

Committee Member

Powsiri Klinkhachorn

Committee Member

Xin Li

Committee Member

Keith Morris

Committee Member

Matthew C. Valenti

Abstract

Deep models have provided high accuracy for different applications such as person recognition, image segmentation, image captioning, scene description, and action recognition. In this dissertation, we study the deep learning models and their application in improving the performance and reliability of person recognition. This dissertation focuses on five aspects of person recognition: (1) multimodal person recognition, (2) quality-aware multi-sample person recognition, (3) text-independent speaker verification, (4) adversarial iris examples, and (5) morphed face images. First, we discuss the application of multimodal networks consisting of face, iris, fingerprint, and speech modalities in person recognition. We propose multi-stream convolutional neural network architectures to incorporate person recognition traits introducing three multimodal frameworks: multi-level abstraction, generalized compact bilinear pooling, and quality-aware multi-sample multimodal fusion. Then, a novel cross-device text-independent speaker verification architecture which consists of spectro-temporal and prosodic features is introduced. Through intensive experimental setups the performance of each proposed framework is studied.

Although biometric recognition systems are fast becoming part of security applications, these systems are still vulnerable to image manipulations. To study the reliability of deep models in person recognition, we focus on adversarial examples and morphed images. We introduce adversarial examples for iris recognition framework with non-targeted and targeted attacks and study the possibility of fooling an iris recognition system in white-box and black-box frameworks.Then, we present defense strategies to detect adversarial iris examples. These defense strategies are based on wavelet domain denoising of the input examples by investigating each wavelet sub-band. Finally, we study the morphed face images in which a facial reference image can be verified as two or more separate identities. Here, a novel differential morph attack detection framework using a deep Siamese network is proposed. Then, we improve the performance utilizing landmark and appearance disentanglement through contrastive representations.

Embargo Reason

Publication Pending

Download

Share

COinS