Projects

  • Facebook Habitat AI challenge

    • Implemented imitation learning algorithm namely behavioral cloning for point goal navigation task in a photo-realistic, indoor environment without GPS and compass information. Developed a benchmark based on RNN for preserving the temporal information of trajectories and predicting optimal actions given the state of an embodied agent with nearly 60% accuracy. Compared performance with a reinforcement learning baseline (Proximal Policy Optimization).

  • Visual Question Answering

    • Researched and designed VQA models that leverages both visual and textual cues to answer questions about an image. Developed and tested models for questions with binary, numeric and multiple-choice answers. Used a novel fusion strategy along with parallel and alternate co-attention models on the VQA 2.0 dataset.

  • Visual Relationship Detection

    • Extracted visual relationships between objects in an image to get a sense of the overall scene semantics through few-shot learning approach and triplet loss. Trained CNN models with bounding box masks for each positive and negative instance of a visual relationship along with glove embeddings for the relationship predicate. Achieved an accuracy and precision of 70% and recall of 65%.

  • Semantic Segmentation on Antarctic Landsat-8 Imagery

    • Extracted rock-outcrop through semantic segmentation of Landsat-8 satellite imagery. Employed various data processing techniques for hyperspectral, multi-band images. Developed and trained deep learning models to study ice sheet depletion and effects of global warming in Antarctica. Carried out a comprehensive analysis of factors (sun elevation, cloud cover etc.) responsible for differentiation of ice from rock pixels and their effects on the results obtained.

  • Genre-specific Lyrics Generation

    • Automated the process of generating lyrics specific to ten different genres from Million Songs dataset. Developed machine learning models to predict their popularity. Employed RNN model and different clustering techniques for lyrics generation. Achieved a BLEU score of 0.3.

  • Contextual Sentiment Analysis of Tweets

    • Implemented SentiCircle approach for capturing contextual semantics. Employed a novel aggregation technique to predict sentiment orientation of tweets. Carried out a comparative analysis between different distance measures used to create SentiCirles. Achieved 5-6% improvement in sentiment classification over a well-established baseline.

  • Predicting Click Through Rate (CTR) of Online Advertisements

    • Employed a soft-clustering technique for determining CTR of advertisements, thereby enhancing quality of ad recommendations and user experience.