Samsung Research

Seoul, South Korea
Computer Vision Engineer, Next Generation Digital Appliances Research Division
2023 - Present

Key Achievements:

• AI Model Development:
  Engineered and deployed vision-based AI models for Samsung’s next-generation smart refrigerators, reaching millions of users worldwide. Utilized TensorFlow, PyTorch, and OpenCV to create systems that revolutionize user experience through automated food identification and intelligent inventory management.

• Core Technical Expertise:
  Specialized in developing advanced computer vision capabilities including:
  • Real-time object detection and instance segmentation
  • Feature extraction and matching algorithms
  • Multi-class classification systems
  • Temporal trajectory analysis
  • Precise position detection
  • Gaze prediction models
  • Human activity recognition
  • Label detection with geometric transformation and matching

• Out-Of-Distribution (OOD) Detection:
  Tackled critical OOD detection challenges by implementing innovative approaches using openset loss functions, contrastive learning techniques, and ensemble methodologies. These improvements boosted model accuracy by 4% while significantly reducing misclassifications and enhancing reliability across diverse real-world conditions.

• Neural Processing Unit Optimization:
  Optimized complex AI models for efficient NPU deployment, achieving exceptional performance metrics:
  • <40MB model size (enabling on-device deployment)
  • <70ms inference latency (ensuring real-time user experience)
  • >90% accuracy (maintaining high reliability)
    These optimizations ensured seamless integration into refrigerator systems while preserving both performance and user privacy.

Product Videos:

Samsung AI Inside Refrigerator

Bespoke 4-Door Refrigerator

AI Features Demonstration

CES 2024 Showcase

Lessons Learned from Applying On-Device AI Models in Refrigerators

1. Strategic Service Planning for On-Device AI

Our project began with implementing food recognition in refrigerators to provide instantaneous feedback to users. We initially deployed comprehensive models but quickly realized the need for optimization. By strategically separating AI models by function and redesigning the system architecture from first principles, we significantly reduced computational costs while enabling additional features.

This approach delivered two critical advantages: enhanced user experience through millisecond-level response times and product differentiation through capabilities competitors couldn’t match with cloud-dependent solutions.

2. Mastering NPU Optimization

Working with Neural Processing Units (NPUs) presented unique challenges that required deep technical expertise. Through iterative refinement, we developed specialized techniques for model optimization including:

• Quantization to reduce model size while preserving accuracy
• Knowledge Distillation to transfer capabilities from larger to smaller models
• Architecture modifications to accommodate NPU-specific constraints

For layers not natively supported by the NPU, we implemented custom solutions to maintain performance. This process demanded a comprehensive understanding of each network component, ultimately expanding our technical capabilities and expertise.

3. Data Collection Strategies for Appliance AI

We discovered that model performance improved dramatically when we refined our data processing approach. Despite resource constraints, we implemented strategic data collection protocols to maximize efficiency. This included:

• Distributed collection systems across regional offices
• Targeted annotation prioritization
• Continuous data quality improvement

Our experience highlighted that thoughtful data strategy is as important as model architecture for real-world AI applications.

4. Cross-Functional Collaboration

Developing AI for home appliances required seamless coordination across multiple disciplines. Hardware teams (mechanics, circuits), software teams (applications, servers), UX designers, product planners, quality assurance, marketing, sales, legal, and patent specialists all contributed essential expertise.

Through intensive collaboration sessions and cross-functional problem-solving, we aligned diverse perspectives to create a cohesive product. This experience enhanced not only my technical skills but also my project management capabilities.

5. Pioneering New User Experiences

Our real-time On-Device Food Recognition feature represents a first-of-its-kind innovation not yet matched by competitors. While the initial release required extensive experimentation, we’ve seen steadily increasing daily active user metrics. Our global monitoring system allows us to analyze usage patterns, identify issues, and continuously improve the service.

Our vision is to establish cameras as standard components in all refrigerators, creating AI services that seamlessly integrate into users’ daily lives while delivering meaningful convenience.

Software Engineer, Data Service Lab

January 2022 - January 2023

Key Achievements:

• Samsung Health Research Platform Development:
  Architected and deployed the Samsung Health Research Application, creating a bridge between consumer technology and clinical research. This platform enables comprehensive participant life-log data analysis at Samsung Medical Center and partner hospitals, facilitating groundbreaking health studies. Implemented Clean Architecture principles to ensure scalability, maintainability, and future extensibility.

• Full-Stack Development:
  • Engineered the client-side application using Kotlin and the Android Framework, creating an intuitive interface that simplifies complex data collection while ensuring participant privacy and compliance with medical research standards.
  • Developed robust server-side infrastructure using Node.js, implementing secure data handling protocols, efficient storage solutions, and seamless integration with existing medical research systems.
• Research Data Visualization:
  Created a comprehensive data visualization dashboard using Apache Superset, transforming complex health metrics into accessible insights for medical researchers. This system enables intuitive exploration of multidimensional health data, accelerating the research process and facilitating discovery of meaningful patterns. Worked directly with medical researchers to optimize the platform for clinical workflow integration.

Recognition and Impact:

Official Press Release Product Launch Announcement UX Design Award 2023

Developer Resources:

Samsung Health SDK Documentation

Data Science Engineer, Data Intelligence Lab

October 2020 - January 2022

Key Achievements:

• Large-Scale Health Data Analysis:
  Led comprehensive data analysis initiatives on Samsung Health’s 200+ million user database, developing sophisticated models to analyze weight fluctuations, BMI trends, sleep patterns, exercise habits, and daily activity metrics. Utilized advanced cloud technologies including AWS EMR, Athena, Spark, Glue, and BigQuery to process and derive actionable insights from this massive dataset.

• Data Pipeline Engineering:
  Designed and implemented secure pipelines for decryption and analysis of 300+ terabytes of anonymized health data stored on AWS S3. Conducted in-depth trend analysis segmented by demographics and contextual factors such as COVID-19 lockdowns, revealing critical insights into global health behavior patterns and correlations.

• Health Marker Correlation Analysis:
  Established statistically significant relationships between physical and mental health markers using advanced statistical methodologies and time series forecasting. This work created the foundation for personalized health interventions based on multidimensional factors including sleep quality, exercise intensity, daily activity levels, demographic variables, device usage patterns, and geographic location.

• Technical Expertise:

  • Advanced SQL for complex health data queries
  • AWS ecosystem (EMR, Glue, S3, Athena, Spark, Hadoop)
  • Cloud security and IAM implementation
  • Google Cloud (BigQuery, Cloud Storage)
  • PySpark for distributed data processing
  • Scientific visualization (Matlab, Matplotlib, Seaborn)
  • Statistical analysis and significance testing
  • Time series modeling and A/B testing
  • Explainable AI and deep learning for temporal data

Research Visualizations:

Publications and Media Coverage:

Global Sleep Health Study Press Release

Comprehensive Sleep Analysis Findings

News Feature on Sleep Research

Expert Interview: Technology and Sleep Optimization

Technical Portfolio at Samsung

Cloud Infrastructure & Data Engineering

• AWS ML Pipeline Architecture
  • Designed end-to-end machine learning pipelines for production deployment
  • Implemented security best practices for sensitive health data
  • Optimized resource utilization and cost efficiency
• Samsung Health Data Processing
  • Engineered decryption workflows for petabyte-scale health datasets
  • Optimized data formats (Avro, Parquet) for analytical efficiency
  • Created high-performance query systems using Glue, Athena, and EMR

Federated Learning Systems

• On-Device Training Framework
  • Implemented privacy-preserving federated learning architecture
  • Integrated NNTrainer and NNStreamer for efficient model updates
  • Developed communication protocols for secure model aggregation
• PYREC Recommendation Engine
  • Developed state-of-the-art recommendation algorithms
  • Optimized for on-device deployment with minimal resource consumption
  • Implemented personalization features while preserving privacy

Edge Computing & Optimization

• Edge NNTrainer Implementation
  • Conducted experiments on Google Cloud infrastructure
  • Optimized neural network performance for edge devices
  • Reduced inference latency while maintaining accuracy
• Reinforcement Learning Applications
  • Applied RL techniques to optimization problems
  • Implemented algorithms from Berkeley CS188 and Alberta coursework
  • Created adaptive systems for dynamic environments

Mobile & Wearable Development

• Android Application Development
  • Built applications using Kotlin and Jetpack Compose
  • Implemented Firebase Cloud Messaging for notification systems
  • Created responsive interfaces for diverse device form factors
• Healthcare Research Applications
  • Developed specialized applications for medical research
  • Implemented secure data collection and transmission protocols
  • Designed interfaces optimized for clinical environments
• System Integration
  • Utilized OCMS, Privilege SDK, and Samsung Health SDK
  • Implemented RoomDB for efficient data management
  • Designed background communication protocols for continuous monitoring

Recommender Systems

• PYREC Package Development
  • Contributed to Samsung’s recommendation engine framework
  • Implemented and optimized state-of-the-art algorithms
  • Developed evaluation metrics for recommendation quality

System Engineering

• FreeBSD Kernel Management
  • Optimized kernel parameters for performance
  • Implemented FIRE time graph visualizations
  • Enhanced system stability for production environments

Health Analytics Research

• Samsung Health Analysis
  • Contributed to scientific publications and white papers
  • Developed specialized metrics for sleep quality assessment
  • Created user behavior and churn prediction models
  • Conducted pre/post COVID-19 comparative analysis
  • Designed and implemented A/B testing frameworks for feature evaluation

Unbinding Bodies

Unbinding Bodies explores the complex intersections between human bodies, technology, consciousness, and perception. The exhibition examines the transition from closed to open systems, challenging traditional Western metaphysical frameworks.

Visitors experience their own senses oscillating between excess and deficiency as they interact with technology through the installation (2024). The exhibition also features <≈> (2024), a performance work that traces relational encounters between diverse heterogeneous subjects.

Main Artist: Hwia Kim

Website: Unbinding Bodies

Hosted by: Ministry of Culture, Sports, and Tourism

Organized by: Korea Arts Management Service, Art Korea Lab

Technical Contribution

I designed and implemented the end-to-end Computer Vision software powering the exhibition. The system tracks visitors’ faces in real-time, controlling robot arm movements to create an interactive experience. I also developed an algorithm that visualizes how AI “sees” humans, creating a thought-provoking layer of technological interpretation.

Exhibition Highlights

Development Process

Harvard University, School of Engineering and Applied Sciences

Massachusetts, USA (Remote)
Research Intern, Visual Computing Group
March 2020 - August 2020

Research Contributions:

• Deep Learning for Computer Graphics: Enhanced the integration of deep learning with computer graphics by optimizing the PyRedner differentiable rendering library. This work improved inverse rendering techniques, enabling more accurate 3D scene reconstruction from 2D images and advancing the state-of-the-art in graphics-AI integration.

• GPU-Accelerated Monte Carlo Simulation: Developed a sophisticated Monte Carlo differentiable graphics simulation tool with GPU acceleration, significantly improving computational efficiency for real-time applications. This system enabled more realistic physics-based rendering while maintaining the differentiability needed for deep learning optimization.

Technical Impact:

The optimized PyRedner library and simulation tools developed during this internship have been integrated into Harvard’s visual computing research pipeline, supporting ongoing work in computer vision, robotics, and computational photography. The GPU acceleration techniques implemented resulted in significant performance improvements for rendering tasks.

Key Skills & Technologies:

• Deep Learning Frameworks: PyTorch, TensorFlow
• Computer Graphics: Differentiable Rendering, Inverse Graphics
• Simulation: Monte Carlo Methods, Physics-Based Rendering
• GPU Programming: CUDA Optimization, Parallel Computing
• Robotics Integration: ROS, Baxter Robot Control Systems

Samsung Research, Data Analytics Lab

Seoul, South Korea
Research Intern
May 2019 - July 2019

Project Achievements:

• Business Intelligence System: Developed a comprehensive system for extracting actionable semantic information from Business-to-Consumer (B2C) email communications, enabling data-driven insights across multiple business verticals and improving customer relationship management.

• Advanced Email Processing Architecture: Reverse-engineered and enhanced Gmail’s Juicer system architecture, addressing its limitations in data constraints and generalizability. Created a more flexible framework capable of handling diverse email formats and content types.

• Multilingual NLP Pipeline: Designed a sophisticated architecture for processing emails in multiple languages and formats, successfully handling both structured data elements and informal conversational text through a unified processing pipeline.

• Integrated NLP Toolkit: Leveraged and extended advanced natural language processing tools including SpaCy, KlonPY, NLTK, OpenIE, and StanfordIE to create an adaptive model capable of extracting meaningful information from diverse communication contexts.

• Domain-Specific Optimization: Implemented custom post-processing heuristics that significantly enhanced field extraction accuracy and semantic analysis quality, with specialized optimizations for different business verticals and communication types.

Technical Impact:

The semantic extraction system developed during this internship was integrated into Samsung’s business intelligence pipeline, providing valuable insights from customer communications that informed product development and customer service improvements. The multilingual capabilities expanded the system’s applicability across Samsung’s global markets.

Key Skills & Technologies:

• Natural Language Processing: Information Extraction, Named Entity Recognition, Semantic Analysis
• NLP Frameworks: SpaCy, NLTK, OpenIE, StanfordIE, KlonPY
• Multilingual Processing: Cross-language information extraction
• Data Analytics: Pattern recognition, Semantic clustering
• System Architecture: Scalable NLP pipelines, Modular design

Kyutech Institute of Technology, Shibata Labs

Fukuoka, Japan
Research Intern, Brain and Human Intelligence Systems
May 2018 - July 2018

Research Innovations:

• Advanced Cloth Segmentation: Enhanced the Mask R-CNN architecture for precise cloth segmentation by developing a custom dataset and training methodology. This improvement significantly increased the accuracy of fabric recognition in varied lighting and folding conditions, a critical capability for assistive robotics.

• Robotic Manipulation Optimization: Engineered an algorithm for determining optimal picking points from Kinect point cloud data (PCD), enabling the Baxter robotic arm to manipulate clothing items with greater precision and reliability, even when handling complex fabric types and configurations.

• Integrated Control Interface: Developed a comprehensive software interface using the QT library that seamlessly integrated Kinect sensor data, robotic motion planning, region of interest detection, and segmentation results into a unified workflow, significantly improving system usability for non-technical operators.

• Synthetic Data Generation: Created an extended labeled dataset using innovative heuristic-based algorithms, addressing the data scarcity challenge in specialized robotics applications and improving the robustness of deep learning models in real-world scenarios.

Societal Impact:

• Elderly Care Innovation: Developed a groundbreaking elderly care solution that combined deep learning with robotic systems to assist Japan’s aging population with daily activities. The system enabled the Baxter robot to identify, sort, and assist with clothing—addressing a critical need in elderly care facilities.

• Award-Winning Technology: The project’s custom hardware solution, including specialized grippers and computer vision models, was featured at RoboMech 2018 and received recognition on Japanese National Television for its potential to improve quality of life for elderly individuals with limited mobility.

Key Skills & Technologies:

• Computer Vision: Object Detection, Instance Segmentation, Mask R-CNN
• 3D Sensing: Kinect Depth Camera, Point Cloud Processing
• Robotics: Baxter Robot Programming, Motion Planning, Gripper Design
• Interface Development: QT Library, Human-Robot Interaction
• Deep Learning: Custom Dataset Creation, Model Training, Transfer Learning

Awards and Scholastic Achievements

Research and Development Projects

Recovery-Aware File Allocation System

July 2018 – April 2019
Supervisor: Prof. Rajkumar Buyya, University of Melbourne, Australia

Developed an innovative file system with optimized data recovery capabilities:

• Engineered a FUSE-based implementation with custom File I/O operations for intelligent data allocation
• Created a reinforcement learning module using Q-learning to simulate and optimize file system processes
• Designed the system with strict integrity constraints, space efficiency, and computational limitations for edge computing environments
• Achieved significant improvement in recovery time compared to standard file systems

Operating Systems Design and Implementation

January – April 2019
Supervisor: Prof. S. R. Sarangi, IIT Delhi, India

Advanced the capabilities of the XV6 virtual operating system:

• Implemented sophisticated inter-process communication mechanisms and distributed algorithms
• Developed virtualized containers with syscall independence, resource isolation, and memory paging
• Created a parallel implementation of Jacobi’s function for Maekawa’s mutual exclusion algorithm
• Integrated kernel modifications to support both unicast and multicast system calls
• Achieved performance improvement for specific distributed computing tasks

Kernel Optimization for FreeBSD

January – April 2019
Supervisor: Prof. S. R. Sarangi, IIT Delhi, India

Enhanced FreeBSD kernel performance through innovative memory management:

• Optimized page replacement algorithms using Minimal Counting Bloom filters
• Reduced memory overhead and decreased page laundry times significantly
• Conducted comprehensive comparative analysis between Linux’s CFS and FreeBSD’s ULE schedulers
• Utilized industry-standard benchmarks including Phoronix and Sysbench for performance evaluation

Deep Reinforcement Learning for Game AI

July – November 2019
Supervisor: Prof. Parag Singla, IIT Delhi, India

Recreated and extended AlphaGo Zero’s capabilities:

• Built AlphaGo Zero from first principles using deep reinforcement learning and Monte Carlo Tree Search
• Developed a sophisticated AI for trading card games that accounted for probabilistic elements
• Implemented strategic gameplay across multiple difficulty levels with adaptive learning
• Created a system capable of competing effectively in tournament settings against human players

Near-Infrared Skin Detection

November 2017 – May 2018
Supervisor: Prof. Prathosh AP, IIT Delhi, India

Pioneered novel approaches to skin detection using near-infrared imaging:

• Developed specialized models for skin detection and segmentation using NIR imagery
• Created and manually labeled custom datasets for training and validation
• Integrated the technology with IoT systems for heart rate monitoring and biological signal detection
• Achieved high accuracy in varying lighting conditions where traditional RGB methods failed

Disjunctive Rado Numbers Research

December 2019 – May 2020
Supervisor: Prof. Amitabha Tripathi, IIT Delhi, India

Advanced mathematical research in combinatorial number theory:

• Investigated theoretical bounds for Disjunctive Rado numbers across various equation systems
• Designed efficient algorithms to navigate exponential search spaces
• Implemented mathematical constraints and memory optimization techniques
• Contributed to the theoretical understanding of pattern avoidance in integer sequences

Software Development Projects

Peer Review System & File-Sharing Platform

January – April 2018
Development Club, IIT Delhi

• Developed a comprehensive anonymous peer review platform using Django with modified RSA encryption
• Built an internal file-sharing solution with peer-to-peer WebRTC architecture for secure transfers
• Implemented end-to-end encryption and intuitive interfaces for both systems

Machine Learning Algorithm Implementation

January – April 2019
Supervisor: Prof. Parag Singla, IIT Delhi, India

• Created implementations of fundamental machine learning algorithms including weighted regression, Gaussian discriminant analysis, SVMs, and decision trees
• Applied deep reinforcement learning techniques to the ATARI game Breakout
• Developed custom evaluation metrics for model performance assessment

Game-Playing AI Systems

August – November 2018
Supervisor: Prof. Mausam, IIT Delhi, India

• Designed advanced game-playing artificial intelligence for the abstract strategy game Yinsh using minimax with alpha-beta pruning
• Created a Blackjack AI using reinforcement learning and probabilistic Markov Chains
• Implemented Monte Carlo simulation for strategy optimization

Digital Image Processing & Computer Architecture Projects

2018
IIT Delhi, India

• Implemented Floyd-Steinberg dithering and image in-painting techniques for content-aware restoration
• Created a two-player strategic game in ARM Assembly with predictive algorithms
• Implemented a fully functional pipelined processor with ARM instruction set on Basys 3 FPGA hardware

Programming Language Implementation & Chatbot Development

2018
IIT Delhi, India

• Built Krivine and SECD virtual machines for functional language execution with both lazy and eager evaluation semantics
• Created a complete Prolog interpreter with lexer, parser, and Horn clause solver
• Developed an intelligent information retrieval chatbot with OCR capabilities and automatic summarization

Machine Learning & Robotics Projects

June - July 2017
IIT Delhi, India

• Built recommendation systems using collaborative filtering techniques
• Developed spam detection models using SVM and ensemble classifiers
• Created handwritten digit recognition systems using deep neural networks
• Designed a Bluetooth-enabled home automation system and a functional bomb disposal robot for obstacle evasion and object retrieval

Leadership & Extracurricular Activities

Samsung Global Newcomers Program

April 2024
Crew Member, Samsung University

Served as a mentor and facilitator for Samsung’s international talent development program:

• Delivered engaging lectures and panel discussions on technical topics and professional development
• Led networking and team-building activities for 32 international participants
• Facilitated cross-divisional collaboration between Samsung’s global offices including SDS, DX, Cheil, and E&A
• Shared insights and best practices from my experience in AI and computer vision development

Literary Analysis for Indian Fiction in English

February - April 2019
Prof. Rukmini Bhaya Nair, IIT Delhi

Engaged in advanced literary studies:

• Conducted in-depth analysis of classic and contemporary Indian literature in English
• Explored the nuanced relationship between fiction and reality-based narratives
• Developed critical thinking and analytical writing skills through engagement with award-winning literary works
• Applied interdisciplinary perspectives connecting technology and humanities

Technical Development Executive

January 2017 - February 2019
Development Club, IIT Delhi

Led software development initiatives for campus-wide projects:

• Spearheaded the development of essential campus services including the File Send Service
• Designed and implemented a comprehensive peer review system for academic use
• Created and maintained multiple institute websites and digital platforms
• Mentored junior developers and coordinated cross-functional technical teams
• Established best practices for web development and deployment within the organization

Microsoft Student Partner

August 2017 - May 2020
IIT Delhi

Represented Microsoft’s educational initiatives on campus:

• Organized and conducted technical workshops on cloud computing, AI, and software development
• Facilitated hackathons and coding competitions to promote practical skill development
• Delivered presentations on emerging Microsoft technologies and developer tools
• Created a bridge between Microsoft’s resources and the student developer community
• Helped students leverage Microsoft’s educational programs and technical resources

Technical Lead, Rendezvous Cultural Festival

April - September 2017
IIT Delhi

Led web development for IIT Delhi’s premier cultural festival:

• Architected and developed a comprehensive multi-platform website using Node.js, AngularJS, and Django
• Implemented a scalable backend system capable of handling traffic spikes during the event
• Integrated DynamoDB for efficient data management and Bootstrap for responsive design
• Coordinated a team of developers to ensure timely delivery of all technical components
• Created systems for event registration, ticketing, and real-time updates during the festival

Platform Development Intern

May - July 2017
Anhad Music, Hauz Khas, New Delhi

Developed a digital platform connecting artists with audiences and event managers:

• Created a comprehensive backend system using Python and Django framework
• Implemented user authentication, profile management, and booking systems
• Designed database schemas and API endpoints for efficient data access
• Built search and recommendation features to match artists with appropriate events
• Integrated payment processing and scheduling functionality