Artificial Intelligence Towards Intelligent Environments

DIA: A Drone-based System for Intelligent and Autonomous Homes

Homes are becoming more intelligent due to the growth of smart sensors and devices found in typical homes. However, most of these sensors and devices function independently from one another, limiting the amount of utility and services a truly “smart” home can provide. We introduce two key ideas towards more intelligent homes. First, we explore the usage of mobile drones in the home environment. Second, we propose DIA, a system that seamlessly connects to the home environment and automatically discovers and jointly utilizes smart sensors and actuators around the home to provide services that are otherwise not possible.

[SenSys’21 Demo][Demo]
[Best Demo Award]

SoFIT: Self-Orienting Camera Network for Floor Mapping and Indoor Tracking

We present SoFIT, an easily-deployed and privacy-preserving camera network system for occupant tracking. Unlike traditional camera network-based systems, SoFIT does not require a person to calibrate the network or provide real-world references. This enables anyone, including non-professionals, to install SoFIT. Once installed, SoFIT automatically localizes cameras within the network and generates the floor map leveraging movements of people using the space in daily life, before using the floor map and camera locations to track occupants throughout the environment.

Artificial Intelligence for Mental Health

aiMSE: Towards and AI-based Online Mental Status Examination

Mental status examination (MSE) is an important tool used by mental health providers for assessing mental health. Currently, MSEs are conducted by licensed professionals, which is a barrier for patients in low-income and remote areas. We propose an AI-based Personal Online Mental Status Examination (aiMSE), the first interactive mental status examination platform, where users can self-administer MSEs at home through a web browser, using only a camera and microphone. aiMSE uses multimodal image, speech and natural language processing algorithms to detect signs of abnormalities in mental functioning and recommend them for further examination by a mental health specialist.

Embedded Artificial Intelligence for Acoustics

AvA: Adaptive Audio Filtering for Mobile, Embedded, and Cyber-Physical Systems

There are many applications that would benefit from a platform that allows users and developers to select which sounds to keep and enhance and sounds/noises to remove and filter out. Such a platform is difficult to realize because of the large number of different sounds, models, and signal representations that developers may use in their applications. We introduce, AvA, an acoustic selective filtering architecture that intelligently integrates the physics of sound waves with a wide range of data-driven models in an adaptive feedback architecture to filter and enhance sounds depending on the application.

[Project Page]

EV-interfaced Smart Grid with Human in the Loop

Optimal Power Flow Estimation of Microgrid Considering the Grid Services of EV Batteries

Demand for the grid state estimation with partial power network observation is growing rapidly with the increasing amount of distributed energy resources (DER) connected to the grid with incomplete measured information. The grid services that could be provided by these DER, such as electrical vehicles (EVs), have the potential to affect the resilience and efficiency of the grid. This project proposes a constrained optimization solver based on the AC power flows to recover the incomplete information of the grid. Further reactive power constraints following the specifications in IEEE 1547 are added to the solver to explore the effects of adding grid services to the steady-state microgrid.

Platforms for Continuous Fever Screening

SIFTER: Low-Cost In-Situ System for Continuous Multi-Person Fever Screening

With the recent societal impact of COVID-19, companies and government agencies alike have turned to thermal camera based skin temperature sensing technology to help screen for fever. However, the cost and deployment restrictions limit the wide use of these thermal sensing technologies. In this work, we present a low-cost system based on a RGB-thermal camera for continuous fever screening of multiple people. This system detects and tracks heads in the RGB and thermal domains and constructs thermal heat map models for each tracked person, and classifies people as having or not having fever.

[Project Page]

Building Energy Optimization

ePrints: Personal Energy Footprinting

With the rise of population and emergence of Smart Buildings and Smart Cities, conserving energy is of prime importance. While people are careful about their usage at home, energy wastage in public buildings/offices goes unaccounted for as users do not pay for it. We develop a system that can generate real-time energy footprint for all users and provide related data analytics to help conserve their energy and even implement advanced functionalities such as prediction and  modeling.

[Project Page][BuildSys’17][TOSN’18]

RecEnergy: An Energy Saving Recommender System

Example Energy Saving Recommendation, with energy metrics.

Recent research efforts have made significant progress in reducing commercial building energy consumption through a variety of methods, including optimizing building heating, ventilation, and air conditioning (HVAC), lighting, and personal electric devices. However, these works focus on reducing energy consuming resources while treating occupants as immovable objects separate from the building energy optimization problem.

RecEnergy is a recommender system for reducing energy consumption in commercial buildings with human-in-the-loop. We formulate the building energy optimization problem as a Markov Decision Process, show how deep reinforcement learning can be used to learn energy saving recommendations, and effectively engage occupants in energy-saving actions.

[Project Page][UMAP’18][IoTJ’20]

CityEnergy: City-Scale Energy Footprinting

Example energy and population estimates for different buildings at 8 am in Manhattan.

In urban cities such as New York City, buildings are responsible for up to 75% of total greenhouse gas emissions, and over 90% of total benchmarked energy consumption. A significant portion of the energy consumed directly services humans in retail, commercial, and residential buildings. As sustainability increasingly becomes an important factor in modern society, energy consumption in the built environment is one area where reduction can have a major impact.

CityEnergy is a scalable, real-time system for computing personal energy footprint estimates. As shown in the figure, CityEnergy estimates the energy consumption and population for each building in real-time; these estimates are used to calculate the per capita energy footprint at the building level. CityEnergy is able to provide personal energy footprints to people who provide their location data through a mobile application.

[Project Page][BuildSys’19]

Wearables for Emotion Sensing

SPIDERS: Wearable Emotion Sensing

We present a System for Processing In-situ Bio-signal Data for Emotion Recognition and Sensing (SPIDERS)– a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user’s facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions, using the bio-signals acquired from three non-contact sensorsm, running continuously for up to 9 hours and costing less than $20.

[Project Page][IPSN’20 Demo][IoTDI’20]

Audio Wearables for Urban Safety

PAWS: Pedestrian and Biker Safety

PAWS (formerly SEUS) is a smart system that alerts pedestrians and bikers of any possible vehicle collision. Addressing multiple areas of study, PAWS uses sensor fusion and advanced recognition algorithms to locate incoming cars and effectively warns the user to prevent accidents, especially in urban environments.

[Project Page][SenSys’16 Demo][IoTDI’18][IoTJ’19][Github]

CSafe: Construction Worker Safety

Construction worker safety is one of the most challenging scenarios in urban safety because construction sounds are often orders of magnitude louder than vehicles. As such, we developed and integrated a novel adaptive noise filtering architecture into a construction helmet wearable that filters out construction sounds, improving vehicle detection and localization.

[Project Page][IPSN’21][Presentation]

Mobile Health

Smart Textiles for Continuous Perspiration Sensing

We present a novel wearable sensor system that measures an individual’s sweat level. The system consists of multiple cotton-covered conductive threads that are then braided into one sensor. The braided 3D structure allows robust perspiration level estimation despite the sensor distortion or the motion of the person.


SPINDLES: A Smartphone Platform for Intelligent Detection and Notification of Leg Shaking

For many people, the habit of leg shaking is an annoyance to themselves and people around them. By simply becoming aware of the motion, one is able to voluntarily stop shaking. SPINDLES is a low-power, low-latency, non-intrusive and high accuracy mobile application to detect leg shaking and notify the user in a timely manner. SPINDLES is robust to the placement of the phone and implements a signal processing pipeline that eliminates the need for training and calibration phases common in machine learning approaches.


Intelligent Privacy Preserving Pillbox

We present a novel privacy-preserving pillbox system to help patients improve medication adherence while maintaining privacy and security. Existing smart pillbox systems have been successful in notifying, tracking and reporting medication adherence, but do not provide sufficient security and privacy for the patients’ medication adherence records. In the proposed system, we protect data that needs to be reported to the doctor by ensuring that the data can only be decrypted by the doctor, authorized by a face-to-face key exchange process through out-of-band communication channels. Our system also gives patients fine-grained control over how their information is transmitted and shared, which has been a major privacy concern for many patients.


Zika Virus Monitoring

The goal of this project is to identify and map possible breeding grounds of Zika carrying mosquitoes, using geotagged images, temperature, humidity and wind velocity information from a drone. We aim to visualize heat maps encoding the probability of a location being a potential breeding ground of Zika carrying mosquitos and make these visualizations widely available to stop the spread of the disease.

[Project Page]