By integrating smart capabilities such as sensing and energy harvesting, Dr. Jeeeun Kim is transforming passive interfaces into adaptive interfaces, altering hardware to be used in non-traditional ways. These interfaces will assist people with disabilities, automate domestic tasks, and power millions of computers.

Kim, assistant professor in the Department of Computer Science and Engineering at Texas A and M University, is a recipient of the National Science Foundation's Faculty Early Career Development (CAREER) award, which will support this research.

"Daily design issues are hard to attend if prior experiences are used as a baseline," said Kim. "Even to those with known personal goals, like reducing utility bills, adopting the latest scientific advances in real life demands expertise because tools to support end-users, like you and me, are lacking."

This research will build on new methods to capture interaction properties, which are highly conceptual, as well as critical fabrication parameters for complex augmentations that are efficient and accurate. This will allow users to have access to smart augmentations at minimal cost.

"Unfortunately, recent advances in AI research of today only focuses on object, instance level detection from millions of images and videos such as a toilet, microwave, not the human interactions, lifting a toilet lid and tapping on microwave's button panels, where all human-centered computing challenges actually occur," Kim said.

To accomplish this task, Kim has developed a new framework for creating a large-scale dataset that helps describe human interactions with daily objects, fine-grained parts and their interaction properties. Kim also developed an end-to-end software system for lay persons to aim their smartphone camera toward their indoor environment to scan for hidden accessibility barriers and overlay detected information on the Augmented Reality (AR) views.

"My research is not limited to accessibility solely," said Kim. "I hope my toolkit for end-users can scan the multi-residential buildings and interactions at multiple dimensions, so residents also can notice how inaccessible their common physical environments are, how much energy is being wasted and how they can make changes toward smart, sustainable buildings by themselves."

While the complete replacement of these interfaces with smart devices is not feasible and may create e-wastes causing huge impact on the environments, augmentations promise cost-effective reconfiguration of daily interfaces. Kim hopes this research is a blueprint for altering physical interfaces in ways that make them more energy and cost effective, while also being easier to use.

"By tackling multifaceted, interdisciplinary approaches across digital fabrication, end-user programming, deep learning, robotics, and design, this project lays the foundation for a future where every individual creates daily innovations in assistive computing devices, smart homes, and green buildings," said Kim.