Customizable Smart Devices for Everyday Living

2019-present

People with cognitive disabilities often face difficulties in remembering, learning, and making decisions, which can impact their everyday activities. To overcome challenges, individuals use reminder applications on handheld devices, like smartphones or tablets. However, these multipurpose devices are compact and can easily be misplaced. Furthermore, individuals need to constantly pause their tasks to view instructions on these devices. Repeatedly switching contexts can negatively impact their ability to complete tasks and may result in discontinued practice or abandonment. Smart home devices, like voice assistants and “smart” appliances, are promising solutions to naturally integrate prompts within everyday activities. Specifically, Augmented Reality (AR) based smart assistants can situate visual information in the location of tasks; thereby, reducing the amount of contextual switching between tasks and assistive devices. Besides contextual prompts, individuals often require devices that adapt support based on their abilities. Therefore, smart devices could combine contextualization and customization by embodying individualized AR-based prompts and feedback. One approach to support customization is to allow individuals to write rules or algorithms on how devices should support them. Introductory block-based programming tools can enable people with cognitive disabilities to write algorithms and reflect on support for everyday tasks.

Towards augmented reality coaching for daily routines: Participatory design with individuals with cognitive disabilities and their caregivers
Varsha Koushik and Shaun K.Kane, International Journal of Human-Computer Studies, 2022
paper link

Accessible Introductory Programming

StoryBlocks

2017-2018 Block based languages like Scratch, Blockly, MIT App Inventor, Snap etc. are widely popular as a first step towards traditional programming. They offer many benefits including simpler syntax, easier interactions, and support creativity and expression through games, animations, stories and other interactive media. However, these visual based languages are not accessible to people with visual impairments, and thus they limit access to introductory programming tools. In this research, I have designed a tangible programming game called StoryBlocks, which uses physical blocks to create story programs, and the physical stories are converted to audio stories with different voices and sound effects using computer vision. This multimodal interface has been designed to support collaborative activities around introductory computing in K-12 classrooms.

StoryBlocks: A Tangible Programming Game to Create Accessible Audio Stories
Varsha Koushik, Darren Guinness, and Shaun K. Kane. 2019. StoryBlocks: A Tangible Programming Game To Create Accessible Audio Stories. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19). Best Paper Honorable Mention Award [top 5%]
ACM paper

Tangibles + Programming + Audio Stories = Fun
Varsha Koushik and Shaun K. Kane. 2017. Tangibles + Programming + Audio Stories = Fun. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘17).
ACM paper

STEM Blog Article

PseudoBlocks

2016-2017

Block-based languages are used to teach programming to novices, through the creation of interactive games, stories and animations. However, programs are created using visual representations, which are not accessible to blind and visually impaired users. We designed a non-visual block based language called PseudoBlocks, which uses pseudo-spatial keyboard movements to create programs, and produces audio outputs. In its current form, PseudoBlocks can be used to create simple melodies.

An Accessible Blocks Language: Work in Progress
Varsha Koushik and Clayton Lewis. 2016. An Accessible Blocks Language: Work in Progress. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘16).
ACM abstract

Work in Progress: A Nonvisual Interface for a Blocks Language
Varsha Koushik and Clayton Lewis, 2016, VL/HCCThe Psychology of Programming Interest Group (PPIG) 2016
paper link

Accessible Sports Wearable Assistive Technology

Today, blind athletes use a tapper to warn them, when they approach the swimming pool wall. The tapper walks along the pool with a long stick with a tennis ball like round object attached to the end of the stick, and taps on the shoulder or the head when blind swimmers approach the pool wall. This approach is inefficient, and it can be expensive to employ a tapper. We designed a wearable activity tracker called Goby, which is worn on the thigh, and uses a downward facing camera to trace the swimmer’s position in the pool. Goby detects the black T- shaped line and gives audio feedback to warn the swimmer, when they approach the wall, or swim outside their lane.

Goby: A Wearable Swimming Aid for Blind Athletes
Annika Muehlbradt, Varsha Koushik, and Shaun K. Kane. 2017. Goby: A Wearable Swimming Aid for Blind Athletes. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘17).
ACM abstract

CU Boulder Today ATLAS Expo Article
CU Boulder New Venture Challenge Finalist 2017: Social Impact