‘Think to Speak’ App Technology Is a Reality for People with Impaired Verbal Communication

Santa Barbara, California-based sensory communication startup Smartstones recently unveiled the first sensorial communication platform allowing any sensory internet of things (iOT) device to become a communication tool — a major breakthrough in the world of assistive communication technology.

With Smartstones’ :prose app and an EEG headset, you can send text messages or in-app push messages, simply by thinking them. The technology can benefit people with several disorders, including cerebral palsy.

Since 2013, the Smartstones application :prose has enabled nonverbal people to speak on an iPhone or iPad with simply a swipe, tap to speak with the Apple Watch, and gesture to speak with their patent pending Touch device. The company says :prose has become the defacto standard for nonverbal sensory communication, and “think to speak” technology is another breakthrough.

Using brainwave commands and a wireless connected EEG headset from the Australian electronics company Emotiv Systems, the technology can also send a thought message to anyone via a push notification (in-app SMS).

“There are hundreds of millions of people in the world affected by various communication challenges, including autism, ALS, cerebral palsy, strokes, brain and spinal cord injuries, and deafness, many looking for greater independence, social inclusiveness, self expression, and to be understood,” said Smartstones Founder and CEO Andreas Forsland in a news release. “Communication is the most important part of the human experience and we want to ensure every person has the ability to express their most basic needs. With 2.3 million nonverbal children in the USA with autism alone, and over 70 million on the autism spectrum worldwide, everyone has a unique need when it comes to a communication interface.”

“Smartstones are about connecting people in new ways, creating a community of support and feedback, and establishing a comforting presence for those in need,” Forsland said, noting that the inspiration came from spending seven weeks at his mother’s bedside as she was in an intensive care unit struggling to survive, intubated and unable to move, talk, write, or use any fine motor skills.

“Maybe if you can’t talk, you can still express that you need help, are happy, or upset,” Forsland said. “In return, your loved ones can give you peace of mind by letting you know that although they aren’t there physically, they’re thinking about you. Sort of a tactile moral support or way to communicate if something urgent arises and help is needed.”

For her 70th birthday, Forsland’s mother Sarah, who lived on the east coast, came out to visit her son in Santa Barbara. Shortly after her arrival, she developed severe pneumonia while Forsland was out of town for a day; she went into septic shock and suffered kidney failure. Fortunately she was able to call 911 and summon an ambulance. When he arrived at the ICU, Forsland was confronted by a team of doctors who asked him if his mother had her affairs in order. With little hope and a lot of prayer, he authorized that she be intubated and put on life support for weeks, during which her arms were tied to bed rails to prevent pulling out tubes and wires, and she was literally unable to communicate.

Forsland’s mother recovered, surprising everyone in the hospital, and a year later he attended Startup Weekend in Santa Barbara where he pitched a solution for this need of sensorial non-verbal communication, for people who are unable to talk, write, or type — to a crowd of about 300 entrepreneurs.

The result was the invention of a product called Hopestone, later renamed Touchstone, designed to enable anyone in a situation where they either cannot, or should not, be using a cellphone to communicate with loved ones or the most important people in which they need to stay in constant contact.

Forsland said he imagined: “What if my mother’s friends back east were praying for her by simply rubbing, moving, or twirling a pebble-sized rubbing stone that sends out a simple message. Messages like, ‘we’re here for you,’ ‘way to go,’ or ‘I love you,’ and she could have simple ways of expressing ‘I’m ok,’ ‘I’m really nervous,’ ‘I love you, too,’ or ‘I need something right now.’ Simple messages being sent anonymously to a larger social network, or privately within a personal cloud network of Touchstone users/friends/family/colleagues/teams — a sensorial way to stay in touch and send good vibes to each other.”

Traditional AAC apps use pictogram-based approaches, which while versatile, can take months or even years to fully learn. By contrast, Smartstone’s :prose app works by translating sensor data such as touch, motion, and now thought gestures, into spoken phrases. Phrases are easily changed depending on which gestures are easiest or which phrases are used most. Integration of EEG technology makes :prose accessible to even more people, including those with cerebral palsy, ALS, Parkinson’s, and other motor-skill inhibiting conditions, who may find touch gestures difficult to perform reliably.

Recently, Smartstones has worked with PathPoint (originally known as Work Training Programs Inc.), a 501(c)(3) nonprofit dedicated to helping people with disabilities or disadvantages to reach their fullest potential, to test the technology.

The philosophy behind PathPoint’s mission is that all individuals with disabilities or disadvantages can achieve self-sufficiency and dignity. The nonprofit is a leader in the fields of employment, community access, residential, and behavioral health services, and has received national recognition, currently serving more than 2,000 people with disabilities or disadvantages in five California counties: San Luis Obispo, Santa Barbara, Ventura, Los Angeles and Kern, alongside their Behavioral Health Division.

Smartstones, in collaboration with PathPoint, introduced the first sensorial communication platform allowing a sensory iOT device to become a communication device. Communication is now possible simply by thinking it. Using :prose’s think to speak technology, PathPoint participants with severe developmental disabilities were able to use brainwave commands to communicate, while connected to the wearable Emotiv EEG headset. Smartstones has an official partnership with Emotiv to provide the headsets.

“Seeing the expression on our participant’s face when she was able to say a word for the first time just by thinking, was a moment I will never forget,” said PathPoint South Santa Barbara’s vice president, Seth Miller.

These mobile device mediated :prose-enabled commands have produced a form of communication that was once never thought possible, allowing the participants to communicate their thoughts with just a smile, frown, or nod of the head.

“People have been trying to use neural devices to communicate for decades,” said Gil Trevino, lead direct support professional at PathPoint, Santa Barbara. “I’m happy to report that Smartstones has achieved this. Because of :prose’s cutting edge in assistive technology, for the first time we were able to see a neural device empower communication for a nonverbal individual. One of our participants now can communicate by using mental commands. Within minutes she was speaking several phrases aloud, compared to years of training with other technologies. This advancement has allowed someone who once was non-verbal communicator, the ability to communicate her thoughts, feelings and answers in a way she never has before.”

Nuri Djavit, EMOTIV chief marketing officer, said, “Emotiv is incredibly proud to be involved in the :prose program and to be providing the wearable EEG component. From the start, our mission has been to advance our understanding of the human brain and to find ways to improve the human condition, especially for those facing adversity.”

Smartstones and PathPoint plan to continue their collaboration empower communication outlets to those who need it most.

“We are at the beginning of the sensorial communication age, moving beyond the dependency of a visual interface,” said Forsland. “Many companies in iOT and wearables are already dealing with sensorial interfaces for data gathering. We want to be the iOT wearable platform that enables communication between the devices.”

Smartstones :prose is available for individuals to download onto their personal Apple iPhone or iPad. The application integrates with a variety of Bluetooth connected sensory devices, including Smartstones Touch, Apple Watch and now an Emotiv Insight or Emotiv Epoc wireless EEG headset as a beta community member.

With every purchase of Smartstones :prose, you may request a special discount code to be used toward the purchase of an Emotiv Insight or Emotiv Epoc brainwear device.

Download Smartstones :prose at:
https://itunes.apple.com/us/app/prose/id945122730?mt=8

For more information about PathPoint, visit:
https://www.pathpoint.org