(noesis) In North America, 4.76 million of community-dwelling seniors are unable to independently leave the house at least once a week, significantly raising the risks of loneliness.

Another way to think about this is that the Individual functional capacity does not meet the demands of the life-space—bedroom, home, outside home, neighborhood, town and distant locations.

This can roughly bebroken down into environmental and personal causes.

(Elena) a. BrioCare is a new, voice-assistant technology solution.

• Similar to Alexa and Siri in operation
• Engages the mind and body of seniors throughout the day; beginning with a greeting in the morning, to asking what they would like to do during the day, and finally ending with their nightly routine

b. CaringWire is a digital platform that makes healthcare more accessible and manageable for individuals and families.

c. Birdie is a holistic care platform providing tailored preventive care for older adults so they can live independently in their own homes.

• Uses apps and home connected devices to track elderly well-being in real-time.
• It has also developed healthcare analytics using data from carers’ reports and connected sensors to deliver better preventative care

Main limitations:

a. BrioCare

• Its services are limited for only voice-based help
• Cannot help elderlies with mobility issues outside of their houses

b. CaringWire

• Although it provides the medical assistance, it doesn't help in moving in and out of the house - the service that seniors with lack of mobility really need

c. Birdie

• Act as a virtual assistants but lack physical help in managing household chores, etc.

These companies offer a combination of software and hardware solutions for senior mobility.

1. Top companies:
   1. Intuition Robotics - The main product is ElliQ.
   2. AIST - PARO
   3. SoftBank Robotics - Pepper
2. Main limitations:
   1. ElliQ
      • unable to move
      • applies a singular proactive solution to seniors with varying needs of companionship
      • some have problem adopting the humanizing form and think it’s patronizing (source)
      • mashing together a large number of underdeveloped features: (1) Despite highlighting the ability to initiate conversations, most last 2 sentences or less, even less than existing digital chatbots. (2) It could play videos and songs from a database, but lacks flexible integrations with youtube, spotify etc. (3) It has no established system of contacting emergency services. There are also implementation limitations like video call lagging, speaker glitches and delays. It’s one time payment and monthly costs higher than medical alert systems. Some users reported feeling dystopian vibes(source)
   2. PARO
      • One survey found 1 in 10 refused to interact with PARO in front of others since they felt it was childlike or had previous negative experiences with animals
      • PARO does not have any voice recognition for calling 911 or a method to report falls, which may cause anxiety in some seniors living at home.
   3. Pepper
      • Appearing like humans prompts people to ask questions it can’t answer
      • It was a master of none, failing at the first step of interaction: face, emotion and voice recognition. Leading to the subsequent lack of trust.
   • Buddy
     • mishears commands, repeats same phrase “I’m tired to talk”

In the recent decade, studies showed that companion robots can help seniors reduce feelings of isolation. After accompaniment, seniors enjoyed the availability and non-judgemental attitude of these robots. They treated the robots as friends, became more facially expressive and felt generally more positive over the long term.

• A cross-sectional study of the effects of COVID-19 on perception of and intention to purchase a social robot found that loneliness was positively associated with reported willingness to buy a robotic companion, indicating that people may perceive that a robotic companion could mitigate loneliness.

• A study that predates the pandemic of non-AI robots, Paro and Giraff (telepresence), suggests a role for psychosocial functioning (depressive mood, loneliness, life satisfaction and social support) in robot acceptance among older adults.

• A small study of robots use in dementia care found that participants were concerned that it could increase isolation for this group.

Companion robots also benefit, not replace, human interaction: Accompanied seniors formed more new connections, and had more conversations with family and other human beings.

While well-intentioned, current designs like ElliQ, ROBEAR and SAM heavily involve rubber and plastic. According to surveys, seniors view such materials as "too cold". It is also unfavourable for robots to resemble toys or humans; be too small; be too fast-moving; be noisy and be a hassle to maintain.

Meanwhile, seniors felt close to robots which look like familiar animals with soft fur, facial expressions, voice and breathing movements.

Companion robot market size estimated to be $9.2 billion USD and expected to expand to $36.3 billion USD by 2028. Financially viable, but will likely be expensive to compensate for costs. Hence, market to middle-upper class in developed countries.

The speech recognition module generates several hypotheses of the uttered phrase ranked in terms of confidence in the N-best algorithm. Text from the first-best is passed into the Natural Language Understanding (NLU) system into symbolic language. e.g. ``a red cup = cup(x) ^ red(x)``. While not every robot converts to symbolic language, social robots which rely on advanced logic would use both first-order predicate logic and temporal logic, e.g. grabApple = False and becomes ``True`` later to reflect the changes across time.

Using perceptions, the robot connects the world representation and the symbolic representation together. Decision-making uses the memory of the conversation and the user environment to decide the next physical or language action. Language action is converted back into the natural language generation module and finally uttered by the text-to-speech system.

NLU you say? What does that mean?

NLU overview: types of language models Language models aim to create a probability distribution over the future/missing tokens.

Statistical language models (SLMs)
This is based on the Markov assumption, which states that the probability of a word depends only on a limited number of preceding words (i.e., fixed-context). SLMs and uses a number of words before the last word to predict the most statistically likely next word. An n-gram language model is a type of SLM that models the probability of a word given the last n−1 words. A bigram model (n=2) considers only the previous word (hence two words in total), while a trigram model (n=3) considers the previous two words.

However, SLMs suffer from the curse of dimensionality: As the number of features (e.g. vocab, punctuation, num words) in the input increases, the amount of possible n-grams required to accurately model the probability also increases exponentially. This leads to data sparsity, meaning that many word sequences are not observed in the training data, making probability estimation unreliable. Thus, performance degrades.

• training: many n-grams
• prompt: a specific n-gram
• output: probability distribution over next possible words

Neural Language Models.
Data is passed into a neural network. Each word is assigned a vector of continuous values, with the dimensions of the vector representing different features of the word. The idea is that words with similar meanings will have similar vector representations, the vector for the word "king" might be located near the vector for "queen”. Approaches like Continuous Bag-of-Words or Skip-gram is used to represent the relation between words. Continuous Bag-of-Words (CBOW): In this approach, the neural network predicts the current word based on the context words surrounding it. The context words are represented as a bag of word embeddings, which are learned during training. The bag of embeddings is then averaged to create a fixed-size vector, which is passed into a neural network to predict the target word. Skip-gram: In this approach, the neural network predicts the context words based on the current word. The current word is represented as a word embedding, which is learned during training. The word embedding is then passed into a neural network to predict the context words. The set of vectors and their geometric relationships are passed into Neural Network to predict the probability. The network contains of an input, hidden and an output layer. word2vec is a relatively effective model.

• training: millions ~ billions of words.
• prompt: word sequence of arbitrary length
• output: word sequence of arbitrary length

Types of deep neural networks used: LSTMs, Transformers, Bidirectional transformers.

Large Language Models As of 2023, most large language models can take in 4,000 words, with the largest taking in 32,000 words.

Word and sentence embeddings. Embeddings can capture analogies well. Each word is assigned some properties that are placed on an axis. e.g. The word dog can be assigned the properties age and size, and is plotted as a point in nth dimensional space based on num properties. This can also be applied on the sentence level. These features are found by the computer and can sometimes be interpreted by a human.

How similarity is measured:

• Dot product. The dot product between two items is calculated across each column. A large number signifies similarity.
• Cosine distance: Take the angle difference between the position vectors. Then take cosine. Similarity ranges from -1 to 1 (itself) Can evaluate similarity between queries and answers to discover the right answer.

Some terminology:

• Encoder-decoder models, i.e., sequence-to-sequence models areGood for generative tasks that require an input, such as translation or summarization.
• Encoder-only models are good for tasks that require understanding of the input, such as sentence classification and named entity recognition.
• Decoder-only models: Good for generative tasks such as text generation.

[30 lines regarding attention mechanisms omitted since they're common knowledge anyway and poor formatting]

First you can code a digital prototype. Play around with convai, character.ai, replika and so on. The user could speak vocally about how their day is going, ask for social support, or muse about the philosophy of life. Then, the companion speaks back to them with animated expressions. The text response will be generated with some LLM and read aloud. Meanwhile, the tone of the text will be analyzed to display the right animations. This should take no longer than two weeks to implement.

Next, build the physical prototype. Find a manufacturer that makes children's stuffed animals, go around in Shenzhen and you'll find a bunch. The previous program will be downloaded into a single-board computer inside the robot. Several new functions are needed: Firstly, generating facial expressions. Secondly, moving animal parts for realism. Thirdly, controlling breathing, by using a linear actuator to inflate or deflate a ball inside the robot - try Pikachu!

A report by Tractica estimates that by 2025, there will be over 33 million social robots deployed worldwide, up from just 3.1 million in 2016. This growth is driven by advancements in artificial intelligence and natural language processing technologies.

last updated spring 2023

The Handbook on Socially Interactive Agents: 20 years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 1: Methods, Behavior, Cognition

A list of recommendations for social robots

LLM and robotics content list

Improving the logic of NLU

Review on Multiple types of companion robot

Hobbit robot for seniors living independently

Robots using language

Emotional support for conversations

Analyzing social robotics with NLP

NLG evaluation

Memory

ConvAI competition ConvAI research Hugging Face