CARE: Cooperation of ai Robot Enablers to Create a Vibrant Society

Demographic changes in our society are putting a heavy burden on care facilities and health-care infrastructure. While the elderly population is steadily increasing, there is an acute shortage of caregiving experts and professionals. This problem is becoming more severe in superaging societies, namely, Japan. Hence, this urges new and practical solutions for welfare facilities to mitigate the burden on caregivers and human supporting partners by introducing robotics assistance through information and communication technology (ICT). In this work, we present a new multirobot cooperation and coordination framework at different intellectual computation levels for care facilities. The framework is developed to bring the health-care 4.0 concept one step closer to reality, under the ongoing project “Moonshot Research & Development,” in Japan. First, we present an Internet of Things (IoT) integration system that is designed to include different passive and active assistive robots. Then, we redesign robot systems and develop a semiautonomous platform that can perform tasks based on user/patient interaction in real-world care facility scenarios. Our framework provides human–robot interaction under shared autonomy between the user and assisting robots to improve the efficacy of the users in everyday tasks. Tohoku University’s new state-of-the-art Living Lab facility is used to prepare a real-world scenario, where we present our experimental results. We also discuss the open problems in future care and human assistance aspects.


I. INTRODUCTION
Care facilities are highly dependent on human assistance and social cooperation.Current demographic conditions in several countries have led to severe challenges due to the acute decline in population.Particularly in Japan, the percentage of older adults increases yearly.It is estimated that by 2036, a third of Japan's total population will be over 65.An aging society puts much financial burden on the nation's resources as its health expenditure increases.On the other hand, the demand for care workers to cater to such a super-aged society has increased drastically due to a severe shortage of skilled labor.Accelerating new research areas in assistive support by integrating information and communication technology (ICT) and robots in care facilities can reduce the burden of nursing care workers and improve the overall efficiency of everyday caregiving.Moreover, mental and physical compatibility is required to help/assist the patients or elderly.Caregivers face immense stress when caring for the recipient and, at times, also face abusive behaviour from patients.This issue becomes sensitive when a patient has an appropriate level of disability.This was also evident during the COVID 2019 pandemic where health workers had to work in a highly contagious environment and provide patient care.Robot support systems can improve patient and caregiver satisfaction and abilities by sharing the workload.For example, the same task can be done with a service robot which would keep both the patient and the caregiver safe.This has motivated different robot technologies to be integrated into care facilities [1]- [3].
The notion of service robots is not new, and several such robots are continuously finding their way into our everyday life.A number of different service robot platforms from the industry were introduced earlier [1], [4].However, there is a serious gap in integrating different robotic platforms and technologies into the current healthcare infrastructure.This open problem exists because robotic systems in care is a broader topic mainly pertaining to service robots that are aimed at helping users with daily activities and/or as companion robots that provide emotional and psychological support for the well-being of the receiver.A more general trend for using robots in care facilities is the term socially assistive robots (SAR) [5]- [7].These robots cover broader subjects in medicine, care facilities, offices, and other public areas, tending to different applications and services ranging from rehabilitation, entertainment, communication, health monitoring and tracking, delivery of goods, and hospitality.Various studies have discussed the importance of automation and robotics in care facilities [8].The earliest noticeable discussion was brought by Kassler [9] where the potential of robots in assisting and giving services to users was envisioned as the next era for health care.This resulted in the development of multiple scenarios where robots provide various services to help patients [1], [3].
Another exciting area where much interest has grown recently is the integration of SARs with assistive ambient living (AAL) [10].Here the focus is on providing assistive care to the individual at one's home.Using social robots and sensor integration, it aims to monitor, assist and provide social and cognitive care to the individual right from the early onset of disease, especially in cases where only reasonable nursing care is required [11].Earlier studies of such integration in several European projects have demonstrated the successful use of SARs and sensor integration for longterm monitoring and tracking of users at homes and facilities, providing relevant assistance, and the interaction between elders and robots in different scenarios [12].ASTROMobile project presented a social robot that interacts and gives certain services to the elderly [13].The Robot-Era project [14], [15] studied the acceptance level, technical feasibility, and satisfaction of elderly users by employing three mobile robots in different service areas such as domestic, condominium, and outdoor environment.CompanionAble [16] and SERROGA [16], on the other hand, studied about longterm use of SARs in private homes.Other cases include the Strands project [17] where the robots were deployed in a large public environment.Very recent works include the MoveCare project [3], [18] that successfully tested and demonstrated the use of SARs with AAL using a social mobile robot GiraffX for long-term operation in private houses of elders living alone and targeting individuals who are at risk of falling into frailty.
With similar goals, Japan's government (Japan Science and Technology, JST) initiated a new large-scale funded research and development project known as the "Moonshot R & D Project 1 " in 2019.This project aims to create a vibrant and symbiotic society by the year 2050 with multiple AI robots installed in various public facilities (commercial, cultural, tourist, sports, nursing, hospitals, and childcare) and maintained as social infrastructure.Within the several goals of the project, the Cooperation of AI Robot Enablers or CARE 2 is one of the projects with a focus on enhancing the human quality of life by creating new robot technology in the coming decade (see Fig. 1).Through the CARE project, we are developing a muli-robot cooperation system for welfare facilities and researching the design and development of advanced assistive robots for homes and welfare facilities by the year 2030.
This paper presents a fundamental concept of multi-robot cooperation for welfare facilities with the framework of CARE.The main contributions of this study are • An Internet of Things (IoT) integration system is designed to include different passive and active assistive robots, • The safety and preferences of the user are considered with input from individuals' interests, disabilities and physical status (self-efficacy), • A new concept of human-robot interaction is designed under the shared autonomy between assisting robots, human-given inputs, human safety and assistive controllers, • A realistic Living lab facility and scenarios are introduced for experimenting with our proposed framework that are evaluated for the potential application of the concepts using expert advice, The paper is organized as follows.In Section II, we introduce the concept of Cooperation of AI-robot enablers and discuss the details regarding the assistance and automation 1 https://www.jst.go.jp/moonshot/en/ 2 https://srd.mech.tohoku.ac.jp/moonshot/en/ level in the proposed caregiving system.Section III describes the Living Lab facility at Tohoku University and explains different living lab robots as assistive and service robots.Next, the interface between local and global intelligence is explained in Section IV.In Section V, we explain the human interaction with AI systems with a description of sensory systems and our developed human detection, tracking and condition evaluation system.Section VI describes local intelligence by considering multi-robot cooperation with the co-existence of humans.Section VII explains an example scenario demonstration for the proposed concept.

II. CARE: COOPERATION OF AI ROBOT ENABLERS
This section discusses the main idea of the developed framework under the Moonshot Project.Our focus is on developing multi-robot collaboration while considering the human factor for long-term intelligent assistance and support for the human by utilizing new sensor development, sensor fusion, robotic assistance, human-robot cooperation, and data processing.Our aim is to develop an AI robotic care system that caters to the needs of the specific user by providing the most relevant robotic support as the user request for a certain task to be done.By learning individual preferences and processing sensor data over a long period, the AI can then recommend the best output and provide support to the user by deploying and/or collaborating with different robots.In contrast to other relevant projects discussed earlier that have used SARs for long-term support in homes or care facilities, our project has several key features summarized in Table I.Our system can provide appropriate support to the user by distributing tasks to multi-robot systems where each robot can do a specific task.Unlike a single robot platform, our proposed system is easier to implement, and task distribution and allotment between the robots can be efficiently handled without hindering the overall task execution.Firstly, we integrated multiple robots with the IoT system and provided support not just in indoor but outdoor environments as well.Our framework include service robots, autonomous walkers, and wheelchairs.For specific tasks, the most appropriate robot is selected by the global intelligence AI.Secondly, our system considers shared autonomy when the user is physically interacting with a robot system (e.g.bed or walker).Analysing human input using wearable and external sensor data makes the response smoother for the user when engaging with the robots.Thirdly, our system considers safety as a feature during the entire assistance phase.The safety of human motion is quantified in real-time which provides information together with incoming obstacles in the environment for creating a shared autonomy policy for the robot and human interaction.

A. The Concept of AI Robot Enablers
Care facilities are critical institutions that get tremendous attention in automation and robot integration.Additionally, the caregivers need to support the patient from many different aspects.The goal of the research and development of adaptable AI robots is not to provide excessive support or services to the user but to realize human-centred care that encourages the user's independent movement, tasks, and other activities.The interaction between the adaptable AI robot and the user is accumulated as experience.The user's success (or failure) experiences are shared between the user and the AI robot to improve the sense of self-efficacy, i.e., the user can actively participate (physically) and perform the desired action or task with support from the AI robot.
Cooperation of AI-Robot Enablers (CARE) is a flexible and supportive assistance technology that helps users to accomplish tasks by combining AI robots, assistive devices, sensors, and user interfaces.In this concept, each user will be assisted based on their disability and required support level.Also, the SARs will work in harmony with heterogeneous order to achieve different assigned tasks by the users.This happens under a global intelligence that monitors the environment and uses other sensory systems to keep the users' satisfaction and health at the uttermost level under self-efficacy boundaries.

B. Assistance and Automation Levels in Care Giving
Caregiving happens in different aspects.Based on the expanded disability status scale (EDSS) [20], the disability can be quantified in different levels where zero states that the person has a normal neurological and physical function and is able perform different tasks easily.However, the person needs assistance for anything over the value of 3.0, either physically or mentally to continue the required activities.In our current research plan, we are considering the EDSS∈ [2.0, 6.0] range.It is important to note that the ultimate goal of the project of adaptable AI-enablers is to achieve till 8.0 level which will be possible with improvement in mechanism and sensory designs as the project progresses.Also, the EDSS level 7.0 is not a consideration for the current work since the restricted immobility of a person might require heavy carriage support that is not practically feasible with currently available robots.
The automation level is dependent on the person's disability and requested tasks.As shown in Fig. 1, if a person with high disability requests assistance, a wheelchair-type assisting robot will go for the support.However, if the EDSS is around 5.0 with considering user request/preference, the walker-type assisting robot will approach the user.Additionally, our service robots work under safety protocols (obstacle avoidance from moving other robots) to bring the required items.We think this framework for automation using IoT systems and other robots can be a stepping-stone for efficient care.Also, the IoT system plays an important role in connecting different sensors and robots using the internet communication framework.The communication can transmit and share information between different processes,

III. THE LIVING LAB
As part of the Japan Ministry of Health, Labour and Welfare's "Project for Establishment of a Platform for Development, Demonstration and Dissemination of Nursing Care Robots3 ," the Living Lab aims to accelerate the flow of development, demonstration and dissemination of nursing care robots as an evaluation and effectiveness verification organization for such robots.The Living Lab is a group of facilities that promotes the development of nursing care robots based on the needs of nursing care settings by reproducing actual living spaces and developing new technologies and services with the use's participation.It supports organizations and institutions that wish to evaluate their nursing care robots in development and verify whether they can be used in actual nursing care settings.Based on evaluation and expertise in the field of nursing care, the Japan ministry selected eight Living Labs to participate in this project nationwide.These Living Labs also aims to build a network through this project and support developers by leveraging their respective strengths.The "Aobayama Living Lab4 " at Tohoku University was selected as one of the eight Living Lab for undertaking next-generation nursing care robot research and development.
The Living Lab is a care facility and home where different robots, advanced sensors and guidance systems are integrated to replicate a concept for future welfare facilities as demonstrated in Fig. 3.It simulates a nursing home environment with toilets, bathrooms, kitchens, living space, and a simulated outdoor environment with stairs and slopes.The layout of the Aobayama Living Lab is presented in Fig. 2. In our Living lab, we have developed a cooperative care support system in which multiple care robots and sensor systems work together to provide support rather than being limited to a single care robot.Different robots with specific abilities with respect to users' required tasks are utilized in our facility.
In the constructed facility, we have active and passive supporting robots (see Fig. 3).The active supporting robots consist of electric wheelchairs and walker robots.We utilize an automated bed system and a service robot in passive supporting robots.In the group of active supporting robots, the automated wheelchair system is re-designed using the commercially available platform (Whill) robot.This robot is equipped with different onboard sensors e.g., encoders, joystick control, and IMUs.The wheelchair was upgraded with additional sensors such as RGBD camera and 2D LIDARS for sensing and autonomous navigation tasks.The walker robot is an automated walker developed by RT-Works Corporation, Japan [21].This robot is also upgraded with external sensors for autonomous navigation within the facility.Also, all robots contain a tablet interface to let the user/professionals have direct interaction with ongoing operations in the robot.This includes a direct interface for the users to remotely operate the robot during its ongoing execution of tasks or making the changing request.
In the group of passive supporting robots, a multi-function bed with onboard actuators has been modified with hybrid switches for head and height adjustment.Furthermore, these smart switches are placed in a way that they can connect directly to the internet cloud to feed information about bed states and the ability to manipulate the bed position concerning ongoing tasks.Also, a mobile service robot based on the Turtlebot2 robot platform is currently developed for applications such as delivering small objects to the user and interacting during different tasks.

IV. WEB-BASED MESSAGE PASSING INTERFACE FOR
ROBOTS USING ROS In the CARE project, we are working on cooperative control of multiple robot groups and sensor groups using a common interface for communication and exchange of information between the user and robots operating in the Living Lab.We focus on developing a smart decentralized multi-robot architecture in which the most appropriate robot will come to the user based on individual needs and requests.
A communication medium is necessary for different robots to pass messages and share information.In the past, plenty of research has been carried out about the topic of communication and coordination between robots, and different approaches have been proposed [22].Recently cloud robotics has been getting a lot of attention [23].It invokes cloud-based technologies such as cloud computing and storage, parallel processing, and internet services for sharing information between different robots or agents.All of these approaches are solutions crafted for a specific application and it's necessary to adapt and implement them for particular applications.
In our framework, we have considered the Robot Operating System (ROS) as the middleware for the sake of modularity and also for the smooth integration of different sensor libraries into our robot framework.Using ROS accelerates the development process vastly, and enables the researchers to implement the complex system quickly.ROS relies on the host computer network capabilities to distribute messages in the system.When multiple robots are run in the same network in order to communicate with each other, all their sensory information travels through the network so that each robot is able to see the other robots' status (i.e., the other robots' topics, where sensor readings are published, messages and services).This can be very data intensive: when the number of sensors in the system increases, the network traffic becomes larger, and this is duplicated when an additional robot is added.To this end, creating a way for different robots to transmit only the necessary data at the right time is desirable.
To achieve this asynchronous communication between different systems using ROS, we proposed an interface based on the Node.jsJavaScript runtime.Node.js is traditionally used to create websites and back-end API services; in this case, we use it to provide an interface between systems.We based this architecture on the framework proposed by Giller et al [24] to connect ROS with IFTTT, which is a software platform that connects apps, devices and services from different developers in order to trigger one or more automation in those apps, devices and services.

A. Interface Structure
The proposed interface's general layout can be seen in Fig. 4. The interface was developed as a ROS node that creates a web server with custom webhooks that the developer can specify via the launch file.The webserver interacts with ROS in a bidirectional way: It can launch ROS services when a webhook is triggered, or it can trigger a webhook in other webservers when a specific type of message is published in topics that the webserver is subscribed to.The relations between the webhooks and the service to be launched (incoming information), as well as the topic name and the webhook to be triggered (outgoing information), are easily configured in the launch file, requiring no further programming.The information between systems is transmitted using the HTTP protocol, and custom fields can be added to the request as JSON payload.

B. Network Requirements and Webserver Address
With this architecture, we can have multiple robots running their own ROS core in their local loopback network, which is much lighter and faster than communicating over wireless networks.If the robots were connected to the same network, they would be able to reach each other by using webhooks formed with each other's IP addresses.To simplify the naming and enable the robots to communicate from any network, we use ngrok, which is a cross-platform application that enables developers to securely expose a local webserver to the Internet with minimal effort.This only requires that the robot has a connection to the internet.In this way, robots can address each other using webhooks formed with the URL assigned via ngrok.

C. Interaction between Systems and IoT
As explained before, to trigger a service in a robot, a web request with a JSON payload is used to trigger a webhook, and the robot will process the request and call service in its own ROS core.This enables us to trigger these services from another robot and any device that can generate a web request, such as voice assistants (Siri, Alexa), internet services (IFTTT) or even a web browser.As the interface enables the robot to trigger remote webhooks (outgoing information), it can also trigger IFTTT webhooks so that the robot can interact with any of the 700+ web services provided via IFTTT.

V. HUMAN INTERACTION WITH AI
The CARE framework including the different subsystems is presented in Fig. 5.There are three main systems: Global intelligence, local intelligence and external sensors with human-interface systems.Global intelligence is responsible for collecting user commands and processing them to detect and track participants in the environment using a previous database of individuals.The next step is to determine the available and suitable robots for the tasks.Autonomous robots perform tasks based on their structure, sensors, and motors within the local intelligence.Each of the components of the framework is discussed in detail below.

A. Request Understanding
The human and robot/guidance system interaction is one of the key points in achieving successful executions based on patient/user requests.However, these interactions have certain bottlenecks since the requests from the user might happens multiple times or the user/patient changes his/her opinion during an ongoing task.Moreover, the safety and ergonomics of the human interface are important; hence, the person can interfere with the execution at anytime one wants to.
The interactions are divided into direct and indirect parts in this work.These interaction interfaces require different sensors which help the general intelligence of the care system to act responsively and on time.For direct interaction, a physical user input device such as a tablet/smartphone and verbal interaction through a commercially available virtual assistant, such as Amazon Alexa voice services or Google voice assistance is utilized.When physically interacting with a smartphone or tablet-based communication, the user can send direct commands to the available robots using custom shortcut buttons.These shortcuts consist of tasks with varying complexities.For example, there is a complete task request for bringing drinks or sending the robot back to the base station.Another example is bed assistance where the user can adjust the bed to his/her desired head angle or height.For voice-based communication, apart from simple task requests, a person can interact with a virtual assistant through normal conversational sentences to ask for service or assistance from various robots.
For indirect interaction, different sensory systems are placed in the environments.These include fixed cameras for person tracking and recognition, a motion capture system for precision tracking and analyzing body posture, force plate sensors mounted near the bed and the sofa for calculating standing/sitting force, RGBD camera system mounted over the user bed for pose estimation and performing sleep analysis patterns.Please note that we maintain a database of the Living Lab users whose information(age, gender, face, voice patterns, assistance level, and other critical data) is fed to the global intelligence for identifying the user commands.Apart from these sensors, the user also wears inertial measurement units, activity recognition devices, and heart-rate sensors, as shown in Figs.5-6.For instance, the person's heart rate and Electrocardiogram data are continuously monitored with a commercially available Hitoe sensor.All these sensors are interconnected through the internet to designate global intelligence to collect, evaluate and distribute the required information between robots.Another purpose of the in-direct interaction is, without requesting much information from the user/patient, our proposed general intelligence can utilize the information from the user and environment to evaluate the

B. Individual Tracking and Safety
Human self-efficacy is a key factor in care facilities.Selfefficacy can be described as: when a person believes in their ability to complete a task.There have been different studies that evaluate self-efficacy with respect to acceptance of robots in healthcare [25] or robots in care [26].For instance, Swift-Spong et al. presented that the participants, under an autonomous robot's guidance and assistance, improve their overall ability to perform tasks.We have tried to consider the user's self-efficacy in different aspects.Based on the interaction between humans and robots, we have utilised different means of communication, e.g., tablet verbal commands that have already been explained.Also, a person tracking and is developed using RGBD camera This visual system can robustly track and recognize persons within the camera's field of view.Then, the recorded person's physical characteristics and preferred choices (for interaction purposes) are called in the global intelligence layer.Thereby the global intelligence directs and informs the relevant robot that matches a person's ongoing disabilities by considering the requested task requests from the user.The tracked position of the person is transformed into a map that the robot utilizes for autonomous navigation.The classification and categorization of the environment information for global intelligence happen by utilizing the Yolo V3 [27] object detector.Also, there is a real-time safety model that we have developed.This safety model uses wireless IMU data from a wearable sensor attached to the person's chest to understand the safety dynamics of the person during motion.The model uses a spring-damperbased safety model with a dimension-reduced safety dataset of the individual [28].We integrated the safety model with our shared autonomy for certain robots e.g., walker and wheelchair, to understand whether the person is interacting safely in the environment (see Fig. 5).

VI. MULTI-ROBOT COOPERATION IN HUMAN-FRIENDLY ENVIRONMENT
For the robots to operate and navigate freely inside the Living Lab environment and respond to the user's request, we developed several new algorithms for their autonomous operation.The three main robots (wheelchair, walker and service robot) in the Living Lab environment are all equipped with sensors capable of performing point-to-point navigation within the fixed environment.They are equipped with range LIDARs for mapping and localization, and performing tasks such as dynamic obstacle avoidance and responding to user's calls when called for [29], [30].At the same time, each robot also has an external camera system for recognizing objects in space and performing high-level planning and object detection.The robots can also be tracked continuously using a motion capture system installed in the facility to get precise localization.As explained in the previous section, each robot works in decentralized architecture and can communicate with other robot platforms by sharing crucial information between the robots through message passing.Each of the robots has its own computing unit for processing the onboard sensor data, while all the processes requiring heavy computation, such as object labelling etc. are handled using an external high-end PC where all the sensor information can be processed smoothly.The general framework for the autonomous operation of the multi-robot system is given in Fig. 3.It uses a multi-layered scheme with one layer utilizing 2D or 3D maps for navigation and a high-level semantic layer for scene understanding [31].Places and objects are tagged with semantic information at the level, while low-level planning and metric goal-based navigation are done at the lower level.Additional high-level layers can be added in the framework based on application and use case scenarios, e.g.tagging objects with RFID information to help users suffering from weak memory or dementia to help find lost objects.This provides us with rich information about the continuously updated environment for a robust exchange of key messages between several robot systems using the proposed dedicated web server.
In the subsequent sections, we explain important autonomous tasks that the multi-robot system can perform.

A. Navigation and Planning
For robots to autonomously navigate between different target positions in an environment, mapping and localization are important.The process is termed Simultaneous Localization and Mapping or SLAM, where the robot has to actively map the environment and estimate its position in the built map based on the sensor information [32].We utilized opensource ROS packages to first map the Living Lab area as a 2D grid map and then used the map to perform active localization.This grid map is shared across all the robots working in the environment for navigation.Each robot also runs its own navigation stack (move base and local planner) that allows the robot to keep track of dynamic objects in the living lab over a period of time.This information is crucial to keep the map data up-to-date and relevant to the layout and obstacles in the Living Lab.Therefore, we utilize a map update process, where if an obstacle is observed over a set duration of time, its position is merged with the map and the new information is then shared across different robot platforms [33].Furthermore, as shown in Fig. 8, we test our algorithms on a simulated test environment, a digital twin that can verify the cooperative operation of multiple robots/sensors.For planning, key positions in the map, such as the bed, toilet, living room, base stations, etc. are stored as topological nodes for navigation [34].Before performing experiments with human subjects, each robot is thoroughly tuned and tested with its planners to avoid collisions with the subjects.A recovery behaviour planner is also considered for the robots to get out of hard situations (surrounded by persons) and robot-stuck scenarios (sensor failure).

B. Obstacle Avoidance and Shared Autonomy
During the planning, the robot can autonomously navigate towards the human while following and keeping the social behaviour as not to get too close to the user as shown in Fig. 7. Based on the sensor information from the onboard LIDARs the robot can keep a safe distance from static and dynamic obstacles while planning.When the subject gets too close to the robot, the planner pauses the current plan and stops moving.By calculating the cost values of the obstacles on the map, the robot then re-plans a new obstacle-free path or waits for the subject to move, and then continue with the planned task.
Motion assists or shared control is one of the ways to fill the gap between direct user control and the robot's safe intended trajectory.From a self-efficacy point of view, having shared autonomy helps the person have more control over his/her decision during the robot's motion.This fact is very important in active supporting robots i.e., wheelchairs or walkers.For example, in the case of a wheelchair user, the joystick input by the user might not be the safe route for the robot to take and the robot might need to follow certain points.
Also, recent researchers tried to combine sensor information to develop shared control strategies in mobile robot users, e.g., wheelchair system [35].In our framework, we utilized our designed assistive control that creates a safe and smooth control strategy which relies on the user inputs [36], [37].Also, we propose a new policy of shared autonomy for the human-robot interaction (shown in Fig. 5 and Fig. 7) that outputs the control velocities u(t) to the robot actuators as follows where n i ∈ [0 − 1], u i and m are the trust ratio, raw control input vector from different systems (planner, human joystick input etc.), and a maximum number of averaged trust ratio, respectively.For example, in our shared autonomy policy, we have considered human input, the planner, and the assistive controller [37], m = 3.Additionally, the trust ratio n i for each input is changing continuously by information of the deviation error from safety model [28] and existing obstacles in the path of the human with a robot in service, as shown in Fig. 7.

VII. EXPERIMENTS AND DISCUSSION
The CARE concept was exhaustively tested as a home care robot system under different use case scenarios.We briefly explain here and show some of the example scenarios with our integrated multi-robot ecosystem.In the end, we checked the quantitative and qualitative performance by doing an extensive questionnaire evaluation with around 80 participants from the engineering, health and social science fields.The whole experiment concentrates on a scenario where a user in-home care carries out his daily activities and how our CARE system supports the user in achieving his/her daily tasks.As a first effort, a series of scenarios were constructed to demonstrate our system from the perspective of "getting ready in the morning".The flow of the experiment scenarios is as follows: Readers are strongly recommended to watch the supplementary video from the link provided here to understand the context of the experiments and description provided below.Video Link: https://youtu.be/ItFXhY1zqg8 A. Experimental Scenarios 1) Getting Ready: In this scenario, as represented in Fig. 9, a user wakes and greets the computer intelligence with voice recognition).Our system keeps track of the person from an overhead camera, greets the person back using a voice assistant, and turns on the lights.Next, the global intelligence adjusts the bed for the person to ease him getting up from the bed.After that, the person requests a drink using voice assistance.The request is immediately processed, and the user is given a positive response using the AI speaker (computer).It asks for the user to wait, and meanwhile, a suitable robot is selected to execute the task using our multi-robot communication approach.Global intelligence can process where the request came from by analyzing voice data from the microphone and the user's position from external sensors.The target user is identified from the database and the selected service robot undocks from its base station and navigates autonomously utilizing the framework explained in 3).Notice that the response is immediate, and there are no delays in executing the task.Important locations of objects in the environment, such as the bed, are previously stored, and the goal is set based on where the person is sitting on the bed.Next, the service robot navigates through the environment carrying drinks, and the local intelligence for the automated bed with information from global intelligence adjusts the height to ease the patient to pick up the drink from the robot.The mobile robot is responsive to obstacles, including the patient (local planner), to keep the appropriate distance from the user.Finally, the global intelligence analyses the task completion using the external cameras on the scene and confirms the action was successfully executed (the robot reached the desired configuration).If no other request is in the queue, it commands the robot to return to the base station.
2) Go to living room: In the following scenario, the person sends requests using voice assistance about his desire to go to the living room.Global intelligence processes the person's statistics (level of walking discomfort and history of similar requests) and suggests that a walker robot will be suitable to use this time.Then, the global intelligence sends the request to the robot walker to complete the task.Based on the person's sitting position on the bed, an appropriate location where the walker robot should be sent is given as a goal.At the same time, global intelligence adjusts the automated bed height such that the person can comfortably get off the bed with maximum ease.It conveys to the user that the bed height is adjusted and the robot is on its way.The walker robot stops at the desired location and activates the brakes to avoid wheel slippage.Using camera information, the monitoring system in global intelligence can predict that the standing task is completed by utilizing the information from the force sensor plates installed under the bed.
Moreover, the global intelligence uses the force sensors, and IMU sensor [28] with a safety model on the walker handrail grip to confirm that the user is in the correct position and releases the brake.Then, the robot goes to shared control mode to assist and support the user to the intended place.In the next step, once the user has released the grip, the robot senses that the user has completed the task, and the computer sends the robot back to its base station (as shown in Fig. 10, t = 2 : 53 s time-stamp).
To demonstrate how our system can work along with a human caretaker, another experiment is conducted as a continuation to the scenario.This scene introduces a human caretaker with control of the robots' operation.This experiment aims to demonstrate how the system can take commands from different users and distinguish between the care receiver and caregiver.The global intelligence keeps a database of different users and, based on where the request is coming from (AI-based voice synthesis, microphone localization, and image recognition), completes the request with the most appropriate selection.The snapshot t = 3 : 44 s in Fig. 10 demonstrates some sensory systems information from the scene.Here is an environment classifier; we utilize the Yolo v3 deep learning network [27] to distinguish objects in the scene, and the classifier gives the semantic information of the objects and people.The system tracks the two persons on the scene and can pinpoint the request source.For example, the caretaker wishes to send the medicine to the care receiver.In the scene, it calls the robot first to its location.The person's position is extracted by information matching and gathering from where the request originated, picks an appropriate robot for the task, and sends the robot to the caller's location.Next, the caretaker puts the medicine on the service robot and asks the global intelligence to send it to another person.Global intelligence accepts the request, processes the voice command for key information (e.g., person's name), and utilizes the camera network and the stored database to recognize the person and their position on the map.It then sends the same robot to the other person and waits for the task to be completed.All the process happens instantaneously, and there is no delay in the communication and message exchange with our proposed system.
3) Going outside: This scenario demonstrates how the CARE system can be flexibly extended to cases even outside the boundaries of the home, giving the user freedom to use robots outside and increasing their self-efficacy in everyday tasks.In our experiments, we consider different case scenarios where the user requests the robot to take them outside the house (e.g., for shopping).In this scenario, a user desires to go outside and requests the computer (global intelligence) for help.Then, the computer processes the request based on individual characteristics, preference, health status, and previous history and suggests to the user to use a wheelchair and prepares to bring the supporting wheelchair to the entrance of the living lab.Our system can also make other recommendations, such as a walker robot based on the user's health conditions that are continuously updated.For example, suppose the computation based on captured sensory data shows that the user has not walked a lot in a while.In    In Fig. 11, at t = 4 : 38 s, the wheelchair is called to the entrance from the outside.The wheelchair navigates the entrance using the onboard sensors.The wheelchair robot has an RGBD camera that uses the classifier for more information about the environment.After the user sits in the wheelchair, the robot takes the person safely to the front of the elevator.Next, the elevator is called for the user using automated switches that can be triggered as the person arrives near the elevator.Finally, with existing shared control for the user, the user starts to move the robot to the desired place in the elevator.In this way, our system can design the behaviour of the AI robot that can assist the user in areas beyond the boundaries of one's home such that the user can feel "I can do it by himself/herself" if he/she works with the AI robot.

B. Expert Evaluation
To understand the potential of our framework and compare the concept from different perspectives of the experts in the field, we have developed a new 9-point evaluation.The method is inspired by the NASA-TLX questionnaire with the aim to have a qualitative and quantitative performance evaluation based on the third eye analysis by different experts in the field.We have considered respondents in engineering, health professionals, and social scientists as well as the public (others), with around 80 participants in total with approximately equal distribution.Over 58.8 % of our participants are aged over 35 and 26.3 % are over the age of 45, which has a high potential to consider the following framework as a technology incubator, incremental research, or a consumer.The questionnaire is designed with an inverse questioning form to avoid biases by the participants with both in-favour and disfavour.The questionnaire was taken for a period of over one week.Please refer to the Dropbox link for details of questions.
After preparing the data in spiderweb 9-point scoring format, we can see the results in Fig. 12.It is important to note that in our 9-point data evaluation, we have considered our framework from three main factors engineering (intelligence, usefulness and convenience), health (frustration, physical and mental demands) and social (anxiety, sociability, and trust) factors.Each expert is asked concerning all questions with values between [1][2][3][4][5][6][7][8][9][10], where value 5 presents a neutral opinion with respect to the question and the lower the value, the more favourable the score is.We can see from Fig. 12-a, that the academics and professionals from the engineering field show great interest in the potential of the proposed framework from an engineering point of view (engineering (1) mean m = 3.607 and standard deviation δ = 1.93), e.g., performance and usefulness.However, they found the system still open to be improved in trust, sociability (social (3), m = 4.117 and δ = 1.91) and frustration (health (2), values around 4).More interestingly, the second group that agrees with the potential of the CARE framework is the health sector.They show high interest in the practicality of the robots in the framework with a slight increase of value by 0.3 scores (engineering (1), m = 3.92 and δ = 1.796).However, they found the trust and sociability (social (3), m = 4.608 and δ = 1.53) hard to score and not enough as highly positive.This clearly shows potential for improvement, for example, by working on human-robot interaction and human psychology to make the person feel more relaxed and keep the user in the loop of what is happening.They also commented on giving more freedom in human-robot interaction where the robot or global intelligence respects the human directly/indirectly.The group pointed out that some patients might require slower and smoother interaction with the robot due to disabilities, e.g., dysarthria or intractable diseases, or even slower motor function due to age issues.This will require global intelligence to build a case-specific user assist that will be one of our future aims.From the social scientist's point-of-view, they had the overall neutral assessment of the robot's capabilities (health (2), m = 4.911 and δ = 1.84 and social (3), m = 4.843 and δ = 1.483) but showed interest in the engineering application of the framework (engineering (1), m = 4.41 and δ = 2.106).However, they have similar concerns regarding the framework's sociability, frustration, and physical demand.To generalize the overview, we check the scores by including all the experts in the field which results in Fig. 12-b.The participants show scores around 4 for health (m = 4.425 and δ= 1.873) and engineering (m = 3.98 and δ= 1.92 ) factors, but they still found space for improvement regarding social factors such as anxiety, sociability and trust (m = 4.693).This issue was aligned with other sectors that still find it challenging to understand the human-robot interaction with certain levels of anxiety.

VIII. CONCLUSIONS AND FUTURE WORKS
The use of assistive robots to support elders and caregivers is an inspiring and inevitable goal in robotics research and development.However, this requires a fundamental concept that future research could build on.
This article proposes a framework for future welfare facilities with a new concept of Living Lab.First, we explained the newly initiated Moonshot Project R & D in Japan, where the following idea is under the sub-group of Cooperation of AI robot Enablers (CARE).Then, we present the framework's main idea by explaining the assistive and service robots and the sensory system.We describe our framework as an adaptable AI that supports the user by sensing the physical conditions, expressions, surroundings, and daily conditions and providing the most appropriate support by transforming it into several robot systems.The global intelligence utilizes an innovative multi-robot IoT interface, and sensory feeds try to correspond to the user's requests by selecting the best options.The aim is to develop an assistive multi-robot cooperative system that promotes the self-efficacy of the user, e.g., safety, individual characteristics, health conditions, and preferences.This framework is explained in different levels of computation from the robot's navigation to the chosen strategy in the communication interface.The system can also aid in helping people from forgetting or mistaking their medicines and provide feedback between family members, friends, and caregiving staff about the same.We also presented a new webserver system for multi-robot communication and sharing of messages between different agents in realtime.Such systems in the home care scenario can provide appropriate support to the user by distributing tasks to multirobot systems where each robot can do a specific task.Unlike a single robot system, our proposed system is less complex, and task distribution and allotment between the robots can be efficiently handled without hindering the overall task at hand.The robots work with humans under the shared autonomy policy that considers the person's safety and obstacles in the area with our proposed sensory information in the facility along with wearable sensors.We did extensive questionnaire evaluation with a new 9-point scale.From the evaluation results, the experts in the field of health and engineering expressed keen interest in the potential of the robot, however, they highlighted open challenges that still exist regarding anxiety, trust and sociability factors.Also, certain experts found it frustrating, which could directly correlate with sociability.The social scientists also had a certain level of interest but indicated their concerns regarding social interactions and physical demands.
Although the concept was successfully presented with several demos in real-world cases, many problems should be addressed in the home care robot scenarios.These problems can be grouped into different levels of engineering, social science, and artificial intelligence aspects.From the engineering aspect, the safety, real-time risk assessment and development of advanced actuating mechanisms in assisting people with higher disabilities is challenging.Furthermore, there is a significant number of research on developing different assistive mechanisms to help the patient from different aspects, such as motion assistance, toilet support, or doing more challenging tasks in severely disabled patients such as transfer support.Also, there are studies on understanding the ethical and psychological issues regarding robot and human interaction in social science.For example, how will the robot understand the person's needs, and to what extent can it support by providing physical and emotional support?Can robots understand human stress and experience level with robots?What if the elderly person cannot get used to the way robots interact with him/her.How to make a quantification for evaluating ethical issues at the low level?Data protection and information privacy is another issue where an informed consent-based evaluation of a user's request by the AI is made.On the side of artificial intelligence, the problems can be focused on global intelligence, improving robot understanding of high-level instructions from the user in human-friendly ways.For example, in a scene where not only patients but also everyday people and other professionals are in the scene, the AI should be able to respond and react based on the person's intention, occupation, and interaction with counterparts.

Fig. 1 .
Fig. 1.The proposed CARE framework for the multi-robot cooperation in homecare scenario.

Fig. 5 .
Fig. 5.The diagram of CARE as the autonomous multi-robot cooperation.

Fig. 6 .
Fig. 6.The person's health and physical information are gathered with different sensors.

Fig. 7 .
Fig. 7.The human-friendly environment navigation with shared autonomy and predictive safety model.

TABLE I THE
COMPARISON OF DIFFERENT PROJECTS WITH CARE FRAMEWORK.