{"id":9,"date":"2015-11-17T21:13:24","date_gmt":"2015-11-17T20:13:24","guid":{"rendered":"https:\/\/project.dke.maastrichtuniversity.nl\/brain-robot\/?page_id=9"},"modified":"2018-03-06T09:50:24","modified_gmt":"2018-03-06T08:50:24","slug":"fnirs-brain-robot-interface","status":"publish","type":"page","link":"https:\/\/project.dke.maastrichtuniversity.nl\/brain-robot\/fnirs-brain-robot-interface\/","title":{"rendered":"fNIRS Brain-Robot Interface"},"content":{"rendered":"

Capturing user intentions and turning them into actions<\/em>.<\/p>\n

Brain-Computer Interfaces (BCIs) are playing an increasingly important role in a broad spectrum of applications in health, industry, education, and entertainment. We present a novel, mobile and non-invasive BCI for advanced robot control that is based on a brain imaging method known as functional near-infrared spectroscopy (fNIRS). This BCI is based on the concept of \u201cautomated autonomous intention execution\u201d (AutInEx), that is, the automated execution of possibly very complex actions and action sequences intended by a human through an autonomous robot.<\/p>\n

Goal<\/h3>\n

Develop a reliable framework for a real-time brain-robot interface (BRI) based on fNIRS.<\/p>\n

Progress<\/strong><\/h3>\n

The work began with the design of the interface architecture. This architecture was intended to build-up the connectivity between the logically and physically separable procedures of Data collection, Data processing, and Action control. In the Data collection, the user of the BRI had to be recorded, according to a prepared acquisition protocol, whilst being informed about the recording state. In the Action control, the robot should be acting autonomously in the given environment and be capable executing the user-specified tasks. Finally, the Data processing should glue the other two procedures together in such a way that the brain data recorded from the user is correctly and in a timely manner mapped to the tasks the robot would perform. The initial architecture was as shown below:<\/p>\n

\"BCI_Schema\"<\/a>
\n<\/a>The architecture above was first tested in an offline setting – i.e. by simulating the recording of a real subject with the stream of the prerecorded data. Thus, originally, the Data acquisition block was entirelly replaced by a set of raw data logs. An offline version of TurboSatori<\/a> (TSI) was used to load the logs, do the data preprocessing and pass it further for the deeper analysis. The offline\u00a0version of ChannelChain<\/a> was used to perform a task\/encoding mapping between the experiment protocol the user had to follow and the available robotic tasks, then to do classification of th e data received from TSI and output the decoded task selection to Robot Operating System<\/a> (ROS) controller. The ROS controller was aimed to manage the robot task performance.<\/p>\n

Once the offline tests indicated that the framework was functioning correctly, the setup was transferred to the online setting. The classification process was improved and the user feedback software RoboFeedback<\/a> was introduced. The RoboFeedback allows to monitor and control the robot placed in the ROS environment. This module is essential in bridging the gap between the user, who might not even see the task being performed, and the robot.<\/p>\n

Then, a few recording experiments were done using the designed real-time framework. For better impression of what the recording session is see this conceptual video:<\/p>\n