Microsoft has developed a chatbot that can comment on what it sees

Microsoft has released a new full duplex chatbot in Japan that takes digital assistants to a whole new level (assuming it works at least). The new chatbot features a new AI called “Rinna”, who can interact with your everyday life. By interact, I mean HAL 9000 (“I’m sorry Dave, I can’t do that”) level of interact, as it can see and respond to your environment, including the ability to identify things as you walk by and share opinions or thoughts with the user as it happens.
 
The best way to describe it is for anime fans who have experience with series like Sword Art Online and the little digital (AI) adopted daughter (Yui) of the main two characters that follows them around on their shoulders (whose character can be compared to similar characters in many other series). Only instead, this is just your smartphone hanging out in your chest pocket (mmm, feel that radiation).
 
Rinna can see the world around you as you walk around, and is able to listen in on everything as well. She (Rinna) can then chat with you about various things like signs, people, places and more. It uses an “Empathy Vision Model” that combines image recognition with emotional responses.
 
The whole point of keeping your phone in your chest pocket is so that the camera can stick out the top with a view of everything around you. Rinna will process everything in real time and comment on things she sees. So if you are looking for something specific for lunch, she might be able to help you find it. It’s like having friend that follows you everywhere to interact with you on daily experiences vs a plain assistant that just manages your schedule and messaging (or for Japan, a new digital girlfriend).
 
 

---