Player is loading...
PRA: The Personal Robot Assistant, pitch
Emmanuel SENFT, Idiap Research Institute
Thursday, Aug. 24, 2023 · 12:46 p.m. · 07m 18s
This project aims to develop an AI-based controller for the Lio robot that performs object manipulations tasks (moving objects) from human-voice instructions. By integrating multiple AI models, including large language models (LLM) and generative models, we aim to enable robots to understand voice instructions and execute the corresponding object manipulation tasks without training the policies for the manipulation tasks. Team members: Emmanuel Senft, André Mayoraz, Chaoran Zhu, Junyoung Seo
Embed
Transcriptions
Note: this content has been automatically generated.
00:00:01
good okay right but i mean it didn't work on yeah i that's what e. b. e.
00:00:15
we had one and you want to do basically to make what actions simple
00:00:22
right now it's very complicated no to me to be does that make it
00:00:26
as easy as possible as easy to use today to use chat g. e.
00:00:32
we have this image of a challenge today which is a vision he's eighteen like never before so i think
00:00:41
oh yeah the
00:00:44
it's a big but what is coming soon
00:00:48
alright six eighteen population so see a vision is getting much older now uh
00:00:54
we are in while having to hide around certain people above sixty by twenty city
00:00:59
and at the same time we either decrees of so work falls to be available
00:01:03
so we are more people need to kill off unless people gonna work for that
00:01:07
as if he was actually the perfect working t. to apply robotic distillation
00:01:11
we can have a robotics are but system providing support for health
00:01:15
care providers and also opposed to work falls but we social thirteen workforce
00:01:20
how about some the limitation at the moment uh first uh robots
00:01:25
currently are only deployed instruction on violence seconds with the handle unstructured environment
00:01:31
uh so only working very specific tasks that the program
00:01:35
and every time you want to chance of passage to reprogram
00:01:38
yes that's going to create a big button it to use robots today and when you look at the cool methods to
00:01:43
date program robots have option one code to to spend lots
00:01:47
of time to learn out code and to understand how robots work
00:01:51
a method to uh he's doing lenny from demonstration so you hold of a button can demonstrate
00:01:56
some the task has to do and so it's just something to the everywhere and used today
00:02:01
and methods reverses something which is it's a product which came out just a six year is vision interfaces
00:02:07
and when you see now that's just method very simple and easy as just to talk to
00:02:12
or what to get to do things as a bit is that that's what we want to do
00:02:16
uh we want to allow people to control what what's buy
00:02:19
votes especially working with a constraint instructions working for unspecified tasks working
00:02:26
in a unstructured environment and family working with unknown objects so that's
00:02:31
why we are here presenting do it as tragically t. four bucks
00:02:37
so i'll position is basically pretty simple is to have
00:02:40
human like interaction based on that and and and and also
00:02:44
to from two from says ellen's you'll gentleness too new
00:02:48
passed in people to be reduced to a variety of instructions
00:02:53
approaches you how the user with a robot as what about just here and the user can say a sentence which is
00:02:59
not the specific as uh we used contract which uh we've as i stands while back so a sentence
00:03:05
could be oh i will drink anymore can it just gets a bottle away and placed onto the table
00:03:10
something that we lose you know that's an constrained uh well then was the brains robot take its information isn't and i can
00:03:17
get into robotic actions dozens of about is finding out what to do like look it's can move just to to pick it up
00:03:25
and then look at the table was moved just we haven't droppings um objects it and so
00:03:30
here we have a team of four people so we are actually two and a computer vision
00:03:36
uh students while twenties expect he's on some understanding and how to combine vision we've language
00:03:42
so uh you need a tune young and charm uh coming from queen mary and we have on how many are in myself in an l.
00:03:49
uh from it yeah looking at the robotic aspect how we can get sees actions and and actually get or what it
00:03:55
and finally we have our most important member he of the team had then then our lovely or what
00:04:01
and we had to working prototype uh says an example e. only just save your coffee place which
00:04:09
is basically so request we have some last for about it so here's what is interpreting what's happening
00:04:17
i just said i will have go to bring his coffee so what gaps that uh only want to the coffee and then is that it it affected
00:04:23
just let me throw sort of machine here we have a coffee already have ready which could be done if
00:04:28
we had to do to give machine is uh what is driving back and impact one way and then eventually
00:04:36
and just have to make to release it it's we have actually on our about some uh sends them to people that can be used to happens at
00:04:45
uh that's when examples actually asking is also multilingual so this is another case
00:04:53
e. o. some can be here it but i basically and saying
00:04:57
in french uh as if they did what they can expect that level
00:05:02
and says answer but understand it it has to pick a bottle it can look at the bottom the scene get close grasp it it
00:05:09
knows where's the table it and try to table and then drops
00:05:11
of levels it just to show some examples of a working system here
00:05:17
so what is the the whole well i'm not setting robots buttons working on software it's we're looking at licensing
00:05:22
so we have to focus on is basically what the companies and integrators so integrated into one building
00:05:29
all buttons are buying robots instances and putting everything together to be able to deliver studies to client
00:05:35
and then we have uh i'll end uses so we're targeting basically full segments when it's posted that it
00:05:40
select the hotels in company a s. m. u. sex small and medium enterprises
00:05:45
uh have institutions and also in a little at home with all the robot could be
00:05:49
paid by instances and if we look and this is the market if by twenty thirty
00:05:55
what's making all the total billions of dollars for the robotic market that
00:06:00
only only work on the software and certain markets only forty five billions
00:06:04
and increased six four from today around seven billions to lesser space
00:06:07
to fall at people want to work on the brain of robots
00:06:11
and we have a potential client three well only cavorting with the naked
00:06:15
that's what we could be interested to license up technology pretty good actually
00:06:21
that's with the i'll contractions like we don't do it at ten to put the rubber uh for but that's what we
00:06:27
did is nine days when asked that they want to work
00:06:29
with them constraints instructions into will be able to with unknown objects
00:06:35
integrating navigation and manipulation and funny also working on our i i. p. development and
00:06:41
um and then where is the model and combining language in addition to get that
00:06:46
uh and the next steps that we are planning to but complete
00:06:49
lies and definitely that that didn't dissuade effect finalise i'll call id
00:06:54
a working to preserve our businesses that has to be honest doesn't work every time at the moment
00:06:59
and maybe signed a contract with uh our departments which would bring
00:07:04
in that we could get some actual revenue by twenty twenty four
Conference Program
ICC'2023 welcome introduction, 1st pitch of the finals and demo of the team "SLIP"
Julien Battaglia, SLIP: Solutions for Liquidity and Investment Planning
Aug. 24, 2023 · noon
843 views
