Welt

Recent research on Robotics and gen ai

Octo: Generalist Robot Policy (Gen AI in Robotics)

Imagine a GPT for robots! That's what this paper is aiming for—a generalist robot that can perform a wide range of tasks. Right now, we have robots designed for specific, narrow tasks. This research is a step towards creating robots that can handle anything you throw at them

What’s a Generalist Robot Policy?

In simple terms, a policy is a neural network that decides what action a robot should take. It takes in observations (like text and images) and outputs control units.

Types of Control Units

  1. Joint Control: The model specifies exactly what each joint should do (like the angle of an elbow).
  2. End Effector: This is a fancy name for the gripper. The model outputs movement and rotation vectors for it.

The Process

  1. Input: Task tokens (text), observation tokens (images), and readout tokens (a summary of task and observation tokens).
  2. Output: Control units from the action head (a diffusion model).

Challenges

  • Speed: Transformers are powerful but can be slow due to processing limitations. MLPs (Multi-Layer Perceptrons) can be faster but lack the generalization needed for different robots.
  • Data: Currently, the available dataset is limited. However, with access to more data, we can expect these policies to become much more efficient.

In essence, this paper is pushing the boundaries in the field of robotics, aiming to create versatile robots capable of performing a wide array of tasks.

link to the paper: https://arxiv.org/pdf/2405.12213

If you want me dive deeper,do let me know

5mo ago
Spectrum
Spectrum
Akamai Technologies5mo

I am a deep learning practitioner with gen ai skills in language and vision (LLMs and Diffusers). Any suggestions on how I could ramp up skills in robotics policies, etc.?

Welt
Welt
Quantiphi5mo

If you are in india,there are very few companies that are working in robotics because of the lack of funding and support from the govt and the profit mindset of investors.

You can start with ROS 2 tutorials just to understand how to write code in simulation.You can check out Ros 2 wiki to get started.

The paper I mentioned above is in the research phase,only companies with access to large datasets can work on these experiments.

Upskill yourself in gen ai,get some professional experience in robotics by hopping companies.

I am pretty sure the opportunities will increase in the future.

Discover more
Curated from across