It feels like nearly every week or so, someone’s quadrupedal robot gains yet another impressive (occasionally terrifying) ability or trick. But as cool as a Boston Dynamics Spot bot’s new capability may be, it’s hard to reliably compare newly developed talents to others when there still aren’t any industry standard metrics.
Knowing this, a team of research scientists at Google are aiming to streamline evaluations through their new system that’s as ingenious as it is obvious: robot obstacle courses akin to dog agility competitions. It’s time to stretch those robotic limbs and ready the next generation of four-legged machines for Barkour.
[Related: This robot dog learned a new trick—balancing like a cat.]
“[W]hile researchers have enabled robots to hike or jump over some obstacles, there is still no generally accepted benchmark that comprehensively measures robot agility or mobility,” the team explained in a blog post published last week. “In contrast, benchmarks are driving forces behind the development of machine learning, such as ImageNet for computer vision, and OpenAI Gym for reinforcement learning (RL).” As such, “Barkour: Benchmarking Animal-level Agility with Quadruped Robots” aims to rectify that missing piece of research.
In simple terms, the Barkour agility course is nearly identical to many dog courses, albeit much more compact at 5-by-5 meters to allow for easy setup in labs. The current standard version includes four unique obstacles—a line of poles to weave between, an A-frame structure to climb up and down, a 0.5m broad jump, and finally, a step up onto an end table.
To make sure the Barkour setup was fair to robots mimicking dogs, the team first offered up the space to actual canines—in this case, a small group of “dooglers,” aka Google employees’ own four-legged friends. According to the team, small dogs managed to complete the course in around 10 seconds, while robots usually take about double that time.
[Related: Dogs can understand more complex words than we thought.]
Scoring occurs between 0 and 1 for each obstacle, and is based on target times set for small dogs in novice agility competitions (around 1.7m/s). In all, each quadrupedal robot must complete all five challenges, but is given penalties for failing, skipping stations, or maneuvering too slowly through the course.
“We believe that developing a benchmark for legged robotics is an important first step in quantifying progress toward animal-level agility,” explained the team, adding that, moving forward, the Barkour system potentially offers industry researchers an “easily customizable” benchmark.
