Modern autonomous vehicles are getting pretty darn good at seeing the world around them. That is, assuming that lighting conditions are ideal. Once rain, snow, or sudden bursts of bright light from first-response vehicles enter the equation, things start to get a bit dicey. A tiny new sensor component that is roughly the size of a grain of sand, could help solve that problem.
Called a photomemristor, the new sensor was engineered by researchers at Penn State. It breaks from the traditional approach to computer vision sensors, and instead takes inspiration from good old-fashioned human eyeballs. In essence, it is akin to extra artificial eyes. In testing, the device adjusted between bright and dark lighting environments faster than contemporary methods.
Human eyes easily and innately transition between light and dark. Building that same ability in future autonomous vehicles could make them more reliable, even in inclement weather. That extra help could go a long way, especially as robotaxi companies like Waymo and Zoox prepare to put more and more of their driverless cars on public roads in the United States and abroad. The findings were published this week in the journal Nature Communications.
“By mimicking the way the eye works, we can create photomemristors that work much more reliably for applications in mixed lighting environments,” Larry Chang, an engineer at Penn State and a study co-author, said in a statement.
Where computer vision falls short
Driverless car vision models (and all computer vision systems, for that matter) are only as good as the data that they’re trained on. Though there’s been considerable effort to improve performance in bad weather and odd lighting environments, a quick look at the cities where driverless services are currently available tells a familiar story. Phoenix, San Francisco, Austin all are known for their long, sunny days.
But weather isn’t the only factor that can trip up a computer’s vision. More specifically, those systems can struggle in what the researchers behind this new sensor call mixed lighting conditions. One example of mixed lighting is something most drivers are familiar with—driving down a long, hilly road in the dark, only to have a car pop up on the other side of the lane with its high beams on. That sudden jolt from darkness to the light of the other car’s headlights and back to dark again is disorienting, but most human drivers can manage it and still keep some awareness of their surroundings.
That’s more difficult for machines. In that same scenario, the team notes, a driverless car stunned by the flash of an oncoming high beam may briefly lose track of other figures in its vicinity. That could include the faint hue of a red stoplight or the blurry outline of a deer scurrying by.
Using rods and cones
To address that issue, the Penn State engineers went back to basics and looked at what makes the human eye work well in that scenario. Our eyes contain rods and cones that help distinguish details in the dark. When a light source suddenly gets brighter, the pigments in the rods get temporarily “bleached” and slowly regenerate. The cone cells, meanwhile, stay unchanged during that process, which is what helps us keep track of contrasting details while the rods readjust.
With that naturally occurring process as inspiration, the team set out to build a custom-made photomemristor meant to more or less copy the interplay between the rods and cones in human eyes. They built the sensor out of two materials, a stretchy, gel-like plastic and a powdery compound called titanium oxide, with water flowing between them. The titanium oxide captures light from the environment, which is then converted into an electrical current. That voltage is then passed through the plastic’s conductive surface. In practice, the plastic would absorb more water and slightly swell up in darker conditions, while exposure to more light would cause it to desorb water. The idea, the team notes, was to create eye-like sensors that can “dynamically adapt to changing light conditions.
The device itself looks like a gold square, with a smaller square inside it and tiny holes dotted throughout. It’s also minuscule: one square measures just half a millimeter across, making it thinner than most credit cards. For this type of sensor to actually work in a computer vision system, several pieces would need to be connected to form arrays.
To test it, the team combined several of the sensors into a 4×4 array. The array was then paired with a neural network, which would act like the computerized brain in a driverless car or robot. Once combined, the team ran their new machine vision system through variations of a standard eye exam you might see at the optometrist. They placed an LED letter “F” against a background with lighting the team could control. The machine vision system, outfitted with the artificial eyes, had to keep track of and identify the F as the background switched from extremely bright to extremely dark.
After some initial training rounds, the system reported 95 percent accuracy in identifying the letter under mixed lighting conditions. That, the team said, outperformed traditional systems. While it still might not score quite as well as some humans on that test, it did have another leg up.Human eyes typically take somewhere between 20 and 30 minutes to fully adjust to major changes in light, but the team notes the system was able to adjust in a matter of seconds.
Seeing beyond cars
Though the artificial eyes worked well in this narrow test, including the sensors in actual cars you may see on the road is still a long way off. The team says the next step involves expanding the sensor set into a multimodal system capable of processing both visual and tactile data simultaneously. Eventually, though, they are hopeful this could help autonomous vehicles see a bit more reliably.
Beyond that, the team even thinks it’s possible a version of these artificial eyes could help create artificial optics that could provide renewed sight for visually impaired people. Those same eyes, they say, could also slot into humanoid robots to help them better navigate warehouses or other facilities where people normally work. That all sounds quite impressive, though it also starts sounding eerily similar to something out of the dystopian video game “Cyberpunk 2077.”
The post How can self-driving cars see better? Make their sensors more human. appeared first on Popular Science.
from Popular Science https://ift.tt/WqPQ5Tg
0 Comments