Tesla's Full Self Driving System Mistakes the Moon(!) for Yellow Traffic Light

• Tesla's Full Self-Driving system, offered for purchase or via a monthly subscription, offers traffic light and stop sign recognition.

• Speed limit and traffic light recognition features debuted as part of FSD in Spring 2020.

• Tesla has begun offering FSD on a subscription basis, but older Tesla models may also require a $1000 hardware upgrade.

Tesla's Autopilot driver-assist system has been on the market long enough to generate plenty of data about its ability to identify (of not) impending obstacles, from other vehicles to various concrete structures. After several years on the market, most of the items that tend to confuse Autopilot on a regular basis have been fairly well studied, if not completely eliminated from the roster of things that tend to produce unexpected reactions from the system. Some of the more consequential items have tended to be road lane markings, since Autopilot has relies on them to steer itself within a lane. Fire trucks parked in highway lanes while responding to emergency calls have been another not uncommon foe for the system.

One issue we hadn't really seen until now—because traffic sign recognition has not been activated for a significant period of time as a part of Tesla's Full Self-Driving suite—is the system mistaking the moon for a yellow traffic light.

A Tesla owner recently posted a video showing the Full Self-Driving system confusing the Moon for a yellow traffic light, which was prompting the car to slow down.

Of course, the yellow tint of the Moon could be related to wildfire smoke in the atmosphere over parts of the US, so perhaps this issue won't be one of any regularity, but we had wondered in the past about various semi-autonomous systems' abilities to identify and correctly distinguish and respond to traffic lights, as such systems have arrived in production passenger cars. Tesla's system is not based on camera sight alone, as it also relies on map data of intersection and light locations, with the system designed to slow the vehicle down for all detected lights.

If anything, we'd be more concerned about such systems correctly responding to traffic lights that apply specifically to them and cars in their lanes, as some intersections can be very complex or positioned too close to each other for traffic light recognition systems to pick out the correct ones. As we all know, traffic lights can at times be positioned in front of other more distant traffic lights, or amid a jumble of other traffic signs, as in the real world intersections can have many different lights applying to various lanes of traffic. Autonomous developers have also had to contend with intersections where traffic light positions are either too high and too close to the front of the vehicle to be seen by the camera-based systems interpreting them. Various light conditions can also easily interfere with all camera-based systems, not just Tesla's.

One of the ways in which automakers and autonomous system developers have sought to bypass this issue entirely is though traffic lights that communicate with the systems in the cars themselves, via vehicle-to-infrastructure (V2I) technology. For example, Audi's Traffic Light Information system, deployed in certain cities since 2016, relies on real-time signal information from a traffic management system via an 4G LTE data connection.

"When the light is red, the TLI feature will display the time remaining until the signal changes to green in the instrument cluster in front of the driver or in the head-up display (if equipped). This 'time-to-green' information helps reduce stress by letting the driver know approximately how much time remains before the light changes," Audi says.

Of course's Audi's system is far from functioning at every intersection, as it's hardware-dependent.

Tesla's system, on the other hand, relies on visual interpretation of the lights rather than V2I technology, and doesn't rely on signals to the traffic lights themselves. It's more versatile, as we have seen, but also somewhat more prone to misinterpreting them.

Of course, one other concern with various systems continuously misinterpreting traffic lights, not just Tesla's system, is that if the vehicle makes braking and acceleration decisions based on erroneous sensor data, then it can reduce its speed on the highway and possibly prompt a vehicle behind it to rear-end it. And as much as the Moon can be mistaken for a yellow traffic light, we have a feeling that the sun could be a much more frequent culprit of the same phenomenon, in addition to other circular lights such as overhead street lighting.