Disclaimer: This article has been automatically aggregated from
Researchers in China have developed a sensor that could reduce the amount of energy that drones and other autonomous hardware use to process visual data.
In a study published Aug. 19 in the journal Nature Sensors, scientists detailed the architecture of a new 2D chip dubbed “LightTok,” which can convert raw light into tokens used by artificial intelligence (AI) models on the chip itself. The device skips several energy-intensive steps that conventional sensor systems depend on, the scientists said.
“Our design idea was to move token generation onto the sensor itself, allowing the chip to directly produce tokens that AI models can process once light reaches the sensor,” Miao Feng, director of Nanjing University’s Institute of Brain-Inspired Intelligence, said in a statement. “These tokens contain complete image information.”
Latest Videos FromLive Science
Collapsing the visual perception process
In conventional visual perception, light signals have to go through multiple stages. A sensor captures the signals, and an analog-to-digital converter turns the light into pixels. That data is then stored temporarily before it’s shuttled to a separate chip, where the image is cut into square “compartments” — much like dividing a photo into a grid of tiles. Each compartment is then converted into a token for an AI model to read.
One study widely cited by other academics found that an analog-to-digital converter is responsible for 66% of an image sensor’s energy consumption, on average. Moving visual data processing off the chip and into the cloud can increase overall energy consumption further.
LightTok’s solution to the energy problem is to collapse the five stages into one by building the sensing, memory and computation processes into the same pixel. The researchers achieved this through an array based on a technology called a single-layer molybdenum disulfide floating-gate phototransistor that can sense light, remember what it sensed, and then factor that into a calculation.
The chip physically eliminates data movement, which is the main source of energy waste.
Liang Shi-Jun, physics professor at Nanjing University
Molybdenum disulfide is a 2D material that reacts well to light and can be grown in sheets one atom thick. The phototransistor converts incoming photons — particles of light — into an electrical current, while the floating gate is an isolated component inside the phototransistor that can trap and hold an electrical charge, rather than disappearing after the light goes away.
Get the world’s most fascinating discoveries delivered straight to your inbox.
LiangShi-Jun, a physics professor at Nanjing University, summarized the chip to Chinese state-run news agency Xinhua. “The chip physically eliminates data movement, which is the main source of energy waste,” Liang said. “Light comes in, tokens come out” — hence the name “LightTok.”
LightTok achieved 87.3% accuracy in image recognition during tests — compared to the conventional, multi-step process described above —- while being 10 times more energy efficient at converting light into tokens, the researchers reported in the study.
A more efficient physical world
At present, LightTok’s maximum resolution is just 32 by 32 photosensitive pixels — the light-sensitive compartments on an image sensor that capture the visual data. This is far inferior to the quality of current smartphone cameras, let alone drones and autonomous hardware.
Nevertheless, Miao said in the statement that there’s an opportunity for the technology to be scaled up using the complementary metal-oxide-semiconductor manufacturing process — the same method used to fabricate chips found in smartphones and laptops as well as sensors in drones.
The researchers believe that if LightTok can eventually scale successfully, it could transform the operation of remote sensing technology. For example, a drone scanning a disaster zone or remote area could potentially fly longer because less energy is required for visual processing.
Kumar Sokka, CEO of Acre Security, a company that provides real-world sensing for critical infrastructure, described the work as a “small-scale demonstration.” However, “the direction [of the research] matters to anyone working in the physical world,” added Sokka, who was not involved in the new research.
Sokka, who previously spent 15 years at industrial automation company Rockwell Automation, told Live Science in an email that too much of the conversation around physical AI has centered around AI models. In truth, the bigger challenge has been the energy cost of “getting what a sensor sees into a form a model can actually use, right where the sensing happens,” he said.
The massive amount of energy typically required to turn raw data, such as light, into tokens “is wasteful when you’re running perception on a robot or an edge device with a tight power budget,” Sokka noted. However, processing at the point of detection may be “an enabler for pervasive physical AI, and a clever one, but it’s not a cure for the whole problem,” he added.
Can you match these ancient devices to their pictures? Find out with our computing quiz!
for informational purposes only. We do not claim ownership, accuracy, or liability for the content provided. All rights belong to the original publisher.
Contains information related to marketing campaigns of the user. These are shared with Google AdWords / Google Ads when the Google Ads and Google Analytics accounts are linked together.
90 days
__utma
ID used to identify users and sessions
2 years after last activity
__utmt
Used to monitor number of Google Analytics server requests
10 minutes
__utmb
Used to distinguish new sessions and visits. This cookie is set when the GA.js javascript library is loaded and there is no existing __utmb cookie. The cookie is updated every time data is sent to the Google Analytics server.
30 minutes after last activity
__utmc
Used only with old Urchin versions of Google Analytics and not with GA.js. Was used to distinguish between new sessions and visits at the end of a session.
End of session (browser)
__utmz
Contains information about the traffic source or campaign that directed user to the website. The cookie is set when the GA.js javascript is loaded and updated when data is sent to the Google Anaytics server
6 months after last activity
__utmv
Contains custom information set by the web developer via the _setCustomVar method in Google Analytics. This cookie is updated every time new data is sent to the Google Analytics server.
2 years after last activity
__utmx
Used to determine whether a user is included in an A / B or Multivariate test.
18 months
_ga
ID used to identify users
2 years
_gali
Used by Google Analytics to determine which links on a page are being clicked
30 seconds
_ga_
ID used to identify users
2 years
_gid
ID used to identify users for 24 hours after last activity
24 hours
_gat
Used to monitor number of Google Analytics server requests when using Google Tag Manager
