buglerbilly
07-05-10, 02:00 AM
Harris Applies Sports-TV Technology to UAV Video Analysis
By WILLIAM MATTHEWS
Published: 3 May 2010
What if viewing military intelligence was more like watching a football game on TV? What if troops could watch instant replays of events and study high-definition views of targets shot from multiple angles? How about an audio feed to accompany the video? And a digital map that could be laid over images to pinpoint locations would be helpful. So would photos of suspects and background information that pop up on command.
That might sound like the C4ISR equivalent of fantasy football, but John Delay says the technology to pull it all together exists today. After all, commercial broadcasters do it every day.
Delay, who has worked on broadcasting technology for decades, says the same technology that broadcasters use to combine information from diverse sources can also be used by militaries to produce useful and, more importantly, "actionable" intelligence. Actionable intelligence is accurate information that arrives soon enough to act on.
Melbourne, Fla.-based Harris, where Delay is director of strategy for government solutions, has developed a Full-Motion Video Asset-Management Engine - FAME, for short - that makes it possible to merge video, audio and other information gathered by a variety of sensors and data bases.
The result is video intelligence - a UAV feed, for example - that is combined with relevant audio, incident reports, maps, annotation by users and any other data.
The U.S. military has tested the system and ordered continued development. But widespread use of the technology is likely to take some time.
"It's not gonna happen in six months," Delay said. "There are some very deep-rooted challenges" in the way the military captures, stores, analyzes and uses video and other intelligence.
The problem is basically this: The U.S. military is inundated by a large and growing volume of intelligence data - video gathered by an expanding fleet of UAVs, unmanned sensors, satellites, intercepted signals and more.
Air Force UAVs alone now collect 1,800 hours of video a month, three times more than they collected in 2007, Harris President Howard Lance said in an address to a broadcasters' convention in April.
Despite the volume of data, or because of it, the military struggles to extract useful intelligence from this digital flood.
"How would you isolate the critical 15 seconds of video from what is otherwise many, many hours of useless video feed? How do we then tag that section of the video so we can retrieve it later?" Lance asked.
And how about attaching other useful intelligence that would appear with the video: photos of suspects, cell phone numbers, incident reports and the like? Manned and unmanned sensors are gathering too much video and other intelligence data to be sifted through and analyzed manually, Lance said. The military needs automation.
The answer? Use the same systems and technology employed by commercial broadcasters. Each day they create and successfully manage 30 times the volume of video and other digital content that the military struggles with, he said.
So the football fan sees high-definition video augmented with explanatory audio. Boxes pop onto the screen to display the score, the down and the time remaining in the quarter. Instant replays present touchdowns from multiple angles. Hand-drawn arrows show how the receiver eluded the defenders.
For the viewer, the result is excellent real-time situational awareness, Lance said.
By contrast, U.S. troops typically receive just the video feed. It's like watching a football game without all the extra information.
"You don't know what down it is, you don't know what the score is. It's just raw video," Lance said.
Metadata to the Rescue
A major reason for the difference between what broadcasters can do and what the military can't is metadata - data that describes data.
Video shot by a UAV, for instance, would include such metadata as the date it was shot, altitude, location, time and possibly other data such as camera angles.
Such information is useful because it enables intelligence analysts to search, for example, for video from certain locations shot on certain dates or at certain times.
But "when the military runs an operation, there's a lot of metadata they don't capture," Delay said.
For example, metadata tags could be added whenever the UAV spots a particular type of truck, or when people on the ground seem to be digging holes or carrying weapons. The tags would then lead intel-ligence analysts to those parts of the video.
Delay said Harris has developed automated video analytics that can add metadata tags to video or audio recordings so that they can be quickly searched. An algorithm attaches a metadata tag to the video at points where the wanted truck appears. Another might record when the vehicle stops. Still other algorithms are designed to tag suspicious activity such as people digging holes alongside roads, or people carrying weapons, Delay said.
Once data are tagged with metadata, various data streams - video, audio, geo-spatial - can be examined and compared. "Event-based correlation" software can automatically tie together related intelligence gathered by different sensors at different times and places.
Thus, the truck seen near where a roadside bomb exploded might be picked out at other locations by an automated search of surveillance videos, possibly tipping U.S. troops to the whereabouts of the bombers, Delay said.
But right now, most military intelligence data are not well meta-tagged, he said.
Harris' FAME was tested during the U.S. military's Empire Challenge exercises in 2008 and 2009.
In the 2008 test, FAME was praised by the National Geospatial Agency as promising commercial technology that enables the military to quickly merge multiple pieces of disparate intelligence data to produce a more complete intelligence picture.
During the 2009 exercise, Harris teamed with Lockheed Martin, combining Harris' FAME with Lockheed's Audacity video management system.
After the exercise, the Joint Forces Command last fall awarded Lockheed a $29 million contract so that it, Harris and a third company, NetApp, can further develop the FAME-Audacity systems in a program known as Valiant Angel.
Delay said the new system should appear soon in Afghanistan and Iraq. ■
E-mail: bmatthews@defensenews.com.
By WILLIAM MATTHEWS
Published: 3 May 2010
What if viewing military intelligence was more like watching a football game on TV? What if troops could watch instant replays of events and study high-definition views of targets shot from multiple angles? How about an audio feed to accompany the video? And a digital map that could be laid over images to pinpoint locations would be helpful. So would photos of suspects and background information that pop up on command.
That might sound like the C4ISR equivalent of fantasy football, but John Delay says the technology to pull it all together exists today. After all, commercial broadcasters do it every day.
Delay, who has worked on broadcasting technology for decades, says the same technology that broadcasters use to combine information from diverse sources can also be used by militaries to produce useful and, more importantly, "actionable" intelligence. Actionable intelligence is accurate information that arrives soon enough to act on.
Melbourne, Fla.-based Harris, where Delay is director of strategy for government solutions, has developed a Full-Motion Video Asset-Management Engine - FAME, for short - that makes it possible to merge video, audio and other information gathered by a variety of sensors and data bases.
The result is video intelligence - a UAV feed, for example - that is combined with relevant audio, incident reports, maps, annotation by users and any other data.
The U.S. military has tested the system and ordered continued development. But widespread use of the technology is likely to take some time.
"It's not gonna happen in six months," Delay said. "There are some very deep-rooted challenges" in the way the military captures, stores, analyzes and uses video and other intelligence.
The problem is basically this: The U.S. military is inundated by a large and growing volume of intelligence data - video gathered by an expanding fleet of UAVs, unmanned sensors, satellites, intercepted signals and more.
Air Force UAVs alone now collect 1,800 hours of video a month, three times more than they collected in 2007, Harris President Howard Lance said in an address to a broadcasters' convention in April.
Despite the volume of data, or because of it, the military struggles to extract useful intelligence from this digital flood.
"How would you isolate the critical 15 seconds of video from what is otherwise many, many hours of useless video feed? How do we then tag that section of the video so we can retrieve it later?" Lance asked.
And how about attaching other useful intelligence that would appear with the video: photos of suspects, cell phone numbers, incident reports and the like? Manned and unmanned sensors are gathering too much video and other intelligence data to be sifted through and analyzed manually, Lance said. The military needs automation.
The answer? Use the same systems and technology employed by commercial broadcasters. Each day they create and successfully manage 30 times the volume of video and other digital content that the military struggles with, he said.
So the football fan sees high-definition video augmented with explanatory audio. Boxes pop onto the screen to display the score, the down and the time remaining in the quarter. Instant replays present touchdowns from multiple angles. Hand-drawn arrows show how the receiver eluded the defenders.
For the viewer, the result is excellent real-time situational awareness, Lance said.
By contrast, U.S. troops typically receive just the video feed. It's like watching a football game without all the extra information.
"You don't know what down it is, you don't know what the score is. It's just raw video," Lance said.
Metadata to the Rescue
A major reason for the difference between what broadcasters can do and what the military can't is metadata - data that describes data.
Video shot by a UAV, for instance, would include such metadata as the date it was shot, altitude, location, time and possibly other data such as camera angles.
Such information is useful because it enables intelligence analysts to search, for example, for video from certain locations shot on certain dates or at certain times.
But "when the military runs an operation, there's a lot of metadata they don't capture," Delay said.
For example, metadata tags could be added whenever the UAV spots a particular type of truck, or when people on the ground seem to be digging holes or carrying weapons. The tags would then lead intel-ligence analysts to those parts of the video.
Delay said Harris has developed automated video analytics that can add metadata tags to video or audio recordings so that they can be quickly searched. An algorithm attaches a metadata tag to the video at points where the wanted truck appears. Another might record when the vehicle stops. Still other algorithms are designed to tag suspicious activity such as people digging holes alongside roads, or people carrying weapons, Delay said.
Once data are tagged with metadata, various data streams - video, audio, geo-spatial - can be examined and compared. "Event-based correlation" software can automatically tie together related intelligence gathered by different sensors at different times and places.
Thus, the truck seen near where a roadside bomb exploded might be picked out at other locations by an automated search of surveillance videos, possibly tipping U.S. troops to the whereabouts of the bombers, Delay said.
But right now, most military intelligence data are not well meta-tagged, he said.
Harris' FAME was tested during the U.S. military's Empire Challenge exercises in 2008 and 2009.
In the 2008 test, FAME was praised by the National Geospatial Agency as promising commercial technology that enables the military to quickly merge multiple pieces of disparate intelligence data to produce a more complete intelligence picture.
During the 2009 exercise, Harris teamed with Lockheed Martin, combining Harris' FAME with Lockheed's Audacity video management system.
After the exercise, the Joint Forces Command last fall awarded Lockheed a $29 million contract so that it, Harris and a third company, NetApp, can further develop the FAME-Audacity systems in a program known as Valiant Angel.
Delay said the new system should appear soon in Afghanistan and Iraq. ■
E-mail: bmatthews@defensenews.com.