The Real Time Regional Gateway (RT-RG) is a data processing and data mining system introduced in 2007 by the US National Security Agency (NSA) and deployed during the American military operations in Iraq and Afghanistan. It is able to store, fuse, search and analyze data from numerous sources, from intercepted communications to open source information. It was effective in providing information about Iraqi insurgents who had eluded less comprehensive techniques.[1]
Development
editThe prototype of the Real Time Regional Gateway (RT-RG) was designed by the military and intelligence contractor SAIC. Instead of storing all data in centralized databases, as was NSA's traditional approach, RT-RG uses distributed computing, which means hundreds or even thousands of individual computers work together on one task.[2]
This is managed by the Hadoop software, that was originally developed by Yahoo. The NSA created its own version, called Accumulo, which can manage extremely large data sets and also includes granular access controls. The successor of the Real Time Regional Gateway is named Nexus 7 and is now under development at the Defense Advanced Research Projects Agency (DARPA) [3]
At NSA the driving force behind the system was former NSA director Keith B. Alexander, who initiated a massive search for every piece of electronic information that could be found, after the Iraqi road side bombings reached an all-time high in 2005.[4] Alexander's "collect it all" strategy is believed by Glenn Greenwald to be the model for the comprehensive worldwide mass collection of communications which NSA is supposedly engaged in.[5]
Origins and deployment in Iraq
editUnder the secret interception programs that were started right after the September 11 attacks, which were codenamed STELLARWIND, the NSA got access to foreign communications at the switching points where international fiber-optic cables entered the United States. This enabled NSA not only to monitor many communications from Iraq, but also to initiate cyber attacks against Iraqi telephone and computer equipment.[2]
When the Iraq War troop surge of 2007 began, the NSA exported this all-encompassing way of intelligence gathering and analysing to Iraq. As its name describes, the Real Time Regional Gateway was a gateway through which all sorts of information related to one region (Iraq) became available to analysts and soldiers in real time and they could search the data similar to a search-engine like Google.[2]
The RT-RG derived its data from all kinds of sources, like raids, interrogations, and signal intelligence collected from ground sensors, as well as by airborne platforms like the RC-135 Rivet Joint and C-12 Huron aircraft, and SIGINT drones and satellites.[6] With the RT-RG, field commanders and intelligence analysts in the field were for the first time able to directly access NSA databases, making that intercepted data were ten times faster available on the ground than in the past.[6][7] On a screen they were provided with "every type of surveillance available in a given territory" and they could also "get back a phone number or list of potential targets".[8]
One of the developers of the RT-RG, was Pedro "Pete" Rustan from the US National Reconnaissance Office (NRO). In a 2010 interview he said that the system was designed to put together pieces from a different types of data sources, looking for patterns, and make this available directly to American warfighters on the ground.[9]
The Real Time Regional Gateway contributed to "breaking up Iraqi insurgent networks and significantly reducing the monthly death toll from improvised explosive devices (IED) by late 2008".[4] According to journalist and author Shane Harris, the RT-RG was "a rare example of successful collaboration within the byzantine federal bureaucracy" and eventually the key for winning the war in Iraq.[2]
Deployment in Afghanistan
editIn 2010, RT-RG refocused on Afghanistan, where it was used to fuse and analyze even more types of data. Besides the more traditional military intelligence information, this included for example road-traffic patterns, public opinion and even the price of potatoes, because changes in the latter could indicate potential conflicts. Initially these data were gathered and analyzed for 30-day periods, but later this was extended to 90 days, as adding more data appeared to give better results. A former US counterterrorism official described the RT-RG as "the ultimate correlation tool" adding that "It is literally being able to predict the future".[3]
See also
editReferences
edit- ^ Ellen Nakashima; Joby Warrick (July 14, 2013). "For NSA chief, terrorist threat drives passion to 'collect it all,' observers say". The Washington Post. Retrieved July 15, 2013.
Collect it all, tag it, store it. . . . And whatever it is you want, you go searching for it.
- ^ a b c d Shane Harris, "@War, The Rise of the Military-Internet Complex", Houghton Mifflin Harcourt, New York, 2014, p. 33-38.
- ^ a b Wall Street Journal: Technology Emboldened the NSA, June 9, 2013.
- ^ a b Washington Post: For NSA chief, terrorist threat drives passion to ‘collect it all’, July 14, 2013
- ^ Glenn Greenwald (July 15, 2013). "The crux of the NSA story in one phrase: 'collect it all': The actual story that matters is not hard to see: the NSA is attempting to collect, monitor and store all forms of human communication". The Guardian. Retrieved July 16, 2013.
- ^ a b Dana Priest, William M. Arkin, Top Secret America: The Rise of the New American Security State, Little, Brown and Company, 2011.
- ^ Matthew M. Aid, Intel Wars: The Secret History of the Fight Against Terror, Bloomsbury, 2012.
- ^ Washington Post: NSA growth fueled by need to target terrorists, July 22, 2013.
- ^ Defensenews.com: Change agent Archived 2015-02-27 at archive.today, October 8, 2010.