Cameron Colquhoun is the managing director of Neon Century, a London-based corporate intelligence consulting company. He uses cutting-edge open source intelligence capabilities to conduct ethical investigations on geopolitical, commercial and cyber risks for our clients.

"Even the disciplined press will sell their country's interests to a careful observer again and again."

"Today you are the media, you have the responsibility to report and keep hope! You are the media, we are one thing!"

Mir Hussein Mousavi (Former Prime Minister of Iran)

Upstate New York, 1883 was a strange place to start a blog about open source intelligence. One of the most influential figures in the intelligence community was born here. William Donovan is the son of a devout Irish immigrant. He grew up in a working-class family and performed well in school and academia.

Donovan's ambition is to become the first Roman Catholic president of the United States. He was indeed close to the presidency-in 1905, Donovan went to Columbia Law School, where among his classmates was the young Franklin D. Roosevelt.

After fighting in the First World War, Donovan had a successful career as an international lawyer, nearly becoming the Attorney General in 1925. During the two world wars, Donovan traveled the world as a lawyer, met with influential foreigners, and subsequently wrote a report to serve the US government in a semi-official capacity.

Donovan's connection with Roosevelt led to the establishment of American intelligence agencies. Before that, the world of intelligence and espionage was regarded as an ungentleman by the United States. Donovan lobbied Roosevelt to formalize his informal work for the U.S. government. On July 11, 1941, Roosevelt established the position of "information coordinator" for Donovan. After the Pearl Harbor incident, the need for intelligence was obvious, and Donovan's department was renamed the Office of Strategic Services, the predecessor of the Central Intelligence Agency. Like British special operations executives, OSS is involved in everything from assassination attempts to secret agent escapes and information warfare.

Readers in 2016 may be surprised that OSS has a branch dedicated to open source intelligence. The Research and Analysis Department of OSS has carefully collected dozens of newspapers, periodicals, news clippings, and radio reports from all over the world, looking for photos and articles that may reveal important information about the enemy. In the words of Donovan:

"Even the disciplined media will sell their country's interests to a careful observer again and again."

OSS reads obituaries in German local newspapers, looking for news about important Nazis. The images of the new battleships, craters, and aircraft have been carefully organized to allow OSS to assess the state of its enemies when evaluated together. It is surprising how similar OSS activities are to modern OSINT surveys, even without computers. From the perspective of OSS and SOE, the roots of open source intelligence can be traced back to nearly a century. Indeed, one can argue that Donvoan’s words are more true today than ever. In billions of posts, uploads, shares, and likes, individuals are betraying their interests to hard-working observers time and time again.

After the Second World War, the discipline of open source intelligence became the backwater of most government and military institutions, with professional librarians and researchers. When the term library became an unpopular term in the mid-2000s, few intelligence professionals tried to work in the field, and the world of sexy, secret agent operations and signals intelligence were available at the time.

For decades, the open source world (in the context of government analysis) has entered a deep sleep, without being disturbed by the Cold War or even 9/11.

But in 2009, the situation has changed. Iran is on the brink of a bottom-up "green revolution"; many of its citizens are protesting against the regime. Millions of young Iranians use the Internet to coordinate their activities, share viral content and encourage others to join the sport. For the first time, the Internet is flooded with citizen information about major political events, thanks in large part to the combination of smartphones, Internet connections and social media. In the first week of the protest, 60% of all blog links posted on Twitter were related to Iranian politics.

The Internet usage rate in Iran jumped from 34% in 2008 to 48% in 2009. This is a huge increase, with mobile phone users rising from 59% of the population to 72%. At that time, the BBC published an article entitled "The Internet Reborn Iran", claiming that a new form of "citizen journalism" was flourishing. During the 2009 protests in Iran, the Washington Post conducted a Q&A with influential digital commentator Evgeny Morozov:

Fairfax, Virginia: There were a lot of reports on Iranian reports last weekend. US news organizations did not really play their role, but Twitter and other similar sites did spread the news, letting people all over the world know what’s going on. Iran proceeded. Comment?

Evgeny Morozov: We have seen many citizen journalists taking photos and videos of the Tehran protests in almost real time. They do fill an important niche market. Similarly, networks such as Twitter have played an important role in attracting attention to this kind of user-generated content. So, Flickr provides great photos-and Twitter pays great attention to these photos. In fact, CNN's lack of coverage related to Iran has received a lot of criticism. Twitter users even organized an event called #cnnfail to deal with this situation. I think they have succeeded: CNN executives/reporters eventually had to answer relevant questions.

This is the first time-anyone around the world can mine these social networks to obtain intelligence-level content, and in the process write articles, forecasts and provide insightful intelligence analysis. Although the protest was ultimately unsuccessful and the Iranian regime quickly regained control of the Internet, we can now look back and think that the (ultimately failed) Green Revolution was a seminal event in the history of open source intelligence.

In hindsight, it is not surprising that Iran is the birthplace of this new OSINT world. Iran has more online users, the sum of Bahrain, UAE, Israel, Saudi Arabia, Syria, Yemen and Jordan. During the revolution, Matthew Weaver of the Guardian expressed surprise at the reality of this new world. "What people say at a certain point in the day will be confirmed by more traditional sources in four or five hours."

It is not surprising that citizen journalists and lone investigators (rather than the government) lead the development of open source intelligence as a discipline—it is a field that develops at the speed of light, honing and creating new tools and technologies every day. The whole world of academia and government is just beginning to realize the potential wealth of these data. Individuals who are not constrained by bureaucracy or bad IT can quickly become skilled OSINT operators and create previously unknown insights and connections.

Just a year after the Green Revolution, the revolution triggered by social media has spread in the Arab world. The combination of public outrage, smartphones and social media shook the dictatorships in North Africa and the Middle East. According to one of its directors, the CIA’s Open Source Intelligence Center “cannot foresee the precise development of Internet-based social activism in the Arab world”. One explanation for this failure is that government intelligence agencies are obsessed with gathering intelligence from powerful elites, rather than "using the vast amount of open source information there."

However, in recent years, countries such as the United States and the United Kingdom have clearly noticed this. Last year, only 23 hours after the jihadists posted a selfie showing the roof structure of the building, the U.S. military destroyed an Islamic State bomb factory. This may be the most powerful military use of OSINT for targeted operations. example of. In the private sector, dozens of open source intelligence companies have emerged in the United Kingdom and the United States, serving a range of public and private sector clients. Recently, as Wired magazine reported in April 2016, a series of government agencies are looking for ways to better understand the wealth of open data available online, realizing that open source intelligence can make or break operations.

Where is OSINT going next?

Looking back at this short history of open source intelligence, it can be said that modern open source intelligence is the result of technological fusion. This started around 2009, when three things happened; firstly-a large number of smartphones with 3G connections were in the hands of dissatisfied citizens. Second, these citizens use a small number of applications to share a lot of content about their domestic political events. Third, these data are free and open, and can be accessed and analyzed by other parts of the world.

However, the world of open source intelligence will not stand still, and other technologies will continue to enhance and change the practice of open source intelligence. From Minnesota to Damascus, the growing appetite for live content presents very real challenges to police, security, and open source intelligence practitioners. Machine learning, virtual and augmented reality, and artificial intelligence will change OSINT again in the next few years. It can be said that open source intelligence is likely to become a new hot spot for technology investment and stand out from network security.

OSINT has increasingly become an area that has attracted great attention from investors, the media and the public. After all, billions of posts, images, streams, records, and data are uploaded to the Internet every day—surprisingly, few of us really realize that they are a good opportunity to find useful intelligence.

Your donation to Bellingcat is a direct contribution to our research. With your support, we will continue to publish groundbreaking investigations and expose illegal activities around the world.

In addition to the content we have published, we will also introduce readers to the activities our employees and contributors participate in, such as noteworthy interviews and training seminars.