New Earth Surveillance Tech Is About to Change Everything, Including Us

On Christmas Eve, 1968, the astronauts aboard NASA’s Apollo 8 spacecraft became the first humans to behold the entirety of Earth with their own eyes. That day, crew member Bill Anders took an iconic photograph called “Earthrise'' that captured our home world emerging from behind the Moon’s horizon.

"We came all this way to explore the Moon, and the most important thing is that we discovered the Earth," Anders famously said of his mission.

More than 50 years later, Earth is being rediscovered from space once again, but this time it is through the "eyes" of satellites, supercomputers, and artificial intelligence (AI) networks. Geospatial science, a sprawling and multifaceted field dedicated to resolving ever-finer details about Earth and its systems, is poised to undergo an unprecedented growth spurt powered by this confluence of technologies across both the public and private sectors.

“With the proliferation of satellite platforms, essentially this is something that's almost become impossible to keep a handle on because there are so many new systems being launched and developed by so many different actors globally,” said Jonathan Chipman, director of Dartmouth College’s Citrin Family GIS/Applied Spatial Analysis Laboratory, in a call. “It's just mind-boggling the amount of data that's now being collected from low-Earth orbit."

The feeling of epiphanic connection with the planet experienced by astronauts gazing at Earth is known popularly as “the overview effect,” a term coined by author Frank White in his book of the same name. The new geospatial view of Earth, however, may offer something closer to an “overwhelm effect,” as our home world is imaged, valued, and monitored by millions of sensors on thousands of spacecraft orbiting Earth.

How will we deal with the petabytes of Earth-observation data that may document the collapse of whole ecosystems or the wreckage of natural disasters? What will we do with geospatial information that predicts such dire outcomes but also demands nimble and dramatic changes to our lifestyles?

It will take foresight to ensure that the deluge of information is managed in a way that equitably benefits communities and ecosystems around the world, and remains as accessible to the public as possible.

“The biggest challenge will be in making sense of all these data,” said Dawn Wright, chief scientist of the Environmental Systems Research Institute (Esri), a major geospatial software and data science company, in an email. “It is one thing to store, to distribute, even to analyze, but how do truly understand it? How do we make sense of it, and quickly so that we can make decisions?”

Tracking climate change, animals, and human beings

There are more than 2,000 active satellites orbiting Earth right now, many of which carry ultra-precise cameras, sensors, and other instruments that can resolve intimate details about our planet’s surface. Though geospatial data is heavily reliant on this complex space infrastructure, it is also augmented by a dizzying array of sources on the ground including smartphone geolocation, drones, government census reports, and even facial recognition software.

The result is an intertwined and cascading flood of information that is greater than the sum of its individual parts. Geospatial data can shed light on everything from the future of our warming planet to the unexplored ruins of bygone civilization to informed responses to pandemics. It is also critical to managing reactions to natural disasters, mass migrations, and urban developments. Meanwhile, satellites constantly keep tabs on planetary processes such as global ice melt, sea level rise, and atmospheric greenhouse gas content.

This incredible sprawl of geospatial data was a hot topic at the meeting of the American Association for the Advancement of Science (AAAS), an annual general science conference, which I attended a few weeks ago in Seattle. The meeting, themed “Envisioning Tomorrow’s Earth,” highlighted how the field unveils a complex global portrait of ecological and anthropogenic patterns, including forest cover, biodiversity, urban development, and even social conflicts.

For instance, University of Virginia oceanographer Scott Doney outlined how patterns in seawater color can be bellwethers of ecological health amid mounting human pressures. Satellites such as NASA’s Plankton, Aerosol, Cloud and ocean Ecosystem (PACE), which is due for launch in 2022, will be able to gather hyper-precise color data that will help fisheries and conservationists anticipate events such as phytoplankton blooms or marine food shortages.

“The new sensors on PACE would have much finer spectral resolution, and also a better spatial footprint,” Doney told me. “There are some sensors that can get down to a few hundred meters [of resolution] in coastal regions, and with the open ocean we can get down to a few kilometers.” Other technologies, such as submersible robots and AI, will help scientists predict future trends in addition to monitoring these patterns in near real-time.

Sophisticated geospatial studies are also occurring for land ecosystems, according to Martin Wikelski, director of the Max Planck Institute of Animal Behavior, who gave a talk on the emerging “internet of animals.”

Wikelski is part of an international project called ICARUS that tags animal populations with wearable sensors that are monitored by instruments on the International Space Station. This information can be used to help forecast the spread of global zoonotic diseases, which is especially relevant given the ongoing COVID-19 pandemic, and can also reveal how wild species are responding to climate change.

“Data is already showing that, for example, bird migration routes, or where populations are wintering, are changing,” Wikelski said. “We see these changes right now and as we get more data, it will become a lot more obvious because we can connect the dots much better within one year and within the same season.”

Scientists are also tracking humanity itself with increasing detail. Robert Chen, director of Columbia University’s Center for International Earth Science Information Network (CIESIN), has used decades of satellite imagery to show where urban populations are in flux and in decline.

Better observations and models of these demographic movements, including the introduction of tools like machine learning, will be essential for projecting these patterns and interpreting their effects on social stability, resource use, and resilience to climate change.

“We’re still somewhat early in the science, and I think there’s a lot of innovation,” Chen told me after a talk about his research. “There’s a lot of room for doing more to incorporate other kinds of drivers of population change and potentially use very large datasets to account for that.”

Scientists use geospatial data not only to project these long-term population trends, but also to respond to sudden and devastating displacements caused by civil war, famines, or natural disasters.

The life-saving potential of geospatial science became particularly clear in the wake of the devastating 2010 Haitian earthquake, for instance, and has continued to mature ever since. Satellite imagery revealed the extent of the infrastructure damage and population movements in the aftermath of the quake, while text and messaging data enabled search and rescue teams to locate trapped individuals.

“Such techniques have been developed over time and are extremely useful in addressing immediate mapping and locational needs after a natural disaster,” explained Marie Price, president of the American Geographical Society (AGS) and a professor of geography and international affairs at George Washington University, in an email. “Much of this work can be done remotely and by volunteers.”

While these advances offer a kaleidoscopic range of benefits for both the public and private sectors, they also raise practical and ethical concerns about privacy rights, data management, and the potential for unequal access to the gains of geospatial data.

Surveillance in space

Experts and institutions around the world are already envisioning how geospatial data might be mishandled in the future, and anticipating ways to counter unsettling or even flat-out dystopian outcomes.

To encourage a new generation of thinkers to engage with these problems, the AGS launched an initiative called EthicalGEO in 2019 that includes a fellowship program for interdisciplinary scholars.

“We know that geospatial technology is advancing so quickly (a vast array of satellites, drones, facial recognition, real-time tracking) and yet the ethical uses and guidelines for this technology require more systematic development,” Price said. “There is enormous potential for good with this technology, yet at the same time there are real threats to civil liberties and individual privacy that were unthinkable even a few years ago.”

One big risk is that parsing such a huge amount of data will often require machine learning. Algorithms that search for patterns in large datasets can be useful, but they also tend to carry the biases and prejudices of their human trainers. On top of that, such systems can evade proper scrutiny by giving a sleek techno-optimist cover to bad ideas.

For instance, a 2018 report from UNICEF warned that “decision-making using geospatial data is frequently captured within big data, which is largely removed from human scrutiny and left to algorithms.”

This is only one manifestation of a broader problem of unconscious discrimination due to algorithmic bias, but it could be significantly amplified by the sheer power of geospatial data. Certain populations, such as rural, nomadic, or impoverished communities, could end up overlooked by algorithms searching for particular settlement patterns, and that could lead to unfair resource allocation or representation in the data.

Price, who has written several books about human migration and globalization, also pointed to the danger of invasive tracking of vulnerable communities.

“There is considerable fear that facial recognition and real-time surveillance systems could be used to track the movements of individuals, especially ethnic or racial minorities, as well as migrants of all kinds,” she said.

This is especially worrisome considering that the commercial and military spheres of geospatial science generally have access to much higher-resolution data than the rest of us. The U.S. government restricts the distribution of commercial satellite images with resolutions sharper than 25-centimeter pixels, but more precise imagery certainly exists.

Such sharp resolutions pose obvious threats to the privacy of individuals and communities. Google Earth is one of the most accessible and widely used geospatial tools available to the public, but the company has had to adapt to requests to blur out sites that are sensitive for cultural or national security reasons. In a 2019 op-ed in the New York Times, archeologist Sarah Parcak highlighted the concerns of Indigenous cultures who might not want to see their sacred spaces exposed in digital satellite imagery.

The idea of authoritarian regimes using geospatial data to further exert control over populations is a nightmare scenario that could conceivably spill far beyond one nation’s borders. As the UNICEF report cautioned, “personally identifiable data or demographically identifiable data stored on servers” could be stolen by any number of malicious parties through various means “irrespective of the location of the data provider and the protective legislation in their home country.”

The flip side of these threats, Price noted, is that geospatial data can also expose humanitarian abuses that might otherwise go unnoticed. Over the past few years, the cruelty of China’s treatment of its Uyghur people, an ethnic and religious minority, has been evident from space, as sharp satellite imagery has revealed internment camps and the destruction of Uyghur religious and burial grounds.

Likewise, geospatial data of wildlife populations and ecological habitats has been essential to combating poachers, even though it could also be vulnerable to “cyber-poachers” that attempt to hack GPS trackers on certain coveted species. The reality that this powerful data can cut both ways is part of the reason geospatial scientists are so eager to engage a wider audience in an interdisciplinary conversation about these issues.

Who owns our digital planet?

As geospatial technologies proliferate and evolve, it will be essential to ensure that the benefits and access to data remain as public as possible, so that people without institutional affiliations can still maintain a stake in the outcomes. That means continued support for a democratized approach to the dissemination of quality satellite imagery, among other datasets.

“So much of the determination of how much value actually comes out of these systems boils down to how the data are being distributed, how widely available are they, and how easy are they to use,” Chipman explained.

As an example, he recalled that the U.S. government started privatizing data collected by NASA’s Landsat satellites during the 1980s, causing the price of images to shoot up to several thousand dollars apiece.

“One consequence of that was that the bottom fell out of the market and people really stopped using the data for most science purposes,” Chipman noted. “The program was de-privatized later on and suddenly the data are everywhere and being used for all kinds of fascinating projects again.”

Wright also emphasized the importance of creating inclusive platforms for sharing geospatial data. “All of this is not useful unless we can get data into the hands of people trying to do evidence-based decision making,” she said, such as “Indigenous communities that are on the front lines of climate change.”

“As we continue to collect and store huge volumes of data with little difficulty, are we sure that the data we are collecting are right data for the decisions that need to be made?” she asked. “Are we collecting data in the right places? Are we able to get data from people and places who have not traditionally shared their data?”

Fortunately, there is a lot of innovation within the geospatial community focused on broadening access to, and engagement with, our evolving picture of Earth. One project that Chipman is involved in enlists volunteers to capture lake data at the exact moment that satellites are taking measurements overhead. This not only helps scientists calibrate remote-sensing instruments, it also gives lakeside residents a whole new perspective of their environment.

Though this type of hands-on opportunity is not available at every location, tools like OpenStreetMap, an editable map generated by volunteers, enable anyone with internet access to interact with geospatial data. In fact, the AGS is working with high schools to introduce students to OpenStreetMap in order to interact with the geospatial dimensions of their own communities. Other specialized projects include Mixed Metro, a map that uses census data to track patterns of racial composition in American neighborhoods, or Global Fishing Watch, which monitors commercial fishing vessels worldwide.

Given that we are now grappling with a frightening pandemic, Wright cited a highly detailed dashboard of the COVID-19 outbreak, developed by Johns Hopkins engineer Lauren Gardner and her team, as a success story of intuitive geospatial technology for public use.

“This dashboard has been particularly effective at pulling together several sources of open data to convey a real-time, map-based understanding of spreading disease,” she said. “Authorities from local to national levels are using dashboards and maps like these to display data at different scales.”

Geospatial science affects our daily lives in an astronomical number of ways, and our reliance on this complicated view of Earth will only grow in the coming years and decades. Like all influential technologies, it has the potential to be a double-edged sword. But humans share an innate fascination with our planet and its intricacies, and we increasingly have access to sophisticated geospatial tools to explore it.

The perspective these tools will bring will be glorious, and terrifying, and it will change everything.