Two Exciting, Optimistic Visions for the 2020s Involving the Earth, Space, and Machine Learning

Introduction

In this essay I present two inspirational visions for the 2020s, one involving Earth and the other involving space — both scenarios are bound together and enabled by advanced machine learning. These two visions are what personally inspire my work, and provide a roadmap for me this decade.

The focus of this article is optimistic. While looking at the downsides of technology is important, we are currently inundated with dark visions of the future. In my opinion it is sometimes good to look up and be inspired rather than horrified. Hopefully I can paint a few positive visions for this decade that inspire you.

First pair of the 28 Planet Labs satellites launched from the ISS via the NanoRacks CubeSat Deployer (2014)

A brief introduction to who I am before we dive in. I'm a Staff Machine Learning Engineer at Planet. Planet has ~200 small cubesats the size of a shoebox in Low Earth Orbit (LEO) that image the landmass of the Earth daily. I do machine learning & analytics on this imagery, performing things like automatic road and building change detection using neural networks. I'm also an advisor with the Frontier Development Lab (FDL), a public-private partnership between entities such as NASA, the SETI Institute, and more that hosts intensive research sprints at the intersection of space exploration/science, earth science, and machine learning.

I have a long history in Silicon Valley of technical and product innovation, having worked at companies like Google & Dropbox. At Dropbox I did industrial R&D to ship deep learning-powered products to millions of users and across billions of files. In my career I've done everything from helping the open foundations of the web blossom into a true application deployment platform through efforts like HTML5; to re-imagining interactive digital textbooks for higher education and how they are published by adopting ideas from computer science; to founding what became the Coworking movement, an international grassroots movement to establish a new kind of workspace for the self-employed, with more than 15,000 coworking spaces now open globally.

Now that that's out of the way, let's dive into our first optimistic vision, involving the Earth.

Earth

In the late 1950s the Advanced Research Projects Agency (ARPA) was assigned the task of creating a global seismology network to monitor tremors on the Earth, with sensors globally from Thailand to Alaska to Tasmania. In the early 1960s this sensor network proved instrumental in outlawing nuclear tests, as well as monitoring underground tests, as any nuclear tests would activate this global seismology network. A global monitoring system allowed global treaties to have "teeth" and be enforceable, in this case making the world safer from nuclear tests and nuclear proliferation. You do not get a global test ban treaty without a sensor network to keep countries honest.

John F. Kennedy signs the Limited Test Ban Treaty (Washington, 7 October 1963)

In the same way, by the end of the 2020s I'd like to see a global network of earth observation satellites combined with advanced machine learning to monitor the environment and human development daily, weekly, and monthly. This would flow up to a State of the Planet & Human Society dashboard monitoring key metrics such as:

• What's the global percentage of degraded forests?
• What's the percentage of humanity living in poverty?
• What are the number of tree crowns across the entire globe, counted individually?
• What percentage of global biomass is capturing carbon vs. acting as a carbon source?
• What are the total gigawatts of renewable energy (solar photovotaic, wind, etc.) currently operating on Earth?
• What communities are at risk of catastrophic flooding?
• etc.

We would also be able to gauge the relative change in these metrics over days, weeks, months, and years, to see if we are getting better or worse.

Automatically detecting and surfacing these environmental and human development trends can give real enforcement mechanisms for global treaties, just as ARPA's global seismometer network contributed to ending nuclear tests. A given country can promise to preserve their biomass and trees, for example, and perhaps have these promises be turned into tradeable instruments — a global earth observation network plus machine learning would make it possible to know, daily, how a country is actually doing preserving their forests, for example, to back these promises and tradeable instruments with verification. "Trust, but verify," as many said during the nuclear disarmament talks of the 1980s.

I believe this will create a revolution in our ability to hold societies accountable by the end of the 2020s. This is my primary focus at Planet, to enable and help this revolution as much as I can.

The 17 Sustainable Development Goals (SDG) of the United Nations. Remote sensing + machine learning can help with some of them.

Rich sensor modalities, with data fusion between them, will be an important part of making this vision true by the end of the decade. This will include:

• High revisit, optical, daily, global scans - this includes Planet's Dove constellation, currently at 3 through 5 meters.
• Very high resolution, tasking-oriented optical satellites - these allow very high resolution tasking collects to be done over specific, focused areas of interest, in the tens of centimeters range. Examples of these include Planet's SkySat and the future Planet Pelican fleet.
• Hyperspectral - these allow the detection of specific chemical composition and materials, including monitoring climate change gases like methane and carbon dioxide from power plants and point pollution emitters, as well as detecting details on surface materials of the Earth's surface. Planet's future Carbon Mapper is an example hyperspectral satellite.
• Synthetic Aperture Radar (SAR) - satellite platforms using radar to penetrate clouds, fog, smog, darkness, and smoke, especially important when monitoring the Earth's equitorial regions which can be frequently cloudy. The Sentinel-1 fleet are examples of SAR satellites.
• Light Detection and Ranging (LIDAR) - satellites using light in the form of a pulsed laser to measure distances to the Earth. This is important for measuring variables such as tree canopy heights in forests, determining surface elevations, etc. An example of these sensors are the Global Ecosystem Dynamics Investigation (GEDI) instrument on the International Space Station (ISS).

To make our global monitoring vision true for the 2020s, we need a greater variety of these sensors monitoring the Earth at greater cadences, with as much resolution as possible, with common methods to do data fusion across them to get different angles of the Earth.

There are also machine learning frontiers that have to be solved to enable this vision. This includes:

• Labeling geospatial data to be used for training neural networks is very error prone & tedious. We need to adapt the latest advances from Self-Supervised Learning (SSL) to work in the geospatial domain.
• It would be great to get a DallE-2/GPT-3 scale generative model but trained on earth observation systems.
• We need generic change detection systems that can scale globally across continents and across a range of change detection tasks, whether for roads and buildings or changes in land cover.
• We need advancements in multi-task/meta-learning to be applied in the geospatial realm, adapting developments like Model-Agnostic Meta-Learning (MAML) or even a generalist agent like DeepMind's Gato but applied to a range of remote sensing problems.
• We need a true digital globe, somewhat like the digital globe Microsoft created for Flight Simulator 2020 but with a remote sensing and earth science focus, to realistically synthesize global training data and for experiments.
• Sensor fusion and automatic tip and cue will become important tools, with automatic tip and cue focused on environmental and human development goals rather than the traditional Defense & Intelligence (D&I) focus that automatic tip and cue currently has. Research and product work need to focus on getting better at automated tip and cue for civilian & environmental goals based on tips from machine learning models and observations, with follow up loops.

Space

The last 15 years have been exciting in space, primarily due to three developments:

• Rapid cost reductions from SpaceX with the Falcon 9 and Falcon Heavy rockets, and increased access to affordable micro-launchers like Rocket Lab's Electron.
• The transformation from NASA moving from cost-plus contracts to results-driven contracts, such as Commercial Orbital Transportation Services (COTS) and Commercial Crew Development (CCDev).
• Rapid miniaturization and cost reductions thrown off from developments in sectors like the smart phone market opening new possibilities for use in space.

If SpaceX's Starship happens, though, we will see developments in space in the mid- to late 2020s that will be, to put it plainly, absolutely bonkers compared to the last 15 years. Starship fundamentally rewrites cheap, rapid, massive access to Low Earth Orbit, and as the science fiction author Robert Heinlein wrote: "If you can get your ship to orbit, you're halfway to anywhere."

If you don't know what Starship is, it's SpaceX's Super Heavy rapidly re-usable launch vehicle, currently being designed in South Texas (right by where I grew up BTW) and targeting the mid-2020s for takeoff. The current paper specs for Starship are 100 through 150 metric tons uplift depending on the exact target orbit and final actual performance of Starship, for perhaps ~3 million marginal cost per launch baseline price, with a payload fairing size of 9 meters in diameter and an in-orbit refuelling ability. These are crazy numbers and abilities, but the magnitude of this change can get lost. Lets do some comparisons to understand just how transformative having Starship could be.

First, take a look at the amount of metric tons different rockets can take to LEO. As you can see with the numbers under the rockets, showing the payload tonnage to LEO, Starships estimated tonnage is incredibly high vs. other rocket systems; even if it "only" ends up being 100 tons rather than 150 tons like in the diagram below its still much larger than most other systems:

Comparison of metric tons to LEO for different launch systems, original image modified from Wikipedia Super Heavy Launch Systems comparison.

This Starship Super Heavy Lift ability also comes at a cost that is estimated to be even cheaper than the Falcon 9:

Payload Cost to LEO in kgs, in FY21 USD

Even if the cost "only" ends up being $50 per kg rather than $14 per kg, that's still vastly cheaper than the closest comparable cheapest rocket, the Falcon 9 at $1,500 per kg.

The headline is that we are getting Saturn V level tonnage to orbit for cheaper than the cheapest launch system, or basically Super Heavy launch capability at Small Launcher prices. This right here is the revolution in a nutshell: the ability to do Saturn V level exploration and science for cheaper than the price of a Falcon 9.

In addition, there are two other important abilities that will make Starship even more capable:

• A very large fairing size to allow bigger and wider objects to be launched into orbit, such as space telescope systems
• The ability to refuel in orbit, allowing more affordable access to the rest of the solar system, including the Moon and Mars

Starship will allow many dead space dreams of the last 60 years to come back to life. The ability to cheaply and rapidly launch large, heavy mass into space will revive efforts such as in-space manufacturing, operations on the surface of the Moon and Mars, building large scientific structures at the Lagrange Points, and more. However, space remains a very unforgiving and harsh environment, and we will need to use autonomous systems as much as possible for these efforts rather than just throwing people at the problem. This means the demand for reliable and very capable autonomous systems in space will greatly increase near the end of this decade.

Rendering of proposed NASA/JPL Lunar Crater Radio Telescope (LCRT), an ultra-long-wavelength radio telescope on the far-side of the Moon.

Here's a glimpse of some of the activities we might be able to do later in the decade or early 2030s if Starship takes off:

• In-Space Resource Utilization (ISRU), which means to use resources already in space such as asteroids and on the surface of the Moon and Mars, to build structures and refine useful materials. This can include harvesting water from icy asteroids for use by people, split into propellant for use by rocket engines, etc., as well as creating habitable shelters on planetary surfaces using the regolith already present. This will require manufacturing systems that are robust and autonomous.
• In-space construction of large structures, such as very large free floating optical synthetic aperture telescopes to image continents on exoplanets around other stars, or building very large radio telescopes on the dark side of the moon, such as the NASA/JPL proposed Lunar Crater Radio Telescope. These efforts will require precision, autonomous robotics and manufacturing in micro gravity environments.
• The ability to have heavier, larger, and more ambitious scientific missions to planets throughout the solar system, including the icy outer planets.

To unlock these kinds of abilities, we need advances in machine learning that includes:

• The ability to use modern compute tailored for neural networks, such as GPUs and neural cores, in high radiation space environments, by leveraging software-level resiliency even if dealing with non-rad hardened compute hardware, similar to how the Mars Ingenuity helicopter uses Commercial Off The Shelf (COTS) Qualcomm processors for its computer vision.
• The ability to harness reinforcement learning with hard guarantees on system behavior, similar to what traditional control theory provides but with the learning and flexibility of modern deep reinforcement learning. This will be needed for the advanced robotics and perception challenges that in-space and on-surface ISRU, manufacturing, and construction requires, while strong guarantees on autonomous behavior are necessary in the space environment to know that a system will work as expected.
• The ability for algorithms to automatically do exploration and scientific work when exploring the outer icy planets, such as doing flybys and exploration of Europa without any humans in the loop, as the radio distance is too great for scientists on Earth to really direct an outer planet mission in detail.

Conclusion

These two visions, one focused on taking care of the Earth and the other on opening up positive visions for space, are a personal roadmap for me this decade and one of my focuses for my machine learning work. I hope they help you in determining how you would like to contribute this decade. While humanity has real challenges to solve, I think there are more silver linings and positive possibilities in the 2020s than many give credit to. So put down your doomscrolling and roll up your sleeves to help move the ball forward in the 2020s.