With FastAPI and ngrok
Geospatial knowledge has been in excessive demand. It reveals how our planet modifications over time. Once we speak about geospatial, we consider Google Earth Engine (GEE). This service has a number of benefits. It hosts many giant knowledge collections that span over 37 years. All computations run on Google’s highly effective cloud infrastructure. What’s extra, it’s free for non-profit initiatives. With GEE, we are able to research the land use land cowl (LULC), the vegetation, the native local weather (right here and right here), and even the crop manufacturing within the US without spending a dime.
Nonetheless, GEE does have a excessive barrier to entry. Firstly, proficiency in JavaScript or Python is a should. Secondly, we must be conversant in many geospatial ideas, akin to picture assortment, geometry, and satellite tv for pc bands. Thirdly, its asynchronous request-response sample takes some getting used to for newcomers.
This poses a small problem for a lot of knowledge scientists. Most of the time, they simply need some values for a set of coordinates rapidly, such because the soil pH or the imply land temperature. As of this writing, they would wish to undergo a good bit of coding as a result of GEE gives no RESTful API. Wouldn’t it’s good if we are able to fill that hole ourselves (Video 1)? Our APIs ought to wrap up some frequent GEE computations and provide HTTP entry throughout the web.
Let’s do it on this article. I select FastAPI for the job. This fashionable library permits us to construct high-performance APIs in Python, one of many two service languages in GEE. And Google Colab is the platform of selection. Colab not solely is built-in with GEE seamlessly, however it could actually additionally serve the APIs on HTTP publicly with the assistance of ngrok (Video 1). On this article, I’m going to explain the development of two APIs. One returns the land temperature and the opposite calculates the soil pH for a given set of coordinates. Afterward, I’ll exhibit tips on how to use the APIs to get the metadata for some micro organism from BacDive.
The APIs are outlined right here.
That is the Python app that validates knowledge in BacDive with our GEE APIs.
First off, you will want a Google Earth Engine account and an ngrok account. Discover your ngrok Authtoken in your account web page (Determine 1).
1.1 Initialization
Create a Colab pocket book. After the library imports, let’s initialize each GEE and ngrok. First, comply with Google’s directions to authenticate and initialize GEE (Strains 1 & 2). Afterward, the enter operate will immediate you on your ngrok Authtoken (Line 3). Paste your Authtoken within the enter field and make sure. The code will then authorize your ngrok occasion at Line 4.
# Set off the authentication movement.
ee.Authenticate()# Initialize the library.
ee.Initialize()
ngrok_key = enter("Your ngrok Authtoken")
!ngrok authtoken {ngrok_key}
1.2 GEE wrapper capabilities
Subsequent, we are able to create some wrapper capabilities that encapsulate the GEE interactions.
def generate_collection(geometry, dataset, startDate, endDate):
assortment = ee.ImageCollection(dataset).filterDate(startDate, endDate).filterBounds(geometry);
return assortmentdef get_mean(image_collection, level, property, scale_factor):
picture = image_collection.choose([property]).cut back(ee.Reducer.imply()).multiply(scale_factor)
fcPoint = ee.FeatureCollection([ee.Feature(point)])
geojson = picture.sampleRegions(assortment = fcPoint, scale = 1000, geometries = True)
return geojson.getInfo()["features"][0]["properties"]
def generate_image(dataset):
return ee.Picture(dataset)
def get_image_value(picture, level, property, scale_factor):
return picture.reduceRegion(ee.Reducer.imply(), level, 100).get(property).getInfo() * scale_factor
Basically, GEE shops measurement leads to pictures. Some measurements, akin to elevation and soil pH, had been finished solely as soon as. They’re saved in single pictures. In distinction, different measurements, akin to soil temperature and precipitation, are finished periodically. They’re as an alternative saved in picture collections. The 2 capabilities generate_collection (Strains 1–3) and generate_image (Strains 14–15) return picture collections and single pictures, respectively. Then, we want to get the imply measurement values for the goal areas (Strains 5–12 & Strains 17–18).
1.3 FastAPI
It’s time to arrange FastAPI in our pocket book.
app = FastAPI()@app.get('/')
async def root():
return {'hiya': 'world'}
@app.get("/land-surface-temperature")
async def get_land_surface_temperature(lat: float, lon: float, start_date: datetime.date, end_date: datetime.date):
dataset = "MODIS/061/MOD11A1"
level = ee.Geometry.Level([lon, lat])
image_collection = generate_collection(level, dataset, start_date.strftime("%Y-%m-%d"), end_date.strftime("%Y-%m-%d"))
outcome = get_mean(image_collection, level, "LST_Day_1km", 0.02)
return {'outcome': outcome}
@app.get("/soil-ph")
async def get_soil_ph(lat: float, lon: float):
dataset = "OpenLandMap/SOL/SOL_PH-H2O_USDA-4C1A2A_M/v02"
level = ee.Geometry.Level([lon, lat])
picture = generate_image(dataset)
scale_factor = 0.1
ph = get_image_value(picture, level, "b0", scale_factor)
return {"outcome": {"pH": ph}}
On this code block, we first initialize a FastAPI app. Then we outline three routes: the foundation, land-surface-temperature, and soil-ph. Within the latter two, we use our capabilities from Part 1.2 to request the measurement values from GEE. The temperature values come from the MOD11A1.061 Terra Land Floor Temperature and Emissivity Each day World 1km dataset (MODIS knowledge and merchandise acquired by way of the LP DAAC don’t have any restrictions on subsequent use, sale, or redistribution), whereas the pH values are supplied by the OpenLandMap Soil pH in H2O dataset (CC-BY-SA-4.0).
1.4 ngrok
Now let’s begin the API service. The code right here is borrowed from a solution on stackoverflow.com.
ngrok_tunnel = ngrok.join(8000)
print('Public URL:', ngrok_tunnel.public_url)
nest_asyncio.apply()
uvicorn.run(app, port=8000)
The code will generate an endpoint URL and hold working (Video 1).
1.5 Check
We are able to now take a look at the service. Click on the endpoint URL, and you’ll be greeted by the “hiya world” message on the root (Video 1).
Amazingly, FastAPI will generate the documentation mechanically. You’ll be able to go to it on the /redoc route (Determine 2).
Let’s take a look at the land-surface-temperature API. Encode the required parameters as key-value pairs after the “?” mark in your URL. For instance, the next question string permits us to get the land floor temperature for (52.72389418745157, -92.03593750000002) between 2020–01–1 and 2020–05–01.
/land-surface-temperature?lat=52.72389418745157&lon=-92.03593750000002&start_date=2020-01-01&end_date=2020-05-01
Remember that the returned worth is in Kelvin (Determine 3).
The soil-ph API works equally. Nevertheless it doesn’t require a sampling interval. So the next question string alone can request the soil pH for a similar pair of coordinates (Video 1).
/soil-ph?lat=52.72389418745157&lon=-92.03593750000002
The BacDive database by DSMZ collects details about bacterial isolates, together with their progress temperature, pH, and metabolic profiles. Remember that almost all its data is generated within the lab. And it’s possible that microorganisms behave fairly in a different way within the wild.
Lately, BacDive has built-in Microbeatlas. The embedded maps of Microbeatlas present the worldwide distributions of 16S sequences of many micro organism (Determine 4).
This crossover is nice. Now, researchers cannot solely learn the way the micro organism behave within the lab (BacDive) but in addition the place they are often discovered around the globe (Microbeatlas). However we are able to do extra. We are able to collect the wild metadata from our freshly minted GEE APIs and examine them with BacDive’s lab outcomes. This comparability can present us whether or not micro organism dwell in a different way within the lab and within the wild.
Let’s take the bacterium Rhodopseudomonas palustris R1 as our instance. Its Microbeatlas web page reveals us that this bacterium might be present in a protracted listing of samples (Determine 5).
As of 2023–01–05, the DOWNLOAD button in Microbeatlas doesn’t work. So I filtered these samples with the key phrase “soil”. After which I examined the highest samples and selected these with full location and time knowledge (Desk 1).
With the next Python code, we are able to get the land temperature and pH values for these samples (additionally accessible in my Colab hyperlink above).
sheet_id = "YOUR GOOGLE SHEETS ID"
sheet_name = "YOUR GOOGLE SHEETS NAME"url = f"https://docs.google.com/spreadsheets/d/{sheet_id}/gviz/tq?tqx=out:csv&sheet={sheet_name}"
df = pd.read_csv(url)
api_url = "YOUR ngrok ENDPOINT"
land_temperature_route = "/land-surface-temperature/?"
land_ph_route = "/soil-ph/?"
for index, row in df.iterrows():
date = datetime.strptime(row["date"], "%Y-%m-%d")
start_date = datetime.strptime(f"{date.12 months}-{date.month}", '%Y-%m').date()
end_date = datetime.strptime(f"{date.12 months}-{date.month + 1}", '%Y-%m').date()
lat = row["lat"]
lon = row["lon"]
temp_url = f"{api_url}{land_temperature_route}lat={lat}&lon={lon}&start_date={start_date}&end_date={end_date}"
ph_url = f"{api_url}{land_ph_route}lat={lat}&lon={lon}"
temp = requests.get(temp_url).json()["result"]["LST_Day_1km_mean"] -273.15
ph = requests.get(ph_url).json()["result"]["pH"]
print (f'{row["sample"]}t{temp}t{ph}')
The outcomes are proven in Desk 2.
On the one hand, the temperature values of those prime 5 samples vary from 11 to 19 °C. They’re much decrease than the expansion temperature of 28–30 °C proven in BacDive. Then again, BacDive doesn’t document the expansion pH for this bacterium. However our outcomes point out that Rhodopseudomonas palustris R1 might be discovered abundantly in acidic soil and thus fill the knowledge hole in BacDive.
