Segmenting remote sensing imagery with box prompts¶

This notebook shows how to generate object masks from box prompts with the Segment Anything Model 2 (SAM 2).

Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator.

Install dependencies¶

Uncomment and run the following cell to install the required dependencies.

1	`# %pip install segment-geospatial`

import leafmap
from samgeo import SamGeo2
from samgeo.common import raster_to_vector, regularize

Create an interactive map¶

m = leafmap.Map(center=[47.653287, -117.588070], zoom=16, height="800px")
m.add_basemap("Satellite")
m

Download a sample image¶

Pan and zoom the map to select the area of interest. Use the draw tools to draw a polygon or rectangle on the map

if m.user_roi is not None:
    bbox = m.user_roi_bounds()
else:
    bbox = [-117.6029, 47.65, -117.5936, 47.6563]

image = "satellite.tif"
leafmap.map_tiles_to_geotiff(
    output=image, bbox=bbox, zoom=18, source="Satellite", overwrite=True
)

You can also use your own image. Uncomment and run the following cell to use your own image.

1	`# image = '/path/to/your/own/image.tif'`

Display the downloaded image on the map.

m.layers[-1].visible = False
m.add_raster(image, layer_name="Image")
m

Initialize SAM class¶

Set automatic=False to enable the SAM2ImagePredictor.

sam = SamGeo2(
    model_id="sam2-hiera-large",
    automatic=False,
)

Specify the image to segment.

sam.set_image(image)

Display the map. Use the drawing tools to draw some rectangles around the features you want to extract, such as trees, buildings.

Create bounding boxes¶

If no rectangles are drawn, the default bounding boxes will be used as follows:

if m.user_rois is not None:
    boxes = m.user_rois
else:
    boxes = [
        [-117.5995, 47.6518, -117.5988, 47.652],
        [-117.5987, 47.6518, -117.5979, 47.652],
    ]

Segment the image¶

Use the predict() method to segment the image with specified bounding boxes. The boxes parameter accepts a list of bounding box coordinates in the format of [[left, bottom, right, top], [left, bottom, right, top], ...], a GeoJSON dictionary, or a file path to a GeoJSON file.

sam.predict(boxes=boxes, point_crs="EPSG:4326", output="mask.tif", dtype="uint8")

Display the result¶

Add the segmented image to the map.

m.add_raster("mask.tif", cmap="viridis", nodata=0, layer_name="Mask")
m

Use an existing vector file as box prompts¶

Alternatively, you can specify a file path to a vector file. Let's download a sample vector file from GitHub.

url = "https://github.com/opengeos/datasets/releases/download/samgeo/building_bboxes.geojson"
geojson = "building_bboxes.geojson"
leafmap.download_file(url, geojson)

Display the vector data on the map.

m = leafmap.Map()
m.add_raster(image, layer_name="Image")
style = {
    "color": "#ffff00",
    "weight": 2,
    "fillColor": "#7c4185",
    "fillOpacity": 0,
}
m.add_vector(geojson, style=style, zoom_to_layer=True, layer_name="Bboxes")
m

Segment image with box prompts¶

Segment the image using the specified file path to the vector mask.

output_masks = "building_masks.tif"

sam.predict(
    boxes=geojson,
    point_crs="EPSG:4326",
    output=output_masks,
    dtype="uint8",
    multimask_output=False,
)

Display the segmented masks on the map.

m.add_raster(
    output_masks, cmap="jet", nodata=0, opacity=0.5, layer_name="Building masks"
)
m

Convert raster to vector¶

output_vector = "building_vector.geojson"
raster_to_vector(output_masks, output_vector)

Regularize building footprints¶

output_regularized = "building_regularized.geojson"
regularize(output_vector, output_regularized)

m.add_vector(
    output_regularized, style=style, layer_name="Building regularized", info_mode=None
)