Skip to content

Download notebook

SAM3 Image Segmentation for Remote Sensing

image

This notebook demonstrates how to use the Segment Anything Model 3 (SAM3) for segmenting remote sensing images using the samgeo3 module.

Installation

First, make sure you have the required dependencies installed:

1
# %pip install "segment-geospatial[samgeo3]"

Import Libraries

1
2
import leafmap
from samgeo import SamGeo3, download_file

Download Sample Data

Let's download a sample satellite image covering the University of California, Berkeley, for testing:

1
2
url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif"
image_path = download_file(url)

1
2
3
m = leafmap.Map()
m.add_raster(image_path, layer_name="Satellite image")
m

Request access to SAM3

To use SAM3, you need to request access by filling out this form on Hugging Face: https://huggingface.co/facebook/sam3

Once you have access, uncomment the following code block and run it.

1
2
# from huggingface_hub import login
# login()

Initialize SAM3

When initializing SAM3, you can choose the backend from "meta", or "transformers".

1
sam3 = SamGeo3(backend="meta", device=None, checkpoint_path=None, load_from_HF=True)

Set the image

You can set the image by either passing the image path or the image URL.

1
sam3.set_image(image_path)

Generate masks with text prompt

1
sam3.generate_masks(prompt="building")

Show the results

1
sam3.show_anns()

annotation

1
sam3.show_masks()

Generate masks by bounding boxes

1
2
3
4
5
6
7
8
# Define boxes in [xmin, ymin, xmax, ymax] format
boxes = [[-122.2597, 37.8709, -122.2587, 37.8717]]

# Optional: specify which boxes are positive/negative prompts
box_labels = [True]  # True=include, False=exclude

# Generate masks
sam3.generate_masks_by_boxes(boxes, box_labels, box_crs="EPSG:4326")

1
sam3.show_boxes(boxes, box_labels, box_crs="EPSG:4326")

bbox

Show the results

1
sam3.show_anns()

Save Masks

Save the generated masks to a file. If the input is a GeoTIFF, the output will be a GeoTIFF with the same georeferencing. Otherwise, it will be saved as PNG.

1
2
3
# Save masks with unique values for each object
# Since the input image is a GeoTIFF, the output will also be a GeoTIFF
sam3.save_masks(output="building_masks.tif", unique=True)

1
2
# Save as binary mask (all foreground pixels are 255)
sam3.save_masks(output="building_masks_binary.tif", unique=False)

Save Masks with Confidence Scores

You can also save the confidence scores for each mask. The scores indicate the model's confidence for each predicted mask.

1
2
3
4
5
6
7
# Save masks and confidence scores
# Each pixel in the scores image will have the confidence value of its mask
sam3.save_masks(
    output="building_masks_with_scores.tif",
    save_scores="building_scores.tif",
    unique=True,
)

1
sam3.show_masks(cmap="coolwarm")

scores

Visualize Confidence Scores

Let's visualize the confidence scores to see which masks have higher confidence:

1
2
3
4
5
6
7
8
9
m.add_raster("building_masks.tif", layer_name="Building masks", visible=False)
m.add_raster(
    "building_scores.tif",
    layer_name="Building scores",
    cmap="coolwarm",
    opacity=0.8,
    nodata=0,
)
m

map