Video Segmentation with SAM 3¶

This notebook demonstrates how to use SAM 3 for video segmentation and tracking. SAM 3 provides:

Text prompts: Segment objects using natural language (e.g., "person", "car")
Point prompts: Add clicks to segment and refine objects
Object tracking: Track segmented objects across all video frames
Time series support: Process GeoTIFF time series with georeferencing

Installation¶

SAM 3 requires CUDA-capable GPU. Install with:

1	`# %pip install "segment-geospatial[samgeo3]"`

Import Libraries¶

import os
from samgeo import SamGeo3Video, download_file

Initialize Video Predictor¶

The SamGeo3Video class provides a simplified API for video segmentation. It automatically uses all available GPUs.

sam = SamGeo3Video()

Load a Video¶

You can load from different sources: - MP4 video file - Directory of JPEG frames - Directory of GeoTIFFs (for remote sensing time series)

url = "https://github.com/opengeos/datasets/releases/download/videos/cars.mp4"
video_path = download_file(url)

sam.set_video(video_path)

sam.show_video(video_path)

Text-Prompted Segmentation¶

Use natural language to describe objects. SAM 3 finds all instances and tracks them.

# Segment all car in the video
sam.generate_masks("car")

Visualize Results¶

# Show the first frame with masks
sam.show_frame(0, axis="on")

# Show multiple frames in a grid
sam.show_frames(frame_stride=20, ncols=3)

Remove Objects¶

Remove specific objects by ID and re-propagate.

# Remove object 2 and re-propagate
sam.remove_object(2)
sam.propagate()
sam.show_frame(0)

Point Prompts¶

Add objects back or refine segmentation using point prompts.

# Add back object 2 with a positive point click
sam.add_point_prompts(
    points=[[335, 203]],  # [x, y] coordinates
    labels=[1],  # 1=positive, 0=negative
    obj_id=2,
    frame_idx=0,
)
sam.propagate()
sam.show_frame(0)

Refine with Multiple Points¶

Use positive and negative points to refine the mask.

# Refine to segment only the shirt (not pants)
sam.add_point_prompts(
    points=[[335, 195], [335, 220]],  # detect windshield, not the car
    labels=[1, 0],  # positive, negative
    obj_id=2,
    frame_idx=0,
)
sam.propagate()
sam.show_frames(frame_stride=20, ncols=3)

Save Results¶

Save masks as images or create an output video.

os.makedirs("output", exist_ok=True)

# Save mask images
sam.save_masks("output/masks")

# Save video with blended masks
sam.save_video("output/segmented.mp4", fps=25)

Close Session¶

Close the session to free GPU resources.

sam.close()

To completely shutdown and free all resources:

sam.shutdown()