Video Segmentation and Object Tracking with SAM 3¶

This notebook demonstrates how to use SAM 3 for video segmentation and object tracking.

Installation¶

SAM 3 requires CUDA-capable GPU. Install with:

1	`# %pip install "segment-geospatial[samgeo3]"`

import os
from samgeo import SamGeo3Video, download_file

The SamGeo3Video class provides a simplified API for video segmentation. It automatically uses all available GPUs.

sam = SamGeo3Video()

You can load from different sources: - MP4 video file - Directory of JPEG frames - Directory of GeoTIFFs (for remote sensing time series)

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/basketball.mp4"
video_path = download_file(url)

sam.set_video(video_path)

sam.show_video(video_path)

Use natural language to describe objects. SAM 3 finds all instances and tracks them.

# Segment all players in the video
sam.generate_masks("player")

Customize player names:

player_names = {}
for i in range(15):
    player_names[i] = f"Player {i}"
sam.show_frame(0, axis="on", show_ids=player_names)

# Remove objects and re-propagate
sam.remove_object(obj_id=[5, 8, 12, 13])
sam.propagate()
sam.show_frame(0, show_ids=player_names)

Save masks as images or create an output video.

os.makedirs("output", exist_ok=True)

# Save mask images
sam.save_masks("output/masks")

# Save video with blended masks
sam.save_video("output/players_segmented.mp4", fps=60, show_ids=player_names)

sam.show_video("output/players_segmented.mp4")

Close the session to free GPU resources.

sam.close()

To completely shutdown and free all resources:

sam.shutdown()