Skip to content

Smurfstudio COLMAP Use

What COLMAP Does

COLMAP performs Structure-from-Motion (SfM): it estimates camera calibration and camera motion from overlapping images.

Its main outputs are:

  • Camera intrinsics, such as focal length and distortion
  • Camera poses for the registered images
  • A sparse 3D point cloud
  • Feature observations and image correspondences

For Smurfstudio, COLMAP is primarily used to recover the camera poses needed to train a NeRF or 3D Gaussian Splatting model.

The general workflow is:

  1. Prepare unposed images or extract frames from a video.
  2. Extract visual features from each image.
  3. Match features between overlapping images.
  4. Reconstruct camera poses and sparse 3D points.
  5. Check the reconstruction.
  6. Convert the COLMAP model into a Nerfstudio transforms.json.
  7. Train the radiance-field model.

COLMAP and Nerfstudio on Curiosity

The most up-to-date COLMAP executable currently used on Curiosity is:

/data/smccurry/smurf/data/tools/colmap-cudss

Use this executable for direct COLMAP commands such as:

feature_extractor
sequential_matcher
exhaustive_matcher
mapper
global_mapper
view_graph_calibrator
model_analyzer

The COLMAP version bundled with the existing Nerfstudio environment is older and does not include some newer commands, including global_mapper. However, commands beginning with ns-, such as ns-process-data, must be run inside the Smurfstudio container because they require the Smurfstudio/Nerfstudio environment.

Therefore, the normal workflow is:

  1. Run COLMAP directly on Curiosity with colmap-cudss.
  2. Enter the Smurfstudio container.
  3. Run ns-process-data using the completed COLMAP reconstruction.

Flag names may differ between COLMAP versions. Use the help output from the exact executable being run when needed:

/data/smccurry/smurf/data/tools/colmap-cudss feature_extractor --help

COLMAP Processing Stages

1. Feature Extraction

Feature extraction reads each image, detects distinctive image locations, and computes a descriptor for each one. COLMAP normally uses SIFT features, which are designed to remain recognizable under changes in image scale, rotation, and moderate viewpoint.

The extractor stores the following in database.db:

  • Image records
  • Camera models and initial intrinsics
  • Feature keypoints
  • Feature descriptors

Common controls include:

  • ImageReader.single_camera: share one camera calibration across all images.
  • ImageReader.camera_model: choose how focal length and lens distortion are represented.
  • SiftExtraction.max_num_features: control how many features are retained.
  • ImageReader.mask_path: prevent features from being extracted in unwanted image regions.

Increasing the feature count can help difficult or low-texture datasets, but increases processing time and memory use.

2. Feature Matching

Feature matching identifies descriptors that likely represent the same physical point in multiple images. COLMAP then checks whether those matches agree with a common geometric relationship: Inliers are matches that agree with the estimated geometry, outliers are matches that do not. Image pairs and camera poses with more inliers are generally considered more reliable.

Sequential Matching

sequential_matcher is intended for ordered data such as:

  • Video frames
  • Camera sweeps
  • Orbital or arc-based image sequences

It primarily matches images that are near each other in filename order.

Exhaustive Matching

exhaustive_matcher attempts to match every image against every other image. It is useful for smaller, unordered image collections but becomes expensive as the number of images increases.

3. Sparse Mapping

Incremental Mapper

The standard mapper command reconstructs the scene incrementally:

  1. Select a strong initial image pair.
  2. Estimate the relative pose.
  3. Triangulate initial 3D points.
  4. Register additional cameras against the existing reconstruction.
  5. Triangulate more points.
  6. Repeatedly perform bundle adjustment.

Bundle adjustment jointly refines:

  • Camera poses
  • Selected camera intrinsics
  • Sparse 3D point positions

The resulting sparse model is normally written to a numbered directory such as:

sparse/0

It contains:

  • cameras.bin
  • images.bin
  • points3D.bin

Global Mapper

global_mapper estimates the camera trajectory from the image view graph more globally before triangulating and refining the structure. Global mapping can work well when the image graph is strongly connected and the relative pose estimates are reliable. Incremental mapping can be more robust when the dataset is difficult, symmetric, or locally ambiguous.

An example command sequence for a real dataset is available in the ADRAS-J Dataset COLMAP Reconstruction Example.