Skip to content

Transition Plan Outline

Relevant Files

  • Relevant 3DGS Files:
    • /workspace/smurfstudio/model_training/splatfacto_train_example.py
    • /workspace/smurfstudio/nerfstudio/models/splatfacto.py
    • /workspace/smurfstudio/nerfstudio/utils/splatfacto_utils.py
    • /workspace/smurfstudio/nerfstudio/fields/splatfacto_appearance_field.py
    • /workspace/smurfstudio/nerfstudio/utils/splatfacto_field_utils.py
    • /workspace/smurfstudio/nerfstudio/data/dataparsers/splatfacto_dataparser.py
    • /workspace/smurfstudio/nerfstudio/utils/render_helpers.py
    • /workspace/smurfstudio/nerfstudio/utils/shadow_splatting_helpers.py
  • Relevant gsplat files:
    • Density Control Strategies: /workspace/smurfstudio/submodules/gsplat/gsplat/strategy/
    • /workspace/smurfstudio/submodules/gsplat/gsplat/rendering.py
    • /workspace/smurfstudio/submodules/gsplat/gsplat/cuda/_wrapper.py
    • /workspace/smurfstudio/submodules/gsplat/gsplat/cuda/csrc/rasterize_to_pixels_fwd.cu
    • /workspace/smurfstudio/submodules/gsplat/gsplat/cuda/csrc/bindings.h
  • Dataset Preparation Files:
    • /workspace/smurfstudio/specific_data_scripts/combine_flat_boxsat_sun_angles.py (change into nerfstudio datasets)
    • /workspace/smurfstudio/specific_data_scripts/generate_transforms_all.py (change into nerfstudio datasets)
    • /workspace/smurfstudio/nerfstudio/scripts/datasets/stl_scene_to_ply.py
    • (Occasionally others in /workspace/smurfstudio/nerfstudio/scripts/datasets/)
  • Primary evaluation Files
    • /workspace/smurfstudio/experiments/basic_dynamic_relighting_shadowing_eval.py (change into nerfstudio)
    • /workspace/smurfstudio/experiments/splatfacto_3d_metrics.py (change into nerfstudio)

Important Topics:

  • Rasterization
  • Components of a 3DGS model and composing multiple 3DGS models
  • Shadow mapping and shadow splatting mechanics

Minimum Workflow:

  • Modify splatfacto_train_example.py to desired configuration
  • Train splatfacto model
  • Run basic_dynamic_relighting_shadowing_eval.py to evaluate performance of a given model on a specific dataset
    • Optionally run comparisons between evaluated runs using /workspace/smurfstudio/experiments/compare_dynamic_relighting_shadowing_runs.py

Flashcards:

  • Cameras
    • Extrinsic + intrinsic matrices
    • Perspective vs. orthographic projection + effect of focal length in nerfstudio

  • Rasterization/Rendering terms

    • Transmittance
    • Alpha compositing/blending
    • Support radius

  • Shadowing methods

    • Shadow mapping
    • Shadow splatting
    • Shadow acne
    • Peter panning
    • Traditional receiver bias
    • per-Gaussian SRB
  • Metrics and what they do:
    • Fill in