3D Vision — Point Clouds & NeRFs
3D vision comes in two flavours. Point clouds are the sensor's raw output. NeRFs are the learned volumetric field. Both answer "what is where in space." Distinguish explicit (point cloud, mesh, voxel) and implicit (signed distance field, NeRF) 3D representations and when each is used. Understand PointNet's symmetric-function trick that makes a neural network permutation-invariant over an unordered set of points. Trace a NeRF forward pass: ray casting, volumetric rendering, positional encoding, MLP density+colour head. Use nerfstudio or instant-ngp for pretrained 3D reconstruction from a small set of posed images. A camera produces a 2D image. A LIDAR produces a set of 3D points with no ordering. A structure-from-motion pipeline produces a sparse cloud of 3D keypoints. A NeRF reconstructs an entire 3D scene from a handful of posed images. All of these are "vision" but none of them look like the dense tensor a CNN wants. 3D vision matters because almost every high-value robot task runs in 3D: grasping, obstacle avoidance, navigation, AR occlusion, 3D content capture. A vision engineer who only understands 2D images is locked out of the fastest-growing slice of the field (AR/VR content, robotics, autonomous driving stacks, NeRF-based 3D reconstruction for real-estate or construction). The two representations dominate for different reasons. Point clouds are what sensors give you for free. NeRFs and their successors (3D…
3D Vision — Point Clouds & NeRFs: 3D vision comes in two flavours. Point clouds are the sensor's raw output. NeRFs are the learned volumetric field. Both…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.