From pixels
to perception.

Warehouse perception needs to recognize pedestrians reliably across changing lighting, camera angles, clothing, and partial occlusion. I fine-tuned RF-DETR on 7,000 annotated warehouse images to improve pedestrian recall from 62% to 97%, achieve approximately 78% mAP@50:95, and extend the effective detection range from 4 to 8 metres.

To bring the model onto Jetson AGX Orin, I optimized inference with TensorRT FP16 and built a GPU processing pipeline using CUDA preprocessing, GPU postprocessing, asynchronous streams, and CUDA Graphs. These changes accelerated RF-DETR Medium by approximately 2.2–2.6×, with the processing pipeline taking about 8–12 ms per image.

I developed an RGB-D visual odometry pipeline to maintain continuity in reference-image-based forklift localization. The system matches ORB features between frames, uses depth to establish 3D–2D correspondences, and estimates motion with PnP-RANSAC. Kalman filtering refines the vehicle’s position and trajectory from a known starting pose.

I then combined visual odometry with IMU measurements and CAN-derived wheel odometry through an Extended Kalman Filter, bringing multiple sources of motion information into the position and orientation estimate. Asynchronous processing and frame buffering reduced processing latency from roughly 6 to 3 ms, while dropped frames fell from 10–12 to about 6 per second on a 30-FPS stream.

This project connected pedestrian detection with depth measurements so collision-zone checks could use real-world distances. I developed checkerboard-based intrinsic and extrinsic calibration, stereo rectification, and SGBM depth estimation with WLS filtering for Leopard stereo cameras. Combining that depth with 2D pedestrian detections reduced false collision-zone triggers by 70%.

I also fine-tuned a unified RF-DETR Medium detector for forklifts, pedestrians, and stop signs, replacing two task-specific models. On AGX Orin, the unified model achieved 93% mAP@50 and 75.3% mAP@50:95, while aggregate model execution time decreased from approximately 14–20 to 7–10 ms per frame.

I built a reusable C++ sensor framework for x86 and ARM64 to make camera integration more consistent. Camera plugins handle buffered, multithreaded acquisition, calibration, and image-format conversion across Framos, Genie, Leopard, and RTSP sources. This reduced the time needed to integrate a new camera from one week to one day.

For faster experimentation, I developed multithreaded Focus and Configuration Managers using the observer pattern. Engineers can update live JSON configurations without rebuilding or restarting the perception pipeline; pause–update–resume coordination keeps concurrent components synchronized. This reduced recompilation time during tuning by 60%.

I also led a three-engineer build-system migration, first to Bazel and later to Conan/CMake. Azure DevOps pipelines covered builds, tests, linting, cppcheck, and coverage, supported by Buildfarm and Redis for Bazel and Linux build images for Conan/CMake.

Static-camera pallet monitoring needs to distinguish real changes in occupancy from clutter and people moving through the scene. I added depth-based geometric detection and human masking to a pipeline using YOLOX pallet counts.

The combined system uses object counts and geometric occupancy signals to identify pallet additions, removals, and persistent occupancy anomalies. These changes reduced clutter-driven false positives by 14% relative to the baseline.