This project is part of my bachelor’s thesis.
The goal is to get a gopigo car to detect lanes provided by the raspberry pi camera v2. Return an information about the direction of the lane and keep the lane with a p-controller. Additionally, I implemented an obstacle detection with a haar cascade for cars.


Pipeline Pipeline

The workthrough of the lane detection and lane keeping is the following:

  • A possibly distorted input image is provided by the raspberry pi camera. With the file the input image is getting undistorted.

Undistorted distorted Image Undistorted Image

  • After that the region of intested is being set. A lot of different lane detection projects use a trapezoid for the ROI, but this wasn’t possible for this project since in turns the inner lane disappears and I need a the information that I get. ROI also helps with the computing power needed -> smaller images, faster computation

cropped image

  • The image process contains of canny edge detection and a threshold image. The combination of both is the combo_image and is used to warp the image. To visualize the lanes a hough transformation is used (right image)

canny image theshold image combo image combo image

  • The warping of the image into a birdeye-view provids an optimal image perspective to extract the lane information especially in curved lanes.

birdeye image

  • To calculate the aimed trajectory the birdeye-image is being halfed. This halfed image is being scanned for the lanes. As a return a few middlepoints are generated and averaged. The trajectory is drawn into the birdeye-image from the middle of vehicle (bottom of the image) to the half of the image. As a x-value the averaged middlepoints is used. After that I rewarped the image to display the image in normal perspective as well!

birdeye image with trajectory image with trajectory

  • The detect_lanes_img function returnes the direction of the trajectory and the center of the car for a variance analysis.

  • The last part is the implementation of a p-controller that is correcting the error between the trajectory and the middle of the car

The workthrough of the obstacle detection is the following:

  • First you need a haar cascade for cars -> cars.xml
  • After that take a reference image of an obstacle you want to detect
  • Meassure the distance to the object and the width of the object and correct the variables KNOWN_DISTANCE and KNOWN_WIDTH
  • Adjust the wanted distance before stopping
  • After that drive onto the object and hope that all goes to play and nothing get’s smashed ?
    Obstacle Obstacle


View Github