YAY! I kind of did it! I created a reasonably fast traffic sign/light detector. The software truely does not work perfectly and also detection of gray/white signs is still quite hard. However it basically works, especially for red, blue and yellow traffic signs using only really limited hardware. There is still much room left to improve. Especially a bigger dataset would be really nice. But annotating files is really time-consuming. The lane keeping assistant is also missing. For now this is totaly ok, because I have to concentrate on other things now. I will come back to this project. The next steps for the remaining days include preparation of code (so I can share it with you) and doing a small video driving around. After recording the video I want to extract every 1s one frame and let the current system recognize on every frame the traffic signs. In the future I plan to have something like a webcam inside of my car. However this will come in future.
We have to annotate like 800 images more + 2000-3000 images, which I gonna collect and annotate over the next weeks. After we finished annotating, I will apply some really basic face and car plate detection to censor these parts. On this way I want to avoid running into any privacy problems.
My system I used for the benchmarks was my “Training-PC”, which consists of:
RAM: 2x 4GB Kingston ValueRAM DDR4-2133 DIMM CL15-15-15 Single
Mainboard: ASRock H110M-ITX Intel H110 So.1151 Dual Channel DDR4 Mini-ITX
Harddrive: SSD 128GB Intenso Top III SATA 6Gb/s MLC
CPU: Intel Pentium G4400 2x 3.30GHz So.1151
GPU: 6GB Palit GeForce GTX 1060 DUAL
You see on CPU I get about 1-5 FPS depending on the amount of ROIs found, whereby on GPU I am able to achieve 10-15 FPS. If I would optimize the code running on CPU more (e.g. add multithreading) and I would use a better GPU, I may be able to reach up to 20 FPS, maybe even more. Therefore we would be able to easily find traffic signs in real time.
Some final results, where blue boxes denote ROIs (so regions which are possible traffic signs/lights) and red boxes denote detected traffic signs. These results are not perfect, neither finding ROIs nor the classifier itself. However you have to remember that I wanted to build a software system, which runs completly on CPU with 1-2 FPS. So I had to sacrifice detection accuracy. The presented results were not contained in the trainingset. So these images are all taken from testset.