I’m Walking Here!: Pedestrian Intent Recognition

Spring 2018, CS 426, University of Nevada

Team 6

Advisors: Richard Kelley, Sergiu Dascalu

Instructors: Sergiu Dascalu, Devrin Lee

About

Autonomous vehicles can already drive. That technology already exists today! The big problem with autonomous vehicles currently is that we don’t know how to safely navigate in highly crowded scenarios. Humans already have that ability: we can easily infer where a person might want to go based on where that person is facing and how they are moving.

The goal of our senior project is to allow autonomous navigation to estimate a pedestrian’s intention and future trajectory based on their body posture. This will allow autonomous vehicles to use the data already encoded in how a person is standing or walking to more dynamically plan a path in crowded, urban scenarios.

We plan to do this using a neural network trained on 3D video sequences of pedestrians. We will be detecting the 3D position, posture, and path of the pedestrians using LIDAR and GMSL camera data and the Open Pose library from Carnegie Mellon University. This will be implemented on the newly acquired University of Nevada Autonomous Stuff autonomous platform by May 2018.

This project will allow autonomous vehicles to more fluidly plan safe paths in crowded scenarios. It will also allow the team members experience in working with state of the art technology and software systems!

Gaetano Evangelista

Gaetano will be graduating with a Bachelors Degree in Computer Science with a minor in Mathematics in the Spring semester of 2018 from the University of Nevada, Reno. He will then be pursuing a Doctorate in Computer Science at the University of Nevada, Reno. His professional interests include Machine Learning and Modern Art. His hobbies include 3D Printing, food, and indie gaming.

Cayler Miley

Cayler is a senior (as of 2018) with intentions to go to Stanford for graduate school or work on automation for Ford or Tesla. Cayler is an honors student who studied abroad in Germany to learn about manufacturing and intern at Arculus GmbH. Cayler is also completing the cybersecurity and battery and energy storage technology minors. Cayler’s crowning undergraduate achievement was receiving the Goldwater Scholarship honorable mention. In his free time, Cayler likes to play guitar, golf, and play video games.

Houston Lucas

Houston will be graduating with a Bachelors Degree in Computer Science with a minor in Mathematics in the Spring semester of 2018 from the University of Nevada, Reno. He will then be pursuing a career in software development in industry. His professional interests include Machine Learning and Aerospace Engineering. His hobbies include Kerbal Space Program, skateboarding, and heavy metal.

Jamie Poston

Developer

Jamie will be graduating with a Bachelor’s Degree in Computer Science with a minor in Mathematics in the spring semester of 2018 from the University of Nevada, Reno. She will then be pursuing further education by getting her Master’s degree in Computer Science at UNR. Her professional interests include artificial intelligence and machine learning. Her hobbies include video games, gardening, and crafting.

%

Project Completion

Hardware Installation 100% 100%
Algorithm Design 100% 100%
Sensor Calibration 100% 100%
Pose Recognition 90% 90%
Neural Network Training 60% 60%
Recurrent Neural Network Installation 10% 10%

Innovation Day

Day(s)

:

Hour(s)

:

Minute(s)

:

Second(s)

Automated Driving

Safer and More Efficient Future Driving

Daniel Watzenig and Martin Horn

Recurrent Neural Networks
- This website describes the process of setting up a recurrent neural network to process sequences of natural language. This project will be using recurrent neural networks as well to process sequences of video, so this tutorial will prove useful. While this tutorial is specifically using TensorFlow as the machine learning library, the overall idea of a recurrent neural network that is described in this tutorial will be helpful.
LIDAR Calibration
- During the course of this project, the Velodyne LIDAR sensor will be fixed onto the car. In order to get a correct calibration of the LIDAR sensor, this software will be used in combination with real-world calibration props. This will allow an accurate 3D model of the world to inputted into the PIR software.
3D-to-2D Transposition
- This online lesson details the process of converting a 3D point into 2D image coordinates. As part of the PIR project includes creating a 3D data set from a 2D pose estimation, this lesson will be a particularly useful reference.

Real-time Multi-Person 2D Pose Estimation using Part Affinity Fields; Cao et al. CVPR 2017
- This paper formally describes an approach to estimate the pose of multiple people in a 2D image. This novel approach is much faster and more efficient than other estimations like it. In the PIR project, this 2D pose estimation will be translated into 3D pose estimation and used to create a data set to train the neural network on.
The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Driving; Sunberg et al. CoRR 2017
- This paper proposes a model for the behavior of traffic participants. This is to create an accurate estimation of human drivers for autonomous vehicles. While the method of transportation is different, drivers and pedestrians have similar goals. Thus, this paper gives insight into modeling the behavior of all traffic participants, including pedestrians.
ROS: An Open-Source Robot Operating System; Quigley et al. ICRA 2009
- This paper gives an overview of the ROS software framework. A large majority of the functions on the Autonomous Stuff vehicle run via ROS. As this project will be integrating software into the autonomous system, PIR will need to run within the ROS framework.

You probably won’t own a self-driving car, but you’ll ride in them a lot

You probably won’t own a self-driving car, but you’ll ride in them a lot Popular Science

1. Uber and Lyft Have a Hot New Idea for Screwing Over City-Dwellers Gizmodo
2. Uber and co don’t want you to own your own self-driving car TechRadar