Autonomous Driving Quick Tour

Standards

Automated Vehicles for Safety

The continuing evolution of automotive technology aims to deliver even greater safety benefits and automated driving systems (ADS) that - one day - can handle the whole task of driving when we don't want to or can't do it ourselves. Fully automated cars and trucks that drive us, instead of us driving them, will become a reality.

https://www.nhtsa.gov/technology-innovation/automated-vehicles-safety

• ADAS https://en.wikipedia.org/wiki/Advanced_driver-assistance_systems

Some history

• DARPA Grand Challenge

• Stanley and Sebastian Thrun

Design

Introducing our fifth-generation Waymo Driver

Flexible. All-weather. Human-centered. Take a detailed look at the core design considerations we took in building our fifth-generation Waymo Driver https://b...

https://youtu.be/tXfQc2i-EC4

Vehicle Modeling

Modeling a Vehicle Dynamics System

This example shows nonlinear grey-box modeling of vehicle dynamics. Many new vehicle features (like Electronic Stability Programs (ESP), indirect Tire Pressure Monitoring Systems (TPMS), road-tire friction monitoring systems, and so forth) rely on models of the underlying vehicle dynamics. The so-called bicycle vehicle model is a rather simple model structure that is frequently being used in the vehicle dynamics literature.

https://www.mathworks.com/help/ident/ug/modeling-a-vehicle-dynamics-system.html

Bicycle vehicle model - MATLAB

Define a robot and set the initial starting position and orientation. Simulate Robot Motion Set the timespan of the simulation to 1 s with 0.05 s timesteps and the input commands to 2 m/s and left turn. Simulate the motion of the robot by using the solver on the derivative function.

https://www.mathworks.com/help/robotics/ref/bicyclekinematics.html

Control

PID controller

A proportional-integral-derivative controller ( PID controller or three-term controller) is a control loop mechanism employing feedback that is widely used in industrial control systems and a variety of other applications requiring continuously modulated control.

https://en.wikipedia.org/wiki/PID_controller

Sensing

Ego movement

Inertial measurement unit

An inertial measurement unit ( IMU) is an electronic device that measures and reports a body's specific force, angular rate, and sometimes the orientation of the body, using a combination of accelerometers, gyroscopes, and sometimes magnetometers. IMUs are typically used to maneuver aircraft (an attitude and heading reference system), including unmanned aerial vehicles (UAVs), among many others, and spacecraft, including satellites and landers.

https://en.wikipedia.org/wiki/Inertial_measurement_unit

• Accelerometer, Gyroscope, Magnetometer

• Yaw, pitch, roll, Gimbal lock, and Quaternions

Localization

Lidar

Lidar (, also LIDAR, LiDAR, and LADAR) is a method for measuring distances ( ranging) by illuminating the target with laser light and measuring the reflection with a sensor. Differences in laser return times and wavelengths can then be used to make digital 3-D representations of the target.

https://en.wikipedia.org/wiki/Lidar

Radar

Radar is a detection system that uses radio waves to determine the range, angle, or velocity of objects. It can be used to detect aircraft, ships, spacecraft, guided missiles, motor vehicles, weather formations, and terrain.

https://en.wikipedia.org/wiki/Radar

Sonar

Sonar ( sound navigation ranging) is a technique that uses sound propagation (usually underwater, as in submarine navigation) to navigate, communicate with or detect objects on or under the surface of the water, such as other vessels.

https://en.wikipedia.org/wiki/Sonar

Visual

OpenCV: Camera Calibration and 3D Reconstruction

int cv::recoverPose ( InputArray E, InputArray points1, InputArray points2, OutputArray R, OutputArray t, double focal=1.0, Point2d pp= Point2d(0, 0), InputOutputArray mask= noArray()) int cv::recoverPose ( InputArray E, InputArray points1, InputArray points2, InputArray cameraMatrix, OutputArray R, OutputArray t, double distanceThresh, InputOutputArray mask= noArray(), OutputArray triangulatedPoints= noArray()) bool cv::solvePnPRansac ( InputArray objectPoints,

https://docs.opencv.org/4.4.0/d9/d0c/group__calib3d.html

Point-cloud interaction

nuScenes

https://www.nuscenes.org/nuscenes

Fusion

What Is Sensor Fusion?

Sensor fusion is the ability to bring together inputs from multiple radars, lidars and cameras to form a single model or image of the environment around a vehicle. The resulting model is more accurate because it balances the strengths of the different sensors.

https://www.aptiv.com/newsroom/article/what-is-sensor-fusion

Perception

Key-point matching

• ORB https://docs.opencv.org/4.4.0/d1/d89/tutorial_py_orb.html

• SIFT https://docs.opencv.org/4.4.0/da/df5/tutorial_py_sift_intro.html

vSLAM

Simultaneous localization and mapping

In computational geometry, simultaneous localization and mapping ( SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. While this initially appears to be a chicken-and-egg problem there are several algorithms known for solving it, at least approximately, in tractable time for certain environments.

https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping

Kalman filter

In statistics and control theory, Kalman filtering, also known as linear quadratic estimation ( LQE), is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe.

https://en.wikipedia.org/wiki/Kalman_filter

• Kalman filtering in Baidu's Apollo https://github.com/ApolloAuto/apollo/blob/master/modules/perception/fusion/common/kalman_filter.cc

• George Hotz live coding SLAM

• Bloomberg interview with Hotz

• TartanAir

TartanAir: A Dataset to Push the Limits of Visual SLAM

Published: by We present a challenging dataset, the TartanAir, for robot navigation task and more. The data is collected in photo-realistic simulation environments in the presence of various light conditions, weather and moving objects.

https://theairlab.org/tartanair-dataset/

Long tail problems

Adversarial attack and security

Phantom of the ADAS

Ben Nassi* Yisroel Mirsky* Dudi Nassi* Raz Ben Netanel* Green** Yuval Elovici* *Ben-Gurion University of the Negev **Independent Tesla Researcher Abstract In this paper, we investigate "split-second phantom attacks," a scientific gap that causes two commercial advanced driver-assistance systems (ADASs), Telsa Model X (HW 2.5 and HW 3) and Mobileye 630, to treat a depthless object that appears for a few milliseconds as a real obstacle/object.

https://www.nassiben.com/phantoms

Prediction

Prediction - Lyft

Level 5 Open Data This dataset includes the logs of movement of cars, cyclists, pedestrians, and other traffic agents encountered by our autonomous fleet. These logs come from processing raw lidar, camera, and radar data through our team's perception systems and are ideal for training motion prediction models.

https://self-driving.lyft.com/level5/prediction/

Planning

Companies

• Tesla Demo

• Waymo simulation, semantic map, HD map

• WeRide 文远知行 城中村视频

• Momenta Demo

• Lyft Level5 Demo

• Cruise (GM) Demo in SF

• Zoox Demo

• Pony.ai Demo

Other videos

• MIT Self-Driving Cars

Societal topics

• commute time

• house pricing

• car ownership

• fuel v.s. charging

• car parks

• road ownership

• megacities? suburban?

• more and cheaper (other types of) robots

• flying cars?

• conflicts around morale delimma v.s. technological advances
• https://en.wikipedia.org/wiki/War_of_the_currents
• http://www.dissident-media.org/infonucleaire/radieux.html
• 1.35 million people die in road accidents worldwide every year — 3,700 deaths a day
• Distracted driving is a leading cause of accidents, causing 25–50% of all crashes