Spatial Constraint Generation for Motion Planning in Dynamic Environments

Huawei Noah's Ark Lab | May. 2020 - Jan. 2021

I developed a novel method to generate spatial constraints for motion planning in dynamic environments. Motion planning methods for autonomous driving rely on spatial constraints imposed by a semantic map to generate a collision-free trajectory. These methods may fail without an offline map or where the map is invalid due to changes; road obstruction, construction, and traffic congestion.

I proposed a novel method to generate spatial constraints as an alternative to using semantic maps. It generates a sequence of connected triangles from a set of triangulation meshes as the spatial constraint. This method accounts for the dynamic changes in the mesh connectivity and the velocity information masking problem that the existing triangulation-based methods fail to consider.

Experiments showed that the proposed method results in a more stable, long-term plan with a higher task completion rate, faster arrival time, a higher rate of successful plans, and fewer collisions compared to existing methods.

A Sim-to-Real Pipeline for Deep Reinforcement Learning for Autonomous Robot Navigation in Cluttered Rough Terrain

Autonomous Systems and Biomechatronics Lab | Sep. 2018 - Aug. 2021

This project proposed a novel sim-to-real pipeline that allows a wheeled robot to learn to navigate real-world 3D rough terrain environments using deep reinforcement learning.

The sim-to-real transfer was achieved using:

1. A simulation environment with high geometric fidelity developed by me using 3D mapping and 3D meshing algorithms.
2. Novel domain randomzation parameters proposed by me to specifically address challenges unique to 3D rough terrain.

I led my lab members to conduct experiments in the real-world 3D cluttered environment. We verified that the pipeline allowed point-to-point navigation in rough terrain with a high success rate of 86.6% given a simulation success rate of 90%.

I have led other lab members to conduct a comparison study to demonstrate that deep reinforcement learning based rough terrain navigation is more successful and efficient compared against existing deep learning based and classical methods.

I've built the software architecture and the hardware augmentation of the robot used in the experiments. I implemented a Multi-Master ROS based framework to allow multiple onboard computing units (Intel NUC and Nvidia Jetson TX2) for effcient load allocation. The onboard computing units are connected over Ethernet to maximize data transfer. I equipt the robot with both visual SLAM and LiDAR SLAM for robust and accurate localization under a noisy environment. I also implemented a cascade PID controller to minimize slippage issues and maximize motion accuracy in the rough terrain environment.

Parallel Proximal Policy Optimization

Personal Project | Nov. 2019

This project was a parallel version of PPO reinforcement learning algorithm, modified from an existing implementation. The purpose of this was to decrease training time by using parallel processes to simultaneously collect training samples. The collective experiences of all agents were pooled into a reply buffer that is sampled from when updating the network.

Apprenticeship Inverse Rinforcement Learning

Course Project | Mar. 2019

This was my implementation of the Apprenticeship Learning via Inverse Reinforcement Learning paper using traditional Q-learning.

This inverse RL algorithm uses an expert policy to learn the weights of a reward function assumed to be a linear combination of the input features. It fully trains a student policy with an RL algorithm using the returns computed with the weights at each training iteration. The weights are iteratively improved by moving in the the feature expectation of the expert policy. This direction is found by orthogonally projecting the feature expectation of the expert onto the line between the feature expectation of the current and the previous student. Any RL algorithm can be used with this algorithm to trian the students.

Toward Smart Cities: Data-driven Road Accident Prevention

Course Project | Sep. 2018 - Dec. 2018

In a team of 5, we developed DriveAlive; a machine learning pipeline to quantify regions prone to traffic accidents within the city. It provides a simple, inexpensive, and robust alternative capable of addressing a growing safety concern in a world moving towards smart cities and higher rates of urbanization.

The pipeline used historical traffic accident data, current road conditions, and the weather as feature inputs to predict the occurrence of traffic accidents. Datasets from Vision Zero Project, Toronto Police Service, and Toronto Road Maps were cross-mapped and cleaned. Negative sampling was applied to the cleaned data and combined into the training data.

Multiple classification models were tested, including Random Forest, and a deep MLP network. The Random forest classifier achieved the highest cross-validated accuracy of 90.6%.

DriveAlive can be integrated into mapping services to predict the safest optimal path between routes. Local municipalities and governments can use it to identify features associated with high-risk areas. City planning and infrastructure project in consideration of the features can mitigate the occurrence of accidents in the future.

Data Analytics Pipeline

Course Project | Aug. 2019

I built a sentiment analysis pipeline that analyzed the customer's opinion of US airlines from raw Twitter feeds using natural language processing. The pipeline achieved high accuracy of 80.9% for sentiment prediction 65% for reasoning precition and under limited data.

I also build a salary prediction pipeline for the 2018 Kaggle ML & DS Survey Challenge. This pipeline used a multiple-choice questionnair to predict the salaries of data science related job.

Autonomous Maze Solving Rover

Course Project | Sep. 2017 - Dec. 2017

This 5 member team project was to design an autonomous rover to find and deliver a wooden block to a location given random start. I was responsible for navigation, localization, obstacle avoidance, and sensor design and implementation.

The rover uses an X-holonomic drive for motion and a 2 DOF manipulator to pick up the block. An Arduino was used as the onboard microcontroller. It uses Bluetooth to communicate with a PC that uses MATLAB for ultrasound-based 2D histogram localization and A* Path planning.

The rover first randomly walks within the maze to localize itself. It then uses A* to plan a path to a predestinated pickup zone. While in the pickup zone, it uses its front-side ultrasonic sensors to search and pick the wooden block. Once picked up, the rover navigates to a randomly selected drop zone to deliver the block.

The X-holonomic drive system had the unexpected challenge of random skidding, which caused significant challenges to the rover's motion control. I implemented features such as virtual walls and real-time localization to avoid collision and reduce the effect of the skid.

Design of Variable Guide Vane Mechanism for Pico-Scale Fracis Hydro Turbine

Water and Energy Research Laboratory | Jan. 2018 - Sep. 2018

This project was a collaboration with a Toronto-based start-up company as a part of a larger effort to develop an energy harvester for a self-powered, inline water disinfection system to provide safe drinking water and electricity to remote communities.

I developed a design of a variable guide vane mechanism for a palm-sized hydro Francis turbine. It can replace the conventional pressure control valve by regulating the pressure with a set of variable guide vanes and harness the energy from the hydraulic head that is overwise lost with a conventional pressure control valve.

The guide vanes regulate the fluid flow by reducing the flow's cross-sectional area and changing its angle of attack to the turbine blades. The guide vanes are actuated via a set of levers and pivot pints connected to a guide ring. A stepper motor with a high reduction ratio gearbox is controlled by an Arduino to rotate the guide ring. A pressurized pipe flow rig was developed in collaboration with two other master students to demonstrate the feasibility of the design.

This research won the 1st place undergraduate poster in the 2018 Computational Fluid Dynamics Society of Canada conference.

autonomous Turtlebot

Course Project | Sep. 2017 - Dec. 2017

This was a 5 member project that consists of 3 contests implmented under the ROS framework.

Contest 1: Autonomous Mapping and Exploration of an Unknown Environment

The objective was to autonomously explore an unknown environment and use GMapping to map the explored area. I developed the motion control and a random walk based exploration strategy that employed a odometry based memory to reduce repeat visits.

Contest 2: Finding Objects of Interest in an Environment

The objective was to navigate the robot to a box object and match the known image on the box using OpenCV. I implemented the feature 2D + Homography algorithm with lowe's ratio test to compute the bounding box area of the image detected from the robot camera to match against the image database.

Contest 3: Follow Me TurtleBot Companion

The objective was to develop a social companion robotthat exhibits emotional responses to external stimuli. I implemented low level emotional state behaviors and their respective transition triggers.

Droplet Impingement

Multiphase Flow and Spray Systems Lab | May. 2016 - Aug. 2017

This research studied the droplet shattering pattern under impingement by an air jet to disover new methods of atomization for spray technologies. It has applications in spray and atomization, aircraft engine propulsion, and fuel jet injection.

This research analyzed the droplet's shatter pattern under varying jet velocity, nozzel diameter, droplet diameter, droplet viscosity, and impact location.

13 droplet shattering pattern were discovered and classified.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.