Graduate Research Assistant (Jun 2014 - Aug 2017)

After I graduated with my Bachelor Degree, I was hired by the professor in charge of the previous apprenticeship program as a research assistant. I was hired using his personal grant funding. I held this position for the summer before I started graduate school and the entire time I was in graduate school.

This job was focused on research in robotics, reinforcement learning, and machine learning. Many of our projects were robotics related. The majority of those projects involved incorporating different kinds of machine learning algorithms. There were far too many projects worked on during this time to discuss in depth in this section. I will begin listing some under the projects section of this website.

The following is a brief list of some select duties and activities performed on this job:


Undergraduate Research Assistant (May 2013 - Aug 2013)

This was a summer apprenticeship under the U.S. Army Research Office Undergraduate Research Apprenticeship Program (URAP).

This apprenticeship consisted of a single professor, a graduate student, and a group of three undergraduate students. A brief summary of the program follows. The project was initially advertised as the following:

The objective of this research is to develop a decision theoretic framework for multi-robotic systems, to serve as a tool for analysis of prevalent swarm intelligence techniques. The students will gain basic knowledge about game and decision theories, reinforcement learning, distributed AI and multi-robotics, and hands-on experience with robot swarms – both in simulation (Webots) and with real robots.

Over the first few weeks of the program, we learned basic artificial intelligence techniques that would be used. We focused mostly on the Markov Decision Process and Q-Learning. While initially planning to use Webots, that program was abandoned in favor of a Kilobot simulator called KBSim. We made several modifications to the code of this simulator to better suit our needs.

Next, we got familiar with the simulator by implementing some simple algorithms. Some of the algorithms used were a foraging algorithm where the kilobots cooperated to search for food. We implemented a simple obstacle avoidance algorithm. We also implemented the Boids algorithm which simulates animal flocking behavior.

We spent the next several weeks reviewing many published papers that related to the research we were working on.

After this research was complete, we began work on one of the problems we were assigned which was using some of the previously studied techniques to cause the robots to spread out covering a maximum area. After implementing and testing these methods in our python simulator to verify that they worked, we needed to do more testing. Since the python simulator was slow, we created a non visual simulator in C++ which let us run the algorithms and get results more quickly. This C++ code would output files that could then be loaded back into the python simulator to visually see what the robots were doing. The files that were loaded were called policies, and they made use of a Periodic Finite State Controller in order to control the robots.

We successfully achieved getting the robots to disperse using the methods we learned during the program. As part of the URAP program, we were required to make a presentation that covered everything we worked on during the apprenticeship. The video of that presentation can be seen below.