Archive of XFC Results

Congratulation to everyone who made the inaugural Explainable Fuzzy Challenge a success! The organizers are grateful to everyone that helped and contributed including the judges from universities and the North American Fuzzy Processing Society (NAFIPS) that all have fuzzy logic/AI expertise, UC, NAFIPS, and this year’s sponsor Thales. We’re excited to build on this to create an even better experience next year and drive this to be an ongoing competition to engage students and push the boundaries of explainable, Fuzzy-based AI. Also congratulations to all of the students that participated this year. They worked hard and developed exciting solutions in a very limited time. Without further ado, the winners for each prize for the competitions are as follows: 

Overall 

• 1st place: Team Asimov – Wesley Bumpus, Kate O’Grady, Lester Roberts 
• 2nd place: Harry Stamper’s Crew – Sam King 
• 3rd place: Team HeiTerry – Daniel Heitmeyer, Matthew Terry

Other Awards

• Most Explainable Award: Harry Stamper’s Crew – Sam King 
• Most Innovative Award: Team Asimov – Wesley Bumpus, Kate O’Grady, Lester Roberts 
• Most Computationally Efficient Award: Team Asimov – Wesley Bumpus, Kate O’Grady, Lester Roberts

Congratulations again to the teams! 

View team’s controller’s in action. Check out the summary and breakdown of the performance of the teams.

One of the biggest issues facing widespread adoption of Artificial Intelligence (AI) in industry is the transparency of the decision-making process. Many AI systems are being used with the sole objective of maximizing both accuracy in the prediction and computational efficiency. However, the value of the explainability of these systems is often underestimated. Explainability, in part, refers to the human interpretability of the processes and predictions given by the system. The more explainable a certain algorithm is, the simpler it will be for a human to understand or explain the underlying reasoning. When it comes to critical decisions, humans expect the basis of those decisions to be such that they are easy to verify and corroborate.

In this challenge, teams of 2-6 students constructed an explainable AI system based on Fuzzy Inference Systems to play the Python arcade game “Asteroid Smasher”. The challenge took place throughout the 2021 Spring semester and culminated in a virtual competition where student-created AIs played game scenarios created by the organizers. The AIs/teams were scored on several criteria including performance during competition, explainability metrics, etc. Winners  received monetary prizes after the competition (details below).

• The importance of an Explainable AI algorithm
• Different techniques for gradient-free optimization, such as Genetic Algorithms
  
• Application of all of these towards development of an AI agent that can play the game “Asteroid Smasher”

The teams' code had three main parts:

• The platform (main python script) for the Asteroids Arcade Game. This is given to the teams. It refers to all the different background commands and functions that the game needs to work. It has different sections empty that teams filled with their own code in order to make it work. 
• The Fuzzy Inferencing System (FIS). This is the primary ‘brain’ of the XAI, a set of if-then rules that decide how the spacecraft moves. The organizers held a session to teach how to code a simple FIS for a different problem, so the teams had the knowledge needed to develop their own FIS and integrate it in the challenge.
• The Optimization Process. This refers to the way in which the FIS is trained. AI requires a preliminary process called “training”, where the algorithm is exposed to training data/scenarios and its parameters are modified in order to achieve the best “score” possible according to some objective function. One particularly powerful type of optimization algorithm is Genetic Algorithms (GA). GAs are based on Darwinian principles of evolution and survival of the fittest. Through the competition, a free lecture was provided to the participants on this topic. The teams  used GAs or other applicable types of optimization algorithms in order to train their Fuzzy Systems to learn to play the game. There were no restrictions on the learning methods including manual tuning.

The challenge was open to all undergraduate and graduate university students. The format of the challenge was such that it complemented the skills of students enrolled in STEM degrees, but is was not limited only to STEM majors.  

Online meeting where we will go through the challenge and have 2 guest speakers from industry discussing the importance of explainability and the future of Fuzzy AI. Details on how to join the meeting and the time will be sent to the participants after registration.

This will be a recorded online session where we go through all the necessary steps to understand how to build a Fuzzy AI system. More information will be sent closer to the event by email.

The teams will need to use the concepts learnt in the first seminar to build their own solution of a simplified real-world problem using Fuzzy AI. The details of the problem will be given during the first seminar. Guidance in the development to those teams that request it will also be granted. This milestone will contribute to final team scores.

In a similar format to the first class, a lecturer will walk you through the mechanics and implementation of a Genetic Algorithm (GA). We will also discuss the use of a GA in the optimization of a FIS. Other gradient-free optimization algorithms will also be discussed. The students will then need to apply these concepts in their second problem. 

The teams will need to use the concepts learnt in the Optimization seminar to optimize a Fuzzy Inference System. Details on the problem will be given at the end of the seminar. Guidance can be provided as requested by teams and submissions will count towards final evaluation scores.

In this meeting the Asteroid Smasher game will be introduced including constraints and details on controller development. Final details on the competition will also be given at this time including scoring metrics, expectations, and other final deliverables/requirements.

This will be the submission date of the final version of your Fuzzy XAI controller for the Asteroid Smasher game. Details on submission format and requirement will be given to you beforehand by the organizers. The final week between the submission date and the competition date will be used to ensure any issues are resolved. Submissions must be received by 11:59 PM EST. 

Finally, after all the hard work, the students and advisors will be recognized in this final event that will take place virtually. Teams will have the opportunity to describe what they did and their approach to creating their Fuzzy XAI systems. Then they will see their creations be put to the test in a series of scenarios created by the organizers. The teams will be scored and awards will be given out based on numerous criteria. 

• Fuzzy System deliverable
• Optimization deliverable
• Final Asteroid Smasher Fuzzy XAI performance: Agents will be evaluated against a portfolio of scenarios with a custom fitness function
• Explainability of the created AI
• Creativity in the solutions

There were several awards for the winners and for those who show creativity in their submissions! Monetary awards were given to the top 3 teams based on their overall competition score:

• First place: $1400
• Second place:$900
• Third place $400

Additionally, the following awards were granted ($100 each): 

• Most Explainable Solution
• Most Innovative Solution
• Most Computationally Efficient AI

Thank you to our Judges!

• Barnabas Bede PhD, Associate Professor; Program Director Bachelor of Science in Computer Science in Machine Learning; Department Chair of Mathematics; DigiPen Institute of Technology
• Alex Gonzalez, Research Assistant at Purdue Computer and Information Technology
• Xiaonan(Shannon) Jing. PhD student major in Computer and Information Technology at Purdue University

• Thales

• Fuzzy Logic
• Evolutionary Algorithms
• Particle Swarms 
• A Visual Guide to Evolution Strategies
• Lightweight Covariance Matrix Adaptation Evolution Strategy (CMA-ES) implementation for Python 3
• Python Magic methods/”Dunder” methods
• Object Oriented Programming
• Software Design Patterns