Artificial Intelligence Competition

Explainable Fuzzy AI Challenge (XFC 2021)

Introduction to the Challenge

One of the biggest issues facing widespread adoption of Artificial Intelligence (AI) in industry is the transparency of the decision-making process. Many AI systems are being used with the sole objective of maximizing both accuracy in the prediction and computational efficiency. However, the value of the explainability of these systems is often underestimated. Explainability, in part, refers to the human interpretability of the processes and predictions given by the system. The more explainable a certain algorithm is, the simpler it will be for a human to understand or explain the underlying reasoning. When it comes to critical decisions, humans expect the basis of those decisions to be such that they are easy to verify and corroborate.

In this challenge, teams of 2-6 students will construct an explainable AI system based on Fuzzy Inference Systems to play the Python arcade game “Asteroid Smasher”. The challenge will take place throughout the 2021 Spring semester and culminate in a virtual competition where student-created AIs play game scenarios created by the organizers. The AIs/teams will be scored on several criteria including performance during competition, explainability metrics, etc. Winners will receive monetary prizes after the competition (details below).

What you and your team will learn

  • The importance of an Explainable AI algorithm
  • Different techniques for gradient-free optimization, such as Genetic Algorithms
  • Application of all of these towards development of an AI agent that can play the game “Asteroid Smasher”

The Challenge

Screenshot of the game

You will have to create a fully autonomous XAI algorithm, in Python, that is able to play the Python Arcade Game “Asteroid Smasher”. In the game, a 2-dimensional spacecraft moves to avoid collisions with numerous asteroids that appear. The asteroids have different shapes, sizes and velocities. The spacecraft also has a weapon that can shoot straight ahead. If the projectiles emitted reach any of the target asteroids, they break into smaller pieces. The smallest asteroid pieces disappear after being hit by a projectile. The control system should be able to consider all the different features of the system and determine the movement and shooting decisions of the spacecraft. 

Your code will have three main parts:

  • The platform (main python script) for the Asteroids Arcade Game. This is given to you. It refers to all the different background commands and functions that the game needs to work. It will have different sections empty that your team will need to fill with your own code in order to make it work. 
  • The Fuzzy Inferencing System (FIS). This is mainly the brain of your XAI, a set of if-then rules that decide how the spacecraft moves. We will learn how to code a simple FIS for a different problem, so you have the knowledge needed to develop your own FIS and integrate it in the challenge. A lecture will be given in this topic, where we will cover all the details and tips needed for the coding of this section.
  • The Optimization Process. This refers to the way in which we are teaching the FIS. AI requires a preliminary process called “training”, where we expose the algorithm to training data/scenarios and its parameters are modified in order to achieve the best “score” possible according to some objective function. One particularly powerful type of optimization algorithm is Genetic Algorithms (GA). GAs are based on Darwinian principles of evolution and survival of the fittest. We will also provide a free lecture to the participants in this topic. The teams will need to use GAs or another applicable type of optimization algorithm in order to train their Fuzzy Systems to learn to play the game. There will be no restrictions on the learning methods including manual tuning.

 

Who can participate?

The challenge is open to all undergraduate and graduate university students. The format of the challenge is such that it complements the skills of students enrolled in STEM degrees, but is not limited only to STEM majors.  

Get together a team (max 6 people) to participate and register! You are welcome to consider an expert advisor from your university to help you in the development (optional). However, all work and development must be performed by the students.

Timeline

1/19/2021: Competition Kick-Off

Online meeting where we will go through the challenge and have 2 guest speakers from industry discussing the importance of explainability and the future of Fuzzy AI. Details on how to join the meeting and the time will be sent to the participants after registration.

1/21/2021: Seminar: Fuzzy Inference Systems

This will be a recorded online session where we go through all the necessary steps to understand how to build a Fuzzy AI system. More information will be sent closer to the event by email.

1/31/2021: First Deliverable Submission Due Date

The teams will need to use the concepts learnt in the first seminar to build their own solution of a simplified real-world problem using Fuzzy AI. The details of the problem will be given during the first seminar. Guidance in the development to those teams that request it will also be granted. This milestone will contribute to final team scores.

2/1/2021: Seminar: Genetic Algorithms and other gradient-free optimization algorithms

In a similar format to the first class, a lecturer will walk you through the mechanics and implementation of a Genetic Algorithm (GA). We will also discuss the use of a GA in the optimization of a FIS. Other gradient-free optimization algorithms will also be discussed. The students will then need to apply these concepts in their second problem. 

2/14/2021: Second Deliverable Submission Due Date

The teams will need to use the concepts learnt in the Optimization seminar to optimize a Fuzzy Inference System. Details on the problem will be given at the end of the seminar. Guidance can be provided as requested by teams and submissions will count towards final evaluation scores.

2/15/2021: Introduction to the Asteroid Smasher game

In this meeting the Asteroid Smasher game will be introduced including constraints and details on controller development. Final details on the competition will also be given at this time including scoring metrics, expectations, and other final deliverables/requirements.

4/4/2021: Final Deliverable – Asteroid Smasher Fuzzy XAI Submission Due Date

This will be the submission date of the final version of your Fuzzy XAI controller for the Asteroid Smasher game. Details on submission format and requirement will be given to you beforehand by the organizers. The final week between the submission date and the competition date will be used to ensure any issues are resolved. Submissions must be received by 11:59 PM EST. 

4/10/2021: Competition Livestream Event

Finally, after all the hard work, the students and advisors will be recognized in this final event that will take place virtually. Teams will have the opportunity to describe what they did and their approach to creating their Fuzzy XAI systems. Then they will see their creations be put to the test in a series of scenarios created by the organizers. The teams will be scored and awards will be given out based on numerous criteria. 

Evaluation Criteria:

  • Fuzzy System deliverable
  • Optimization deliverable
  • Final Asteroid Smasher Fuzzy XAI performance: Agents will be evaluated against a portfolio of scenarios with a custom fitness function
  • Explainability of the created AI
  • Creativity in the solutions

Awards

There are several awards for the winners and for those who show creativity in their submissions! All participants will get a certificate for their participation. Monetary awards will be given to the top 3 teams based on their overall competition score (more details to follow):

  • First place: $1400
  • Second place:$900
  • Third place $400

Additionally, the following awards will be granted ($100 each): 

  • Most Explainable Solution
  • Most Innovative Solution
  • Most Computationally Efficient AI

Judges

Judges will consist of the competition organizers along with others from UC, NAFIPS, and Thales. 

Sponsors

References

Organizers

Timothy headshot

Timothy Arnett
AI Verification Lead, Thales
Email: tim.arnett@psibernetix.com

Dr. Timothy Arnett is the AI Verification Lead at Thales in Cincinnati. His focus is on development of Genetic Fuzzy Tree-based AI methods along with their verification using Formal Methods. His graduate work at the University of Cincinnati was mainly devoted to work on the scalability and verifiability of Fuzzy Systems.

Kelly headshot

Kelly Cohen
Brian H. Rowe endowed Chair and Interim Head, Department of Aerospace Engineering and Engineering Mechanics

Dr. Kelly Cohen is the Brian H. Rowe endowed Chair and Interim Head, Department of Aerospace Engineering and Engineering Mechanics, University of Cincinnati (UC). His main expertise lies in the area of Artificial Intelligence (AI), intelligent systems, UAVs and optimization. He has utilized genetic fuzzy logic-based algorithms for control and decision-making applications in the area of autonomous collaborating robotics as well as predictive modeling for personalizing medical treatment in neurological disorders. During the past seven years, he has secured grants from NSF, NIH, USAF, DHS and NASA to develop algorithms for UAV applications as well as AI for bio-medical applications. He has over 65 per reviewed archival publications, and another 270 conference papers/presentations, and invited seminars.

Nick headshot

Nick Ernest
Chief Architect for Thales Avionics, Inc.

Dr. Nick Ernest is a graduate of the University of Cincinnati’s College of Aerospace Engineering & Engineering Mechanics.  The focus of his research is genetic fuzzy artificial intelligence and he founded Psibernetix Inc. to develop these systems for clients primarily within the defense & aerospace domain.  In 2018 Psibernetix was acquired by Thales, a Paris-headquartered global leader within this area.  Dr. Ernest now serves as Chief Architect for Thales Avionics, Inc., based out of Cincinnati.

Brandon headshot

Brandon Kunkel
Senior AI Engineer, Thales

Brandon is a Senior AI Engineer with a focus on explainable Fuzzy AI  at Thales USA.  He graduated with his BS in Aerospace Engineering from UC in 2017.

Javier headshot

Javier Viana
Ph.D. Candidate, Explainable Artificial Intelligence; Aerospace Engineering

Javier is a Ph.D. candidate in Explainable Artificial Intelligence applied to Aerospace Engineering at the UC. His research is sponsored by the “la Caixa” Fellowship Award. In the past he has interned as designer of algorithms at Genexia, Aurora Flight Sciences (Boeing), Satlantis Microsatellites and the European Space Agency. 

Lynn headshot

Lynn Pickering
Ph.D. candidate, Aerospace Engineering
Email: pickerln@mail.uc.edu 

Lynn is a first year PhD candidate in Aerospace Engineering, with a focus on Genetic Fuzzy Systems under Dr. Kelly Cohen at the University of Cincinnati. She has 3 publications in the field of Artificial intelligence, and is currently funded by the Rindsberg Fellowship. 

Contact Us

Please contact Javier at vianajr@mail.uc.edu for general competition questions.