What is Firekeep?

Firekeep is an autonomous network of drones that can detect wildfires, predict their spread, plan out a response, and deploy resources to mitigate the fire. Firekeep is simulated using Minecraft as a sandbox environment to demonstrate the capabilities of the autonomous fleet. Firekeep drones are able to detect and call for help when they detect a fire. With spread prediction, the drones are also able to preemptively plan out a response to the spread based on wind patterns and deploy resources to mitigate it. Furthermore, the fleet of drones are able to coordinate with each other to get the best coverage of the fire and improve responses.

Our fire spread prediction model in action

How does it work?

Firekeep is built using a combination of a Fabric mod for Minecraft, a Python server backend, n8n, world labs, and a Next.js dashboard. Our Python server backend is the core of the projectt, it is responsible for connecting the drones and the n8n workflows, as well as serving server data to the dashboard. n8n workflows are used to run various systems such as patrol regions, fire detection, spread prediction, resource allocation, and incident report generation. Incident reports are generated after an incident occurs and is used to document the spread of the fire and the resources used to mitigate it in order to help first responders. Furthermore, we used World Labs to convert drone screenshots into realistic scenes in the incident report to demonstrate the capabilities of the fleet in the real world.

Our disaster simulation tool in action

What are the components of Firekeep?

Here are the main components of Firekeep: 1. The Fabric mod is used to create the firekeep drones in-game and expose a REST API to control and read from the drones.
2. The Python server backend is used to connect the drones and the n8n workflows, as well as serving server data to the dashboard.
3. n8n workflows are used to run various AI services to patrol regions, detect fires, predict spread, allocate resources, and generate realistic scenes from Minecraft images.
4. World Labs is used to convert Minecraft images into realistic scenes in the Incident Report generated by the fleet.

Challenges we faced

1. One of the biggest challenges we faced was the live feed from the drones. We wanted our Minecraft server running the Fabric mod to render and stream the POVs of each drone through the Python server backend to the dashboard. However, we soon realised that the Minecraft server JAR does not contain the necessary graphics libraries to actually render Minecraft. In order to combat this, we decided to have each drone launch an independent client instance of Minecraft and join the server, then, the mod would use the clients as a proxy to render the world. This allowed us to stream the live feed from the drones to the dashboard in real-time, thought, it did come with significant performance overhead as it required individual Minecraft instances to be running for each drone.
2. Another challenge we faced was reliably getting the n8n workflows to control the drones in an intentional way. We wanted the workflows to be able to intellibly patrol regions of the map in a manner that optimized map coverage. Our next hurdle was ensuring that the drones pathfinding algorithm was capable of getting past obstacles in Minecraft. We were able to solve this issue by passing coordinate data to the n8n workflows as well as simulated sensor data based on the drone's position and orientation.

Our dashboard displaying a live feed from a drone

Features

• Fleet orchestration and coordination
• Real-time drone feed
• Fire detection and spread prediction
• Resource allocation and deployment
• Incident report generation
• Realistic scene generation from Minecraft images

Our incident report generated by the fleet with screenshots attached

My Contribution

I was primarily responsible for setting up the Fabric mod and the Python server backend, as well as integrating the Python server with the n8n workflows and the dashboard. I also contributed heavily to the pathfinding, fire detection, and spread prediction systems. Lastly, I was responsible for setting up the Minecraft testing environment and running the server and drone clients.

Our dashboard displaying an interactive map of the world

Achievement

Firekeep won 3rd place overall out of 87 projects at Ignition Hacks v.7 and was also awarded the Best Use of n8n award.

Our n8n workflow setup for realistic scene generation