Project Roadmap¶
We've recently changed the direction of Velora, moving from a global RL framework that uses LNNs with all existing RL algorithms, to one that focuses on a custom architecture built up of unique RL techniques.
Our goal is to create a completely autonomous system that learns from it's environment without human intervention. We want to help advance the field forward into the Era of Experience [], specifically, in the cyber domain.
Cyber threats are never ending and always changing so it's easy to get overwhelmed by them. Autonomy is needed. Thus, we are working on a custom solution - NeuroFlow
.
You'll find it's details in the roadmap below along with additions we are planning. We've broken them down into different sections with checkboxes. Items that are green are already implemented.
We've got a lot planned for and are excited for the future of Autonomous Cyber Defense.
Road to 1.0 Release¶
So far, we've got a solid foundation for Velora
and our NeuroFlow
agents but there is still much to do!
Here's our plans and progress so far:
-
Main Components/Modules
- Liquid Neural Networks (CfC) - 2022
- SAC: Continuous - 2018
- SAC: Discrete - 2019
- SAC: Automatic Entropy - 2018
- Replay Buffer
- Small Actor, Large Critics: Honey, I Shrunk the Actor - 2021
- CAT-SAC: SAC with Curiosity-Aware Entropy Temperature - 2020
- PlaNet: Learning Latent Dynamics for Planning from Pixels - 2018
- EWC: Overcoming Catastrophic Forgetting in Neural Networks - 2016
-
Custom Components/Modules
- Liquid NCP Actor, NCP Critics
- Strategy Library
- Adaptive Network using fitness score
-
Utility
- Setting seed and device
- Gymnasium environment search and wrappers
- Saving and Loading models & buffers
- Training and predicting with models
- Agent performance tracking (offline & online)
- Early stopping and checkpoint save system
- Recording episode performance
-
Simulation Environments