Composabl

Appearance

Composabl Historian ​

The Historian is a key component of the Composabl SDK. A Historian is a system that gathers all the data generated by your agents and simulators. This data is then stored in a database system, allowing you to analyze and visualize the data. The Historian is a crucial part of the Composabl SDK, as it allows you to solve a variety of use cases and improve them over time. Some example use cases are:

  • Monitoring the performance of your agents and simulators in real-time
  • Post-mortem analysis of your agents and simulators to understand what went wrong
  • Creating dashboards and reports to share with your team or customers
  • ...

We automatically ship the Historian for you that you are able to start with composabl historian start, which will spin up the system and start collecting data once a simulator and training process is attached. Implementation wise, we work on the foundations of some of the most popular big-data stores and utilize the "Delta Lake" system, bringing a lot of advantages:

  • ACID Transactions: Ensuring data integrity
  • Schema Enforcement: Ensuring data quality
  • Time Travel (data versioning): Providing data versioning and audit history
  • Flat File Storage: Allowing us to store data in a flat file system per training session, making it easier to maintain and manage
  • Cross Language Support: Allowing us to use Delta Lake in multiple languages

Getting Started ​

You can start the Historian system by running the following command:

bash
# Base
composabl historian start

# Specify the output path
composabl historian start --output-path /tmp/composabl-historian

When you now train an agent, a flat file will be created in the output path specified. This file contains all the events that happened during the training session. You can view the file location using the status command:

bash
composabl historian status

which will output a table of the Historian sub-systems and their statuses.

bash
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Service   ┃ Container Name ┃ Status  ┃ Connection Details                            ┃ Ports                                                                    ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ EMQX      │ emqx           │ running │ tcp://admin:PASS!@localhost:1883              │ 18083/tcp,1883/tcp,4370/tcp,5369/tcp,8080/tcp,8083/tcp,8084/tcp,8883/tcp │
│ Historian │ historian      │ running │ file:///tmp/composabl-historian/RUN_ID.delta  │                                                                          │
└───────────┴────────────────┴─────────┴───────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────┘