Composabl

Appearance

Defining Skills ​

You can set up your skills to execute in one of two ways: learned or programmed.

Learned skills are taught and controller skills are programmed. Learned skills are taught through deep reinforcement learning (DRL), and programmed skills are controller actions defined through calculations, rules, or algorithms.

Teaching Learned Skills ​

Learned skills will be trained using Deep Reinforcement Learning (DRL). They learn good actions to take by practicing reading the state of the system (as reported by sensors), taking action, getting feedback (in the form of a reward), and pursuing maximal future reward.

Set the trainable flag to True.

Learned skills input a teaching class that you will use to guide the learning with various functions that you write.

python
increment_skill = Skill("increment", IncrementTeacher)

decrement_skill = Skill("decremement", DecrementTeacher)

target_skill = Skill("target", TargetTeacher)

Teaching skills will be covered in more depth in a later section of this documentation.

Programming Controller Skills ​

You can program a skill by creating a Python function that inputs the sensor feedback (observation space) of the skill and outputs the decisions (actions) of the skill. For example, you could use heuristics, equations, optimization algorithms, or mathematical control to execute a programmed skill.

python
Controller(), trainable=False

When you set the trainable parameter to False, you tell the system that you will specify the output of the skill with a function that you create. The second parameter of the method call to create the skill, executes that skill, if the skill is programmed (trainable= False)

Adding Scenarios to your Skills ​

For each skill in your agent, you must specify scenarios that the skill will navigate and be trained on. Add scenarios to each skill with the add_scenario() method call. Each skill needs at least one scenario. Beyond that, you can add as few or as many scenarios as are needed to train each skill. So, some skills might be trained on all scenarios, because they will need to be performed in all situations, under all possible conditions. Others might not be needed in every situation, so they might be trained only on some scenarios. Here's an example of how to add scenarios to skills:

python
for scenario_dict in increment_scenarios:
    scenario = Scenario(scenario_dict)
    increment_skill.add_scenario(scenario)

Adding Skills to your Agent ​

Now you can add the skills to your agent with the add_skill() method. Here's the code to add the three action skills to the agent:

python
agent.add_skill(increment_skill)
agent.add_skill(decrement_skill)
agent.add_skill(target_skill)

Here is the complete agent.py agent file after we add the skills.

python
from composabl import Agent, Skill, Sensor, Scenario, Trainer

from .teaching import IncrementTeacher, DecrementTeacher, TargetTeacher
from .scenarios import increment_scenarios, decrement_scenarios, target_scenarios
from .sensors import sensors
from .perceptors import perceptors

increment_skill = Skill("increment", IncrementTeacher)
for scenario_dict in increment_scenarios:
    scenario = Scenario(scenario_dict)
    increment_skill.add_scenario(scenario)

decrement_skill = Skill("decremement", DecrementTeacher)
for scenario_dict in decrement_scenarios:
    scenario = Scenario(scenario_dict)
    decrement_skill.add_scenario(scenario)

target_skill = Skill("target", TargetTeacher)
for scenario_dict in target_scenarios:
    scenario = Scenario(scenario_dict)
    target_skill.add_scenario(scenario)

config = {
    "target": {
        "docker": {
            "image": "composabl/sim-demo:latest"
        }
    },
    "env": {
        "name": "sim-demo",
    },
    "license": "<This is your license key>",
    "training": {},
}

trainer = Trainer(config)
agent = Agent()
agent.add_sensors(sensors)
agent.add_perceptors(perceptors)

agent.add_skill(increment_skill)
agent.add_skill(decrement_skill)