Autopilot Strategy Creation & Evolution with AI Agents and Deep RL

Category: Tutorials & Guides
Target Readers: Quantitative Researchers, Algo Traders, Developers

Introduction

One of the most powerful features of the RLXBT ecosystem is the seamless integration of Deep Reinforcement Learning (Deep RL) and AI Agents via the Model Context Protocol (MCP). Instead of hardcoding static trading rules, quantitative researchers can deploy deep Q-learning networks (DQN) that dynamically learn profitable trading policies directly from market indicators.

This guide details how the RL engine operates under the hood, how the LLM agent interacts with it, and how you can train, evaluate, and query models for real-time predictions.

1. The Architecture: Two-Level Optimization

The system operates on two distinct optimization layers:

+-------------------------------------------------------+
|                 1. Strategic Layer (LLM Agent)        |
|  - Defines State space (features) & reward formulas   |
|  - Triggers experiments via MCP tools                 |
|  - Evolves hypotheses on the Canvas                   |
+-------------------------------------------------------+
                           |
                           v  (MCP JSON-RPC)
+-------------------------------------------------------+
|                 2. Execution Layer (Rust Core)        |
|  - Compiles RLEnvironment and calculates features     |
|  - Trains Deep Q-Network (LibTorch) on past data      |
|  - Runs millisecond-level inference (rl_predict)       |
+-------------------------------------------------------+

1. Strategic Layer (AI Agent): The LLM structures the research, modifies the features, sets boundaries, and reviews evaluation metrics (Sharpe, WFE, Monte Carlo).
2. Execution Layer (Rust Daemon): A lightning-fast engine written in optimized Rust that trains neural networks and runs real-time inference in milliseconds.

2. Core MCP Tools for Reinforcement Learning

The rlxbt MCP server exposes three main endpoints to manage the Deep RL lifecycle:

A. Training: rl_train

Launches a full reinforcement learning training cycle. The environment compiles custom indicator matrices, normalizes the state vectors, and trains the DQN weights using PyTorch.

• Key Arguments:
  • episodes: Number of training passes (e.g., 10 to 50).
  • window_size: Lookback window (e.g., 20 bars).
  • reward_type: Optimization target (portfolio, log, sharpe, calmar, or risk).
  • train_split: Training/validation boundary (e.g., 0.8).
• Return: Report ID containing the serialized neural weights.

B. Evaluation: rl_evaluate

Runs the trained model over an entire unseen dataset (Out-of-Sample) to calculate standard trading metrics without training noise.

• Key Arguments:
  • id: The report ID of the trained model.
  • features_path: Path to the evaluation dataset.
• Return: Sharpe ratio, total return, max drawdown, win rate, action distribution (flat/long/short), and equity curve.

C. Live Inference: rl_predict

Interrogates the model for an immediate signal by passing raw OHLCV bars. The Rust environment dynamically compiles indicators on the fly.

• Key Arguments:
  • id: Report ID.
  • bars: An array of recent bars (at least window_size + 1 length).
• Return: Recommended action (long, short, flat), direction (1, -1, 0), confidence percentage, and Q-values.

3. Example: Fetching Live Signals in Python

Here is how to query your trained model for real-time predictions using the running daemon bridge:

import requests
import json

# Setup the bridge url and report ID
BRIDGE_URL = "http://127.0.0.1:8145"
REPORT_ID = "rpt_1783598697206_18"  # Replace with your model's ID

# Recent OHLCV bars (must be >= window_size + 1)
recent_bars = [
    {"timestamp": 1783519200, "open": 61945.49, "high": 62173.56, "low": 61712.0, "close": 61889.83, "volume": 1415.74},
    # ... Add at least 20 more historical bars ...
]

# JSON-RPC request format
payload = {
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
        "name": "rl_predict",
        "arguments": {
            "id": REPORT_ID,
            "bars": recent_bars
        }
    }
}

response = requests.post(BRIDGE_URL, json=payload)
result = json.loads(response.json()["result"]["content"][0]["text"])

if result["status"] == "success":
    signal = result["data"]
    print(f"Action: {signal['action'].upper()} (Confidence: {signal['confidence']*100:.2f}%)")
else:
    print("Error:", result["message"])

4. Evolving Strategies on the Canvas

1. Define a Hypothesis: Set up your indicators (e.g., drift_z_24 and absorption_24) as observations on the Idea Map.
2. Train the RL Agent: Call rl_train to let the DQN discover trading patterns on those features.
3. Validate: Call rl_evaluate on a separate out-of-sample fold to calculate the Walk-Forward Efficiency.
4. Deploy: Feed real-time exchange WebSocket feeds directly into rl_predict to execute live actions.