Transforming Real-Time Forex Data into OHLC Bars with PHP
16 April 2024
Are you ready to enter the exciting world of WebSockets? As this technology continues to permeate the tech scene, many aspiring developers are eager to incorporate it into their work.
Sure, tinkering with a server-client setup and playing around with WebSocket data is fun, but the real excitement comes from harnessing WebSockets to access actionable, real-time data via our Forex API.
In this engaging tutorial, we'll guide you through the process of transforming raw WebSocket data into minute-by-minute High, Low, Open, and Close (HLOC) bars. This transformation will empower developers and analysts alike to uncover valuable insights into market trends over precise one-minute intervals.
Don't worry if you're new to PHP; we've got you covered! While some programming experience could give you a slight edge, it's definitely okay. And hey, if PHP isn’t your cup of tea, don't fret! You can also dive into WebSocket implementations using Python and Golang.
Before we dive into the coding adventure, let's set the stage. Our mission is to fetch real-time Forex data via a WebSocket connection and convert it into minute-based HLOC bars. Our tool of choice is the TraderMade WebSocket API on the server side.
For more details, make sure to check out the documentation section. Let's get started!
Setting Up Your Environment:
Let’s dive into this tutorial. First, ensure you have PHP installed. Open a command window and type php.exe—version. If you see a version number, you're good to go! If not, you'll need to install PHP. Also, grab a free streaming API key for forex data by starting a 14-day WebSocket trial from your user dashboard.
Setting Up PHP
Don't worry; setting up PHP is a breeze! Just follow these 3 simple steps:
1) Download the php.exe build file from the PHP site and unpack it into a new directory.
2) Update your "Environment Variables" or classpath to include the location of php.exe.
3) To verify your setup, open a command window and run the following command:
a> For Windows: php.exe -version
b> For Linux: PHP -version
Running the PHP—-version is like asking your computer to introduce itself. When you run this command in your terminal, it tells you the version of PHP installed on your system. It's a quick way to check which version of PHP you're working with, which can be super useful when troubleshooting or setting up a new project.
php -–version
Setting Up the Project Directory
Let's organize our workspace! Please create a new directory to store our source code and dependencies. I've named mine WebSocket_Client, but feel free to choose any name you like. Next, install the Composer package manager. Once that's done, navigate to your project directory.
Installing Libraries
Run the following command composer requires textalk/websocket in your terminal to install the WebSocket library for your PHP project. This library enables you to work with WebSocket connections in the code.
composer require textalk/websocket
Let us now proceed with the coding
Step 1: Including Required Libraries
Here, we're including the required Composer autoload file. This file ensures that our script loads all the necessary classes and functions from the WebSocket library we are using. This makes it easier to use the WebSocket library without manually including each class or function, streamlining our code and making development more efficient.
require_once("vendor/autoload.php");
Step 2: Connecting to WebSocket Server
The connectWebSocket() function is designed to establish a WebSocket connection to a specific server. This function is crucial for enabling real-time, two-way communication between a client and a server. Here's an expanded description of the function:
1) The connectWebSocket() function takes a serverURL parameter, which is the URL of the WebSocket server to which to connect.
2) Inside the function, a new WebSocket connection is created using the WebSocket API with the provided server URL.
3) Event listeners are attached to the WebSocket object to handle different events:
a) 'open': Triggered when the WebSocket connection is successfully opened.
b) 'message': It is triggered when a message is received from the server.
c) 'error': Triggered when an error occurs with the WebSocket connection.
d) 'close': Triggered when the WebSocket connection is closed.
e) 'timeout': This is a custom event to catch the WebSocketTimeoutException when the connection times out.
4) If the WebSocket connection is closed or times out, the function attempts to reconnect to the server after waiting for 5 seconds using setTimeout().
This function provides a robust way to establish and maintain a WebSocket connection, automatically attempting to reconnect in case of disconnection or timeouts.
function connectWebSocket() {
try {
$client = new WebSocketClient("wss://marketdata.tradermade.com/feedadv");
$message = $client->receive();
echo $message;
$client->text("{"userKey":"api_key", "symbol":"GBPUSD,EURUSD"}");
return $client;
} catch (WebSocketTimeoutException $e) {
echo "WebSocket connection timeout. Reconnecting...
";
sleep(5); // Wait for 5 seconds before reconnecting
return connectWebSocket();
}
}
Step 3: Receiving and Handling WebSocket Messages
1) A global array called dataArray is initialized to store the received data.
2) The connectWebSocket() function remains the same as the previous version, but a new function, saveDataToFile(), is added to save the dataArray to a JSON file.
3) After calling connectWebSocket('wss://marketdata.tradermade.com/feedadv'), the code enters an infinite loop while (true) to receive and process WebSocket messages continuously. Due to the event-driven nature of WebSocket, the actual message handling will happen asynchronously.
4) Inside the message event listener, the received message is checked. If the message is not "connected", it is parsed as JSON to extract the bid and ask prices, calculate the mid-price, and then print and save the received data.
5) The code also includes error handling to catch and display any exceptions and handles WebSocketTimeoutException by reconnecting to the WebSocket server after a 5-second delay.
$client = connectWebSocket();
$dataArray = [];
while (true) {
try {
$message = $client->receive();
if (strcmp($message, "connected") !== 0) {
$decoded_json = json_decode($message, true);
$bid = isset($decoded_json['bid']) ? $decoded_json['bid'] : null;
$ask = isset($decoded_json['ask']) ? $decoded_json['ask'] : null;
if ($bid !== null && $ask !== null) {
$mid = ($bid + $ask) / 2;
echo $decoded_json['symbol'], " ", $decoded_json['ts'], " ", $bid, " ", $ask, " ", $mid, "
";
$dataArray[] = [
'symbol' => $decoded_json['symbol'],
'ts' => $decoded_json['ts'],
'bid' => $bid,
'ask' => $ask,
'mid' => $mid
];
// Write data to JSON file
$jsonString = json_encode($dataArray, JSON_PRETTY_PRINT);
file_put_contents('websocket_data.json', $jsonString);
}
}
} catch (WebSocketTimeoutException $e) {
echo "WebSocket read timeout. Reconnecting...
";
sleep(5); // Wait for 5 seconds before reconnecting
$client = connectWebSocket();
} catch (Exception $e) {
echo "Error: " . $e->getMessage();
}
}
Step 4: Reading Data from JSON File
1) The readDataFromFile() function reads the content of the specified JSON file and decodes it into an object.
2) The checkDataNotEmpty() function checks if the provided data object is empty. If it is, an error message is displayed, and the script exits.
3) After reading the content of websocket_data.json into the forexData object, the script checks if forexData is empty using checkDataNotEmpty(forexData). If forexData is empty, an error message is displayed, and the script exits.
These functions can be integrated into your existing script to handle reading and checking the websocket_data.json file.
$jsonFilePath = 'websocket_data.json';
$jsonData = file_get_contents($jsonFilePath);
$forexData = json_decode($jsonData, true);
if (empty($forexData)) {
echo json_encode(["error" => "No data found in the input JSON file: $jsonFilePath"]);
exit;
}
Step 5: Aggregating OHLC Data
1) The moment-timezone library is used to handle and adjust the timestamps based on the local time zone.
2) The roundTimestamp() function rounds the UNIX timestamp to the nearest interval (1 minute) and adjusts it for the local time zone.
3) An empty object ohlcData, is initialized to store the OHLC (Open, High, Low, Close) data.
4) The code iterates through each data point in forexData, converts the timestamp to a UNIX timestamp, rounds the timestamp using roundTimestamp(), and then creates or updates the OHLC data structure in ohlcData based on the rounded timestamp and bid price.
5) For each rounded timestamp:
a) If the rounded timestamp does not exist in ohlcData, the OHLC data structure is initialized with the open, high, low, and close prices set to the bid price of the data point.
b)If the rounded timestamp already exists in ohlcData, the high and low prices are updated if necessary, and the close price is set to the bid price of the data point.
Finally, the ohlcData object containing the aggregated OHLC data is logged to the console.
$interval = 60; // 1 minute interval
$ohlcData = [];
foreach ($forexData as $data) {
$timestamp = strtotime($data['ts']);
$roundedTimestamp = floor(($timestamp + (5 * 3600) + (30 * 60)) / $interval) * $interval;
if (!isset($ohlcData[$roundedTimestamp])) {
$ohlcData[$roundedTimestamp] = [
'timestamp' => $roundedTimestamp + (5 * 3600) + (30 * 60), // Corrected timestamp
'low' => $data['bid'],
'high' => $data['bid'],
'open' => $data['bid'],
'close' => $data['bid']
];
} else {
$ohlcData[$roundedTimestamp]['low'] = min($ohlcData[$roundedTimestamp]['low'], $data['bid']);
$ohlcData[$roundedTimestamp]['high'] = max($ohlcData[$roundedTimestamp]['high'], $data['bid']);
$ohlcData[$roundedTimestamp]['close'] = $data['bid'];
}
}
Step 6: Saving OHLC Data to JSON File
1) The writeDataToFile() function writes the aggregated OHLC data (ohlcData) to the specified JSON file (ohlc_data.json).
2) The fs.writeFileSync() method is used to write the data to the file synchronously.
3) If the "write operation" is successful, it displays a success message, and the script continues to run.
4) If the write operation fails, an error message gets displayed, and the script exits with an error status using process. exit(1).
You can integrate this writeDataToFile() function into your existing script to handle writing the aggregated OHLC data to the ohlc_data.json file.
$jsonFilePathOutput = 'ohlc_data.json';
if (!file_put_contents($jsonFilePathOutput, json_encode(array_values($ohlcData), JSON_PRETTY_PRINT))) {
echo json_encode(["error" => "Failed to write OHLC data to JSON file: $jsonFilePathOutput"]);
exit;
}
echo json_encode(["success" => "OHLC data saved to JSON file: $jsonFilePathOutput"]);
Running Our Program and Checking the Output
The command php webSocket_client.php runs the webSocket_client.php PHP script. This script reads websocket_data.json, aggregates OHLC data with a 60-second interval, and saves the result to ohlc_data.json.
php webSocket_client.php
And there you have it! You're now streaming live forex rates. Enjoy!
[
{
"symbol": "GBPUSD",
"ts": "1712130980714",
"bid": 1.25664,
"ask": 1.25668
},
{
"symbol": "GBPUSD",
"ts": "1712130980909",
"bid": 1.25664,
"ask": 1.25669
},
{
"symbol": "GBPUSD",
"ts": "1712130980911",
"bid": 1.25665,
"ask": 1.25669
},
{
"symbol": "EURUSD",
"ts": "1712130982178",
"bid": 1.07712,
"ask": 1.07712
},
{
"symbol": "GBPUSD",
"ts": "1712130982180",
"bid": 1.25665,
"ask": 1.25667
}
]
OHLC output:
[
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25664,
"low": 1.25664,
"close": 1.25664
},
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25664,
"low": 1.25664,
"close": 1.25664
},
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25665,
"low": 1.25664,
"close": 1.25665
},
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25665,
"low": 1.07712,
"close": 1.07712
},
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25665,
"low": 1.07712,
"close": 1.25665
},
{
"ts": "1970-01-01 05:30:00",
"open": 1.25664,
"high": 1.25665,
"low": 1.07711,
"close": 1.07711
}
]
In the folder where we save our PHP script, we also get realtime_data.json with output from websockets and output_data.json with OHLC data. Feel free to change the file names in the code accordingly.
Full code:
<?php
require_once("vendor/autoload.php");
// Function to connect to WebSocket server
function connectWebSocket() {
try {
$client = new WebSocketClient("wss://marketdata.tradermade.com/feedadv");
$message = $client->receive();
echo $message;
$client->text("{"userKey":"API_KEY", "symbol":"GBPUSD,EURUSD"}");
return $client;
} catch (WebSocketTimeoutException $e) {
echo "WebSocket connection timeout. Reconnecting...
";
sleep(5); // Wait for 5 seconds before reconnecting
return connectWebSocket();
}
}
$client = connectWebSocket();
$dataArray = [];
while (true) {
try {
$message = $client->receive();
if (strcmp($message, "connected") !== 0) {
$decoded_json = json_decode($message, true);
$bid = isset($decoded_json['bid']) ? $decoded_json['bid'] : null;
$ask = isset($decoded_json['ask']) ? $decoded_json['ask'] : null;
if ($bid !== null && $ask !== null) {
$mid = ($bid + $ask) / 2;
echo $decoded_json['symbol'], " ", $decoded_json['ts'], " ", $bid, " ", $ask, " ", $mid, "
";
$dataArray[] = [
'symbol' => $decoded_json['symbol'],
'ts' => $decoded_json['ts'],
'bid' => $bid,
'ask' => $ask,
'mid' => $mid
];
// Write data to JSON file
$jsonString = json_encode($dataArray, JSON_PRETTY_PRINT);
file_put_contents('websocket_data.json', $jsonString);
}
}
} catch (WebSocketTimeoutException $e) {
echo "WebSocket read timeout. Reconnecting...
";
sleep(5); // Wait for 5 seconds before reconnecting
$client = connectWebSocket();
} catch (Exception $e) {
echo "Error: " . $e->getMessage();
}
}
// Read input data from JSON file
$jsonFilePath = 'websocket_data.json';
$jsonData = file_get_contents($jsonFilePath);
$forexData = json_decode($jsonData, true);
if (empty($forexData)) {
echo json_encode(["error" => "No data found in the input JSON file: $jsonFilePath"]);
exit;
}
// Define the time interval for OHLC aggregation (in seconds)
$interval = 60; // 1 minute interval
// Prepare OHLC data
$ohlcData = [];
foreach ($forexData as $data) {
$timestamp = strtotime($data['ts']);
$roundedTimestamp = floor(($timestamp + (5 * 3600) + (30 * 60)) / $interval) * $interval;
if (!isset($ohlcData[$roundedTimestamp])) {
$ohlcData[$roundedTimestamp] = [
'timestamp' => $roundedTimestamp + (5 * 3600) + (30 * 60), // Corrected timestamp
'low' => $data['bid'],
'high' => $data['bid'],
'open' => $data['bid'],
'close' => $data['bid']
];
} else {
$ohlcData[$roundedTimestamp]['low'] = min($ohlcData[$roundedTimestamp]['low'], $data['bid']);
$ohlcData[$roundedTimestamp]['high'] = max($ohlcData[$roundedTimestamp]['high'], $data['bid']);
$ohlcData[$roundedTimestamp]['close'] = $data['bid'];
}
}
// Define JSON file path
$jsonFilePathOutput = 'ohlc_data.json';
// Save JSON data to file
if (!file_put_contents($jsonFilePathOutput, json_encode(array_values($ohlcData)))) {
echo json_encode(["error" => "Failed to write OHLC data to JSON file: $jsonFilePathOutput"]);
exit;
}
echo json_encode(["success" => "OHLC data saved to JSON file: $jsonFilePathOutput"]);
?>
Conclusion
Dive into the world of WebSockets to unlock real-time forex data with ease! This tutorial guides you through transforming raw WebSocket data into actionable market insights, even if you're new to PHP.
