A newcoming user to the PLAT environment needs to learn how to set up a client in order to implement his/her automated trading strategy. In this tutorial, we are going to describe the steps needed to set up a linux or solaris machine as an environment to develop and execute such clients.
A client written to trade with the Penn Exchange Simulator (PXS) is constituted of three layers. At the topmost level is the user's strategy, which is the high-level source code that performs various trading actions by making calls to the underlying client-shell API. At the middle level is the client-shell API, which performs common actions needed by an automated trader, such as placing buy/sell orders or informing the automated trader about the market status. At the lowest level is the network layer used by the client-shell API, which enables the client to communicate with the PXS server to send and/or receive various information.
To install the plat-client package for strategy development on crux, go to a directory of your choice in your crux account, and execute the command:
/home/pxs/bin/plat-client-installerThis will automatically generate a directory named plat-client, in the directory where you execute the command, which includes everything you need to start development. Specifically, inside the plat-client directory, the installer will create a subdirectory named strat which includes an initial strategy source code and Makefiles, and compile that initial strategy. You should do all your strategy development in the plat-client/strat directory. See below for more details on how to develop and compile your own strategies. You can also develop and run your clients externally (i.e., from your own machine). Click here for more details.
cd CLIENT_DIR/plat-client/stratand then run the following two commands, after replacing XXXX below by one of your assigned port numbers (and use the same number in both commands!).
nohup /home/pxs/bin/pxs -p XXXX -n MSFT -h 09122003093000 -e 120000 >& out.pxs &In the above execution, the server (pxs) output is written to the file CLIENT_DIR/plat-client/strat/out.pxs while the client's output is written to CLIENT_DIR/plat-client/strat/out.mytrader. The "nohup" part is standard Unix; it runs the subsequent command in a way that should still continue even if you log out. The invocation of pxs is described in much greater detail below, but basically the pair of commands above will run the mytrader client on historical Microsoft (MSFT) data from September 12 beginning at 9:30 (that's the -h argument) ending at noon (that's the -e argument).
nohup ./mytrader -p XXXX >& out.mytrader &
The above simulation should take less than 15 minutes to complete. (You will need to know how to track and manage your own background jobs.) You can read example descriptions of a server's output file here and of a client's output file here. Note however that we use verbose mode in the cited examples! You can learn more about the different execution modes here.
The strategy.c file in the directory CLIENT_DIR/plat-client/strat contains instructions to help new users how to implement their new trading strategies. Furthermore, the file in its original form is actually an implementation of a simple automated trading strategy called "Static Order Book Imbalance" (SOBI). For developing your own strategies, you should look at the client API.
To illustrate how you can compile your strategy after implementing it, let's assume for a moment that the strategy.c file is your custom-built automated strategy implementation. Initially, you will find an executable named mytrader corresponding to that strategy in the same directory. If you make changes to the source files (i.e., strategy.c and strategy.h), as you would need to in order to develop your own strategies, you need to recompile the code. To do that, you have to execute the following command in the CLIENT_DIR/plat-client/strat directory:
make
If you add additional source code files to your strategy aside from strategy.c and strategy.h in that directory, then you also need to modify the file Makefile there. Brief instructions on how to do this are included as comments within the file.
After compilation, the executable client mytrader will be created in the CLIENT_DIR/plat-client/strat directory. You may rename the file if you wish. Also, it is possible to change the name of the executable generated when one runs 'make' by directly editing the file Makefile in that directory.
Once you finish implementing a custom-made automated trading strategy, you can execute it by connecting to the PXS. In order to do so, you need to have an account on crux and be able to initiate PXS. PXS is located in /home/pxs/bin, and the command-line for executing the PXS is as follows:
pxs -p port_number -n stock_name [-h begin_time] [-e end_time] [-s number_of_clients] [-V] [-B] [-S] [-T print_frequency]"[ ]" indicates that the argument is optional. Also note that the command-line options should be in order. Here is a brief explanation for the arguments:
You should direct the output of the simulator to a file in your home directory. A single execution of the simulator (pxs) can produce very large output files, especially if you run it in verbose mode, so try to only use this mode when it is really necessary. In general, the most useful information about your strategy will be in the client output, which we describe below, anyway. Hence, after you find no further use for the pxs output files of your simulations, you should remove/erase them from the system. This will help us keep the storage loads on crux down. If you know you are going to ignore the pxs output, we recommend you redirect the output to /dev/null, which will essentially throw away any output from pxs. If it is likely you will not be looking carefully at the pxs output file but are not sure, we recommend you redirect the output to a file in /tmp; that way, in case you will not be using the output file, and forget to erase it, the system will remove/erase it automatically for you after some time.
So PXS should be executed as follows (you should feel free to modify the exact location of output file):
/home/pxs/bin/pxs -p port_number -n stock_name [-h begin_time] [-e end_time] [-s number_of_clients] [-V] [-B] [-S] [-T print_frequency] >& /home/user_name/pxs.output &(It is possible to just type pxs in place of /home/pxs/bin/pxs above if you append the directory /home/pxs/bin to your PATH environment variable.)
After PSX is invoked, you can execute your client to connect to the PXS. As
briefly mentioned above, resulting from the initial installation of the
plat-client package, the executable for the SOBI strategy is is located in
CLIENT_DIR/plat-client/strat/mytrader
For
the purposes of this tutorial, we'll assume this is the automated trading
strategy you'll execute. Instructions for running the SOBI client are in the
following web site
http://www.cis.upenn.edu/~mkearns/projects/sobi.html
You should adapt the instructions to those for executing your particular
strategies accordingly. In general, note that you must execute the client to use
the same port number as that used when you invoked PXS.
When executed, PXS outputs various information during each iteration of its
main algorithm. We will explain parts of one iteration of the sample output file
located in
/home/pxs/pub/tutorial-examples/out.pxs
PXS was executed with the following command-line arguments:
/home/pxs/bin/pxs -p 12324 -n MSFT -h 05092003093000 -e 100000 -V -S >& /home/pxs/pub/tutorial-examples/out.pxs &The explanation of the output is as follows:
Line
1:
Requested end time
10:00:00.0000
Explanation: The Simulator time at which PXS will
terminate.
Lines
2-9:
TODAY'S
ACTIVITY
---------------------------------------------------
Island last price
: [25.9010]
Island inside
market : [25.9000,25.9180]
Island orders
: [2843]
Island volume
: [83396]
Available for virtual
trade : [83396]
===================================================
Explanation: Island activity at the given Simulator
time. Simulator time for this iteration is provided later on in the
output. Island orders indicates the number of book orders on Island
queues. Island volume indicates the total volume of orders on
Island queues. Available for virtual trade is the value the
Simulator uses to decide how many orders can be matched by last price (Think of
this as the "liquidity" provided by Island since the simulator last updated its
Island information.).
Lines
11-30:
ISLAND
UPDATE BUY QUEUE
--------------------------------------------------------------------------------
AGENT REF_NO
SHARE PRICE
===================================================
[-1]
219878 4900 25.9000
[-1]
220135 500 25.9000
[-1]
221054 20 25.9000
[-1]
220566 100 25.8900
[-1]
205235 2400 25.8800
[-1]
440815 2000 25.8700
[-1]
115335 5000 25.8700
[-1]
214304 300 25.8700
[-1]
220423 2000 25.8680
[-1]
214681 4000 25.8600
[-1]
203543 20 25.8500
[-1]
217501 1000 25.8500
[-1]
217896 6000 25.8200
[-1]
158001 100 25.8000
[-1]
214025 1000 25.8000
===================================================
Explanation: The Island buy queue at the given
Simulator time. This is the the last Island order books published
on the web. The agent number "-1" indicates that the origin of the
order is Island itself. The reference number is a unique identifier
for a given order. The share and price tags have their obvious
meaning.
Lines
54-60:
TODAY'S
SIMULATOR ACTIVITY
-------------------------------------------------------------------
Simulator time
: [09:30:00.8164]
Simulator time (in
secs) : [34200.8164]
Simulator Last
price : [0.0000]
Simulator Inside
Market : [25.9000,25.9180]
===================================================
Explanation: The current Simulator time.
Simulator Last price and Simulator Inside Market have their usual
meanings, but in this case they are calculated using the information on matches
done exclusively by the Simulator (This takes into account both matches with
Island orders, including matching by Island last price due to Island's available
liquidity or volume, and virtual order matches - those that are originated by
automated traders connected to PXS.
Lines
62-81:
SIMULATOR TOP 15 BUY
ORDERS
---------------------------------------------------------------
AGENT REF_NO
SHARE PRICE
===================================================
[-1]
219878 4900 25.9000
[-1]
220136 500 25.9000
[-1]
221054 20 25.9000
[-1]
220566 100 25.8900
[-1]
205235 2400 25.8800
[-1]
440815 2000 25.8700
[-1]
115335 5000 25.8700
[-1]
214304 300 25.8700
[-1]
220423 2000 25.8680
[-1]
214681 4000 25.8600
[-1]
203543 20 25.8500
[-1]
217501 1000 25.8500
[-1]
217896 6000 25.8200
[-1]
158001 100 25.8000
[-1]
214025 1000 25.8000
===================================================
Explanation: The top 15 orders in the current order
books of the Simulator . (Same as ISLAND UPDATE BUY QUEUE, but it takes into
account both Island orders and virtual orders - those that are originated by
automated traders connected to PXS.)
Lines
104-110:
---------------------------------------------------
*******
Matched Volume of ALL AGENTS *******
---------------------------------------------------
TOTAL :
0
BUY
: 0
SELL
: 0
---------------------------------------------------
Explanation: The volume of virtual orders that have
been matched.
Lines
121-146:
SIMULATOR
MATCH STATISTICS
---------------------------------------------------------------------
Per Order
Counters
---------------------------------------------------------------------
Number of REAL BUY
orders that are matched : [0]
Number of REAL SELL
orders that are matched : [0]
Number of VIRTUAL BUY
orders that are matched : [0]
Number of VIRTUAL SELL
orders that are matched : [0]
----------------------------------------------------------------------
Per Match
Counters
----------------------------------------------------------------------
Number of REAL BUY
orders that are matched : [0]
Number of REAL SELL
orders that are matched : [0]
Number of VIRTUAL BUY
orders that are matched : [0]
Number of VIRTUAL SELL
orders that are matched : [0]
----------------------------------------------------------------------
Number of complete REAL
BUY order matches : [0]
Number of complete REAL
SELL order matches : [0]
Number of complete
VIRTUAL BUY order matches : [0]
Number of complete
VIRTUAL SELL order matches : [0]
----------------------------------------------------------------------
Number of partial REAL
BUY order matches : [0]
Number of partial REAL
SELL order matches : [0]
Number of partial
VIRTUAL BUY order matches : [0]
Number of partial
VIRTUAL SELL order matches : [0]
===================================================
Explanation: Per match counters are incremented
whenever a match occurs in the Simulator. There are three sets of
per match counters. Every time there is a match, at least one
counter in the first set is incremented. Depending on whether the
order is completely matched or partially matched, counters from the second and
third sets are also incremented.
The rest of the
file is repeated output of the same kind of information at different Simulator
times.
To some extent, you can modify the output of your own strategy. Instructions for doing so are inside the strategy.c file. As a general rule, you should use output strings that do not conflict with the keywords used by the (standard) output of the client-shell process, since this can cause problems to the available plotting scripts (i.e., use "MyCash" instead of "Cash", etc...).
As an example, we will explain the output of the SOBI client invoked with the following command-line parameters:
/home/pxs/bin/sobi -p 12324 -n -V >& /home/pxs/PAT/tutorial-examples/out.sobi &When invoked with the "-V" flag, the client produces verbose output. It is important that the "-V" flag is last in the client invocation. We omitted the explanation for the parts of the output which are similar to PXS output. Explanations for those parts can be found in the preceding section. The explanation of the client-shell (standard) output follows.
Lines
10-12:
IP is
158.130.8.250
Port is
12324
The agent number is
1
Explanation: IP and TCP port information of the PXS the
client connects to. The agent number is used to indicate the origin
of an order. For example, orders with agent number "-1" originate
from Island.
Lines
14-20:
*******
Agent1's Fund Investment Result *******
---------------------------------------------------
Cash
: 0.0000
Share :
0
Simulator
present value : 0.0000
Island
present value : 0.0000
Explanation: The cash spent, the shares held, and the
present value of the agent's investment. The present value is
calculated by the formula: Share*Last_price - Cash. Last price is
the matching price of the last trade executed, and there are two different
present values depending on whether the Simulator last price or Island last
price is used in the calculation.
Lines
470-480: (This example is taken
from an iteration that corresponds to a Simulator Time that is different from
the rest of the examples)
===================================================
MATCHED
ORDER QUEUE
---------------------------------------------------------------------------------
AGENT
TYPE REF_NO SHARE
PRICE
===================================================
[ 1]
SELL -100001 333
26.1200
[ 1]
SELL -100000 333
26.1200
[ 1]
SELL -100003 333
26.1100
[ 1]
SELL -100002 233
26.1100
[ 1]
SELL -100002 100
26.1070
===================================================
Explanation: Information about the executed virtual
orders that belong to this particular agent (agent
1).
We now describe how to set up an external system for strategy development. There are two major steps you need to follow to install everything you need in your system. First, the UNP communication library, used by the PLAT client, needs to be installed. Then, the plat-client package needs to be installed.
Before you begin, you first need to choose two possibly mutually exclusive directories where those UNP library and plat-client packages will be installed. The specific directory names or locations where the UNP library and the plat-client package is installed do not matter. From now on, we'll refer to those directories by UNPV_DIR and CLIENT_DIR.
To develop their automated traders in an external machine, you need to
install the UNP communication library provided by Richard Stevens' Unix Network
Programming book. The source code can be found at:
http://www.kohala.com/start/unpv12e/unpv12e.tar.gz
We assume you saved the file in a directory that we refer to as UNPV_DIR. After that, the user can use the
command (to unzip and untar in the current directory):
tar -xvzf unpv12e.tar.gzThat should create a directory unpv12e in UNPV_DIR, and uncompress all the necessary files inside it.
The next step is to compile the network library without errors. Here is Stevens' original compilation notes together with other comments added to ease up the compilation process.
Once you are inside UNPV_DIR,
cd unpv12eIf you are unable to compile using ' make' (the last step above), then follow the modifications suggested below and try again. The exact nature of the modifications needed might differ from one configuration to another. Below are general tips from our crux installation. After you change something in the UNP source code, we recommend you execute ' make clean' before calling 'make' again to be certain that your changes are fully incorporated.
./configure # try to figure out all implementation differences
cd lib # build the basic library that all programs need
make # use "gmake" everywhere on BSD/OS systems
The following are the particular changes we needed to make for the UNP installation on crux (UNPV_DIR=/usr/local):
/* Commented out the following because it is a redefinition of a structure
* with slightly different/missing fields defined in system's original
* header files.
*/
//struct in_pktinfo {
//struct in_addr ipi_addr; /* dst IPv4 address */
//int ipi_ifindex; /* received interface index */
//};
/* Changed the call to mktemp with call to mkstemp */
if (mkstemp(template) == -1 || template[0] == 0)
err_quit("mkstemp error");
That's all we needed to change for this part of the UNP installation process, and then the compilation worked without errors (for the most part, you can safely ignore any warnings you see during the UNP library compilation).
In general, one additional possible change you might need to make is to edit the file config.cache in the UNPV_DIR/unpv12e directory. If you get an error, either during this step of building the UNP library or the compilation of your strategies, regarding the function getnameinfo, you want to make sure that the following line regarding getnameinfo in config.cache is exactly as follows:ac_cv_func_getnameinfo=${ac_cv_func_getnameinfo=yes}(i.e., make sure it is set to "yes" as above!). If you have to make this change, you should re-execute the following sequence of commands starting at the UNPV_DIR/unpv12e directory,
./configure
cd lib
make clean
make
Before finishing off with the UNP installation, you also need to execute the following:
cd UNPV_DIR/unpv12e/libfree # continue building the basic libraryIf all that works (without errors!), you're done with the compilation of the network layer. If you wish, you can now safely remove the compressed file UNP package file you downloaded (or the tar file that resulted if you directly uncompressed the file) after successfully building the UNP library.
make
The next stage is setting up the plat-client package in the CLIENT_DIR directory. Connect to crux.cis.upenn.edu and retrieve the file
/home/pxs/PAT/pub/plat-client.tar.gz
and save it into your local CLIENT_DIR directory.
For instance, if the unix tool rsync is available in your system, one easy way you can download this file to your system is to type the following, once you are inside CLIENT_DIR,
rsync -e ssh USERID@crux.cis.upenn.edu:/home/pxs/pub/plat-client.tar.gz .where USERID is your user account name on crux. Think of this command as a remote version of the unix command cp . In the particular execution above, it will download that file to the directory where you executed the command.
Here are some quick instructions for setting up the plat-client package. You just need to execute the short sequence of commands that follows (as stated below, you need to make a slight modification of the file Make.defines provided with the package before executing the make command in the sequence!):
tar xvzf plat-client.tar.gzThe tar command will generate the plat-client directory, as well as other subdirectories which include the package source code and other auxiliary files. Running 'make init-install' will (a) compile the plat-client library, (b) set up the strategy development directory named strat inside plat-client, (c) copy the source code for an initial strategy in strat, as well as appropriate Makefiles, and (d) finally compile that strategy to generate an executable named mytrader in the strat directory. Please note that the command 'make init-install' is only meant to be executed once, during the initial installation only, since subsequent executions will erase your strategy's main source code files! See below for more information on how to develop and compile your own strategies. Following is a detailed description of the command sequence above.
cd plat-client
Edit file Make.defines to specify UNPV_DIR and system type: defaults are UNPV_DIR=/usr/local and linux
make init-install
Once you saved the plat-client package (tar.gz) file in CLIENT_DIR, untar and unzip it. One way to do this is by executing the following command:
tar xvzf plat-client.tar.gzThis will generate a directory named plat-client inside CLIENT_DIR.
Now, go to that directory,
cd plat-client
Next, you need to edit the file Make.defines there to specify the location of
the UNP library (See instructions inside the file). Essentially you just need to
edit the line
UNPV_DIR=/usr/local
there to replace
/usr/local by the directory path you
chose to install the UNP library. Also, if your system is solaris, you need to
edit that file as described with comments inside the file (essentially, you have
to comment 2 lines and uncomment 2 others). The default setup is for linux
systems. From this point on, we assume this file contains the information
specific to your system and will also be used to compile your own strategies
later.
To build and install the plat-client library, and also setup an initial strategy and compile it, you just need to execute
make init-installin that directory. It is important that you execute the last command only once (during the initial installation), since any subsequent execution will erase the strategy files in the strategy development directory CLIENT_DIR/plat-client/strat, and therefore your strategy main source code will be erased!
If everything above worked without errors, you are now ready to develop you own strategies in the directory CLIENT_DIR/plat-client/strat. The files strategy.c and strategy.h located in that directory will form the building blocks for building automated trading strategies, and instructions to modify them are inside them. One can build very sophisticated strategies by using these files in association with the client-shell API.