Jcl,
It is stated numerous times all over the manual that
Historical price data for backtests can not be downloaded from IB
or that they are very slow and limited.
This proves true - with Zorro. But not with other platforms.
I have been using both Multicharts and Sierra with IB, and historical data download has always happened quite swiftly.
Just did a test, downloading 5 days of EUR.JPY 1 min data.
MultiCharts- <1min
Zorro - 8min - stopped after loading 1 day..
Looking at the IB Gateway logs, the reasons are as follows:
1. Zorro requests data in 36000s(=10hrs) chunks, and re-requsts every 5hrs (??!)
2017-10-06 09:20:46.293 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:DET] ReqHistoricalData(20)::[version=6,ID=4,action=null,reqDesc=Symbol=EUR Type=CASH Expiry=null Strike=0.0 Put/Call=? Exchange=IDEALPRO CompExch=null Currency=JPY Multiplier=null IbLocalSymbol=null IbTradingClass=null SecIdType=null SecId=null includeExpired=false needLeadFutureMonth=false needContinuousLeadFutureOnly=false newsSource=null Legs=null Special Info=null,endDateTimeStr=20171006 13:20:46 GMT,backfillDuration=36000 S, whatToShow=4,barSizeSettingStr=1 min,formatDate=2,combo=null]
2017-10-06 09:21:57.391 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:DET] ReqHistoricalData(20)::[version=6,ID=6,action=null,reqDesc=Symbol=EUR Type=CASH Expiry=null Strike=0.0 Put/Call=? Exchange=IDEALPRO CompExch=null Currency=JPY Multiplier=null IbLocalSymbol=null IbTradingClass=null SecIdType=null SecId=null includeExpired=false needLeadFutureMonth=false needContinuousLeadFutureOnly=false newsSource=null Legs=null Special Info=null,endDateTimeStr=20171006 08:21:00 GMT,backfillDuration=36000 S, whatToShow=4,barSizeSettingStr=1 min,formatDate=2,combo=null]
while MC requests data for 5d in one go.
2017-10-06 09:54:39.228 [IL] INFO [JTS-EServerSocket-3491] - [21500:63:76:1:0:0:0:DET] ReqHistoricalData(20)::[version=6,ID=1000007,action=null,reqDesc=Symbol=EUR.JPY Type=CASH Expiry=null Strike=0.0 Put/Call=? Exchange=IDEALPRO CompExch=null Currency=JPY Multiplier=null IbLocalSymbol=null IbTradingClass=null SecIdType=null SecId=null includeExpired=false needLeadFutureMonth=false needContinuousLeadFutureOnly=false newsSource=null Legs=null Special Info=null,endDateTimeStr=20170929 21:00:00 UTC,backfillDuration=5 D, whatToShow=4,barSizeSettingStr=1 min,formatDate=2,combo=null]
From IB API guide(
https://interactivebrokers.github.io/tws-api/historical_limitations.html#gsc.tab=0):
Historical Data requests need to be assembled in such a way that only a few thousand bars are returned at a time.
And so, it takes MC 15sec from the request to closing the historical data socket.
2. besides "Ask", Zorro also requests data for "Trades" -
which do not exist for FX.2017-10-06 09:20:47.152 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:DET] ReqHistoricalData(20)::[version=6,ID=5,action=null,reqDesc=Symbol=EUR Type=CASH Expiry=null Strike=0.0 Put/Call=? Exchange=IDEALPRO CompExch=null Currency=JPY Multiplier=null IbLocalSymbol=null IbTradingClass=null SecIdType=null SecId=null includeExpired=false needLeadFutureMonth=false needContinuousLeadFutureOnly=false newsSource=null Legs=null Special Info=null,endDateTimeStr=20171006 13:20:46 GMT,backfillDuration=36000 S, whatToShow=0, barSizeSettingStr=1 min,formatDate=2,combo=null]
2017-10-06 09:20:47.152 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:DET] [6;5;0;EUR;CASH;null;0;2;null;IDEALPRO;JPY;null;null;false;false;20171006 13:20:46 GMT;1 min;36000 S;0;TRADES;false]
.......
2017-10-06 09:20:47.152 [IL] INFO [JTS-cashhmdsDispatcherS42-3413S42-3414] - cdebug: QUERY | WARNING | qm@305bef71 | Query error | 1755;;EUR.JPY@IDEALPRO Trades;;1;;true;;0;;I | No historical market data for EUR/CASH@FXSUBPIP Last 60
It should have been "Bid" (whattoshow=3), correct?
3. There is a huge lot of strange UnSet() messages generating errors.
2017-10-06 09:20:49.031 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:INFO] Handling incoming UnSet(0) message.
2017-10-06 09:20:49.031 [IL] INFO [JTS-EServerSocket-3406] - [1:63:76:1:0:0:0:ERR] Invalid incoming request type - 0
There are also numerous ReqMrktData calls in the first 27 seconds of connection and ReqAccData in the very beginning (even though I pressed [Test], not [Trade]. Script is only "UpdateDays=5"). The first call for historical data is 27sec after pressing the "Test" button (2sec for MC).
Log files are attached.
Btw: MC opens 2 client connections in IB Gateway
