Backfilling data after a disconnect

Posted By: Zheka

Backfilling data after a disconnect - 03/19/18 13:32


Zorro does not seem to backfill missing data after the disconnect of an MT4 terminal(!?)

How to work with this in live trading?
One has to programmatically identify such situations, send email/text alert and then get to manually stop/restart Zorro? This becomes a less reliable and less automated solution.

Also, upon reconnect to an MT4 terminal, Zorro showed "offline" for another 1-2min before actually reconnecting.
Is there a way to make it reconnect as soon as the data source is available again?
Posted By: Zheka

Re: Backfilling data after a disconnect - 03/19/18 14:35

Some more details re behaviour at reconnect:
- if MT4 closed/ restarted quickly, Zorro reconnects quickly
- during the disconnect, seconds in Zorro status line change "chaotically" (as if to the arriving ticks rythm) and not always forward...
- if disconnect is longer then 3-5min (in my case BarPeriod=5min), then the square switches from Red to Purple.
- message in MT4 switches to "Zorro disconnected!" (sometimes - not)
- when MT4 is restarted after longer disconnect, Zorro reconnects in 3.5-4min...Message in MT4 sometimes stays "Zorro disconnected" and sometimes gets back to "Controlled by Zorro bridge 1.15"..
Posted By: jcl

Re: Backfilling data after a disconnect - 03/21/18 12:51

Look also here: http://manual.zorro-project.com/trading.htm under "Getting disconnected".
Posted By: Zheka

Re: Backfilling data after a disconnect - 03/21/18 13:07

OK.
But the questions still remain:

- maintaining data continuity/backfilling is nearly standard a feature of most trading engines; I really took it for granted.
Can Zorro be improved in this respect?

- how to shorten the time of Zorro reconnect attempts? 3.5-4 min is an eternity in many cases.
Posted By: jcl

Re: Backfilling data after a disconnect - 03/21/18 15:42

What do you want to "backfill"?
Posted By: Zheka

Re: Backfilling data after a disconnect - 03/21/18 15:56

On reconnect to the broker server, Zorro should request the number of bars missed since the disconnect and THEN continue script execution.
Posted By: jcl

Re: Backfilling data after a disconnect - 03/21/18 16:03

There are no bars when there's no connection. You could not do much with the requested prices.
Posted By: Zheka

Re: Backfilling data after a disconnect - 03/21/18 16:24

Quote:
You could not do much with the requested prices.
Script execution should continue from the bar it stopped. All system timeframe formation, indicator, etc calculations will be orderly.
Why would we not want that?

Yes, a user might need to be warned that a trade might have been missed, but that is a lesser/ rare evil to handle.

A flag indicating a connection status to the broker and to the price feed would also be extremely helpful.
Posted By: jcl

Re: Backfilling data after a disconnect - 03/22/18 08:04

Do you know a broker API with a connection status flag?
Posted By: Zheka

Re: Backfilling data after a disconnect - 03/22/18 12:15

Quote:
Do you know a broker API with a connection status flag?
Sure. Most broker APIs provide error codes on connection status.

https://interactivebrokers.github.io/tws-api/message_codes.html#gsc.tab=0

http://fxcodebase.com/bin/forexconnect/1.5.0/help/CPlusPlus/web-content.html#SessionStatus.html

https://docs.mql4.com/constants/errorswarnings/enum_trade_return_codes

FIX/DTC use "heartbeats" to confirm that connection is "alive".

My suggestion is for Zorro to expose this to the end-user via some sort of flag or variable,
regardless of how Zorro internally determines that the connection is "offline" (Zorro already does this, right?)

Exposing order rejection errors would also be helpful.
Posted By: jcl

Re: Backfilling data after a disconnect - 03/22/18 12:31

Thanks, but none of your links is a usable connection status flag or similar function. But if you still find one, please post it here. I know that it is difficult.

FXCM does not detect interruptions, only logouts or server outages on their side. The LastErrorCode by MT4 can return ERR_NO_CONNECTION when sending an order. But it's no good idea to send orders for testing the connection status.

Posted By: Zheka

Re: Backfilling data after a disconnect - 03/22/18 13:35

https://docs.mql4.com/check/isconnected

This seems to the flag for MT4.

LastError() code can also be used - to signal a relevant error.

Both MultiCharts and SierraChart use IB API error codes to determine the status of connection. SC then exposes this further to a user via a variable at run-time.
Posted By: Zheka

Re: Backfilling data after a disconnect - 04/03/18 13:29

JCL,

is there any update on this - for MT4 and IB?
Posted By: AndrewAMD

Re: Backfilling data after a disconnect - 04/03/18 13:47

Originally Posted By: Zheka
FIX/DTC use "heartbeats" to confirm that connection is "alive".
This would only confirm the connection between DTC client and DTC server and not necessarily the layer behind it (i.e. Sierra Chart's connection to its server).
© 2024 lite-C Forums