I was wondering how the marketVol function is supposed to be used. When invoked, it returns unexpectedly small values only. When looking at daily asset data from stooq, market volume numbers are very large, going into hundreds of millions, whereas Zorro returns values in the ones and tens. Upon investigation, it looks like the application returns only the first minute volume value, and not the sum of all values within a bar. Consider this script:
I am using the data from the Zorro website. But even if marketVol would return the sum, the number would still be relatively small. Like lets say 5 (approx. volume per minute) * 1440 (minutes) = 7200 (volume per day). This is far from the expected volume of hundereds of millions. In what unit are the Zorro data volume numbers?
The Manual says "trade volume per minute": http://zorro-project.com/manual/en/price.htm But if this is really only ever returning minute volume, how can the user access the daily/hourly/etc volume, when the given function argument takes a bar (as per BarPeriod) only?
I don't understand this. Any help appreciated. Thanks.
Last edited by pascalx; 11/02/1718:24.
Re: About marketVol() function
[Re: pascalx]
#469024 11/02/1719:1211/02/1719:12
No, in D1 history there is no difference of start and end, since it has no hours. The date of the open is identical to the date of the close. This is considered by Zorro. Otherwise the calculation would be based on a wrong time, since a date literally begins at 00:00 and shifted to UTC you would get a date boundary between open and close.
But you would think Zorro would add up those bars anyways, right?
That's because daily data (from Stooq etc) is stored differently than M1 data.
If the volume of M1 data is different to the volume of D1 data, how does someone convert the M1 volume to D1 colume when using BarPeriod = 1440 and M1 tick data?
Re: About marketVol() function
[Re: pascalx]
#469027 11/02/1723:1911/02/1723:19
Whoops, I forgot to address the elephant in the room:
Decentralized instruments will always have different tick volumes and/or volumes between exchanges. The SPX500 CFD is one of those instruments.
Next, I was unclear:
Quote:
If the volume of M1 data is different to the volume of D1 data, how does someone convert the M1 volume to D1 colume when using BarPeriod = 1440 and M1 tick data?
Assuming data is coming from the same data vendor (and using the same metric, not confusing tick volume with actual volume), 1440 M1 data points should always add up to the equivalent D1 bar. A D1 has a DATE float value rounded down to the nearest integer to indicate which day.
No, Zorro does nothing with volume. So what you do with the value returned by marketVol is completely up to the broker plugin and to your script. The "volume per minute" is a suggestion what _could_ be returned.
Re: About marketVol() function
[Re: jcl]
#469147 11/06/1712:4111/06/1712:41
No, Zorro does nothing with volume. So what you do with the value returned by marketVol is completely up to the broker plugin and to your script. The "volume per minute" is a suggestion what _could_ be returned.
Damn. This is troubling. I see MT4/5 does not even support volume data at all:
Quote:
marketVal: Not supported. marketVol: Not supported in historical data, tick frequency in live data.
Why is that? MT4 does know about volume. Wouldn't it be natural to pass that in marketVol instead of 'tick frequency', whatever that is? https://docs.mql4.com/predefined/volume
So you have marketVal and marketVol as additional, optional values. Any particular reason why 2? Are 2 the maximum of optional data values someone ever needs?
This idea of a function called marketVol(int) (assuming "Vol" stands for "Volume") could return something other than volume looks like a design flaw. If the broker sends some other data for it, then ideally the user should not require to add checks in his code prior calling marketVol to make sure it is indeed a volume value. marketVol should just return 0, indicating that it has no known volume data from that current data source.
To accommodate arbitrary data values you would expect a marketVal(int index, int offset) function, where the given index selects the optional data values at their natural position. I would even consider to add a new data structure such as:
Where the pointer of fVal would point to an additional series of arrays. This would allow for true arbitrary optional data values.
I would suggest to mark marketVol(int) and marketVal(int) as deprecated and add new functions marketSpread(int), marketVolume(int) and marketVal(int index, int offset). marketSpread(int) and marketVolume(int) should return spread and volume if the system knows that the values are indeed what they promise, for example by the broker plugin, or t6 data files, otherwise 0.
Now to the next problem: How to correctly collect the optional asset data for a given bar. The current design gives the user no possibility to collect correctly. So marketVol(int) and marketVal(int) are useless in many cases. The proposed marketSpread(int) and marketVolume(int) functions could automatically solve this problem, because since they know what the value is, they can internally collect the value correctly. For example while priceHigh(int offset) collects the highest value within the bar, marketVolume(int) would collect the sum of all values within the bar. Sounds fair?
Now, what about marketVal(int index, int offset)? It probably needs a third parameter taking an optional function pointer to tell how data should be collected. So marketVal(int index, int offset, void* collectFunction), where the collect function could look something like
Code:
var collect(var collectedValue, var nextValue)
{
return collectedValue + nextValue;
}
var collect(var collectedValue, var nextValue)
{
return max(collectedValue, nextValue);
}
The collect function gets called by zorro for each data point within the given bar. This would enable the user to specify how zorro should collect the optional asset data. void* collectFunction should be a proper function pointer if Lite-C supports it.
Anyone agrees with this?
Re: About marketVol() function
[Re: pascalx]
#469148 11/06/1714:3311/06/1714:33
That's unfortunately not our's to decide. Only prices are delivered by all brokers, any other data is up to the broker. If they deliver tick frequency instead of volume, marketVol returns the tick frequency. If they deliver spread, marketVal returns the spread. If they deliver nothing, the functions return 0. Different function names or bigger structs won't help.
For this reason, it makes no sense to guarantee a certain volume format. The only requirement is that marketVol returns at least some proxy of volume. Since almost all volume based indicators only use volume changes and not absolute volume, they will work. How useful such indicators are with a MT4 connection where volume is normally the tick frequency, is a different question.
Re: About marketVol() function
[Re: jcl]
#469151 11/06/1716:2211/06/1716:22
Can we add a new brokerCommand that allows the plugin to specify the output format of *pVolume? This way, Zorro would have no trouble determining how to handle the volume.
* New brokercommand: GET_BROKERASSET_VOLUME_TYPE - It will return the below values: * 0: Not specified. Volume must be manually handled. (What Zorro does now.) * -1: Absolute-point-of-reference number of ticks. * -2: Number of ticks since last BrokerAsset call for this asset. * -3: Frequency of ticks in ticks/minute. * 1: Absolute-point-of-reference number of lots traded. * 2: Number of lots traded since last BrokerAsset call for this asset. * 3: Frequency of lot trades in lots/minute.
And then finally, on the user side, let the Zorro user specify how to interpret the historical volume with a similar type of flag.
That's unfortunately not our's to decide. Only prices are delivered by all brokers, any other data is up to the broker. If they deliver tick frequency instead of volume, marketVol returns the tick frequency. If they deliver spread, marketVal returns the spread. If they deliver nothing, the functions return 0. Different function names or bigger structs won't help.
For this reason, it makes no sense to guarantee a certain volume format. The only requirement is that marketVol returns at least some proxy of volume. Since almost all volume based indicators only use volume changes and not absolute volume, they will work. How useful such indicators are with a MT4 connection where volume is normally the tick frequency, is a different question.
If the broker decides to send a garbage or no value for volume, and MT4/MT5 hands that garbage/no value to Zorro, why is that a problem Zorro would need to tackle, when the problem is on Broker/MT4 side? You also would not tackle the event if the broker would send garbage for open/high/low/close right? You just take the value as is in the remote platform and hand it over.
Furthermore I don't understand how the current implementation can be any satisfactory, nor do I understand how Andrews proposal fixes it all.
Summary of issues: - Cannot access/use optional asset data values with BarPeriod > 1 (e.g. when backtesting .t6 data) - Cannot retrieve volume and spread with MT4/MT5 - Limit of 2 optional asset data values. Cannot use 3 or more - marketVol/marketVal might not be what they pretend to be / user has to find out / adapt code to different use cases
Originally Posted By: jcl
Different function names or bigger structs won't help.
I did not simply propose different function names or bigger structs. Was my previous text really that non-descriptive?
Re: About marketVol() function
[Re: pascalx]
#469173 11/07/1709:2411/07/1709:24
MT4 delivers no volume, and cannot deliver it since it's mainly a Forex and CFD trading program. Forex and CFD have no volume. You confuse that with the tick frequency.
You can see that it's really the tick frequency when you display marketVol() on EUR/USD live with an MT4 connection, and at the same time observe the ticks in the MT4 terminal. That's why the manual mentions "tick frequency" for MT4 marketVol, not "Volume". And no, there's no historical spread by MT4 either, at least as to my knowledge.
Re: About marketVol() function
[Re: jcl]
#469179 11/07/1712:2411/07/1712:24
MT4 delivers no volume, and cannot deliver it since it's mainly a Forex and CFD trading program. Forex and CFD have no volume. You confuse that with the tick frequency.
You can see that it's really the tick frequency when you display marketVol() on EUR/USD live with an MT4 connection, and at the same time observe the ticks in the MT4 terminal. That's why the manual mentions "tick frequency" for MT4 marketVol, not "Volume". And no, there's no historical spread by MT4 either, at least as to my knowledge.
Hi. Yes I might be mistaken on this matter.
A quick google search indeed revealed that the functionality labeled as "volume" in MT4 returns a tick frequency.
Still there appears to be spread information according to the MT4 manual, at least for live feeds. Why is that not bridged?
I will ask a buddy who has great experience with MT4 and forward his assessment on this matter - if he has any.
Do you have opinions about the other 3 issues I mentioned?
Re: About marketVol() function
[Re: pascalx]
#469181 11/07/1712:3711/07/1712:37
I do not really understand the spread issue. All brokers deliver live spread and it's available in the "Spread" variable. marketVal is for an arbitrary additional parameter, not necessarily for spread.
As to retrieving more than 2 additional parameters from the broker - can you give me an example of which other parameters from which broker you would need?
Re: About marketVol() function
[Re: jcl]
#469233 11/09/1710:0811/09/1710:08
I do not really understand the spread issue. All brokers deliver live spread and it's available in the "Spread" variable. marketVal is for an arbitrary additional parameter, not necessarily for spread.
As to retrieving more than 2 additional parameters from the broker - can you give me an example of which other parameters from which broker you would need?
Ok the spread makes sense. I don't have an example for more than 2 additional parameters. It might be no practical issue, and just a theoretic one, considering we are asking that question and are not sure of the answer. Ideally we would not have to ask that question.
Ok down to 2 issues/questions. Any opinions on points 1 and 4 ?
1. Cannot access/use optional asset data values with BarPeriod > 1 (e.g. when backtesting .t6 data) 2. Cannot retrieve volume and spread with MT4/MT5(no issue) 3. Limit of 2 optional asset data values. Cannot use 3 or more (no practical issue) 4. marketVol/marketVal might not be what they pretend to be / user has to find out / adapt code to different use cases
Re: About marketVol() function
[Re: pascalx]
#469247 11/09/1716:2311/09/1716:23
To 1., I'm not sure that I understand the problem - can you give an example?
In the opening post of this topic I added a sample script that shows the problem. When looking into the actual t6 data (with ZHistory tool) then it becomes obvious that the given values are unexpected.
Originally Posted By: jcl
To 4., you can find the returned data by any specific API on the API page in the manual.
Yes. Unfortunately not all pages show this information.
But as I said earlier this could easily be improved by providing functions that really return what they claim to return.
For example instead of a marketVol(int) function you could have a marketTickFrequency(int) and marketVolume() function that internally are aware if they return what they promise.
* -1: Absolute-point-of-reference number of ticks. * 1: Absolute-point-of-reference number of lots traded.
jcl,
I should offer some clarification on how I expect this to work.
Zorro should anticipate that the absolute point of reference volume should occasionally reset. For example, the DTC protocol defines volume as "volume for this session". If, in this case, the connection is lost and must restart, volume will probably return to zero.
Pseudo-code: if (AbsVolume[0] < AbsVolume[1]) VolumeSinceLastBrokerAssetCall = 0;
#define GET_PRICETYPE 150 // type of prices returned by the API
#define SET_PRICETYPE 151
#define GET_VOLTYPE 152 // type of volume returned by the API
#define SET_VOLTYPE 153
As of this writing these are not documented - I look forward to it.
I assume that if I supply absolute or relative volume from the plugin, Zorro will already know how to translate this into the appropriate volume.
For example, if a user's historical volume is volume at that bar, and the plugin provides volume since the last brokerasset call, I expect Zorro to accumulate volume for every given bar. Are my expectations correct?
It sounds like if a user wanted to replicate a volume-per-bar (such as for a money flow index indicator), then this is what he will need to do:
* Set future brokercommand SET_VOLTYPE to absolute volume (either ticks or real) * Therefore, future brokercommand GET_VOLTYPE will acknowledge this setting. * User will then detect changes in absolute volume and use that as real volume or tick volume for a given bar.
Is this correct?
If so, I would definitely use the absolute ticks setting for any forex plugin and an absolute volume setting for stocks, options, and futures.
Question: Were you planning on adding these brokercommands to the IB, FXCM, and Oanda plug-ins?
I also understand that, depending on how the plugin collects history data, whether from newest to oldest or oldest to newest, that it might require absolute volume to be accumulated in reverse order. That is... * Newest volume is zero * Previous tick is higher * The tick before that is even higher.
So you can leave that to the end user to straighten out. (Or I might publish a handy helper function or two in the headers.)
I know no API where you can set the volume type. So SET_VOLTYPE is at the moment only theoretical, a placeholder for a possible future feature that will probably be implemented in Zorro, not in the API plugin. Normally, trade volume is relative to a starting point in live trading, and absolute in historical data. Tick volume is always absolute. History data is always stored from newest to oldest.
Re: About marketVol() function
[Re: jcl]
#470103 12/22/1714:2812/22/1714:28
Is there a way that I can alter historical data in-place? Suppose I want to edit only the marketVal values - all of them. Will I need to edit the historical files explicitly, or is there a high-level Zorro approach available?
My plan is to have M1 historical-only volume in marketVol, and then I can calculate absolute volume in marketVal. Then my plugins will put live absolute volume in marketVal.
This way, I can compare historical fluctuations in absolute volume with real fluctations in absolute volume.
_____________________________ UPDATE:
Well, I did figure out how to read the T6 files, it's literally just a binary array of T6 structs, with the newest entries in the front. I just need to overwrite that file.
_____________________________ UPDATE #2:
On the plugin side, for live data, I see a way to explicitly set an output for MarketVol (double *pVolume) but not for MarketVal, so it appears that my approach outlined above will not work.