OmniTrader Archives
OmniTrader 2018 Technical Support
ATS Trade Ranking Issue
22 replies, 3454 viewings
|
OmniTrader Archives
OmniTrader 2018 Technical Support
ATS Trade Ranking Issue
|
Last Activity 5/11/2020 10:43 AM 22 replies, 3454 viewings |
|
|
|
| Printer friendly version |
Vinay Elite    Posts: 640 Joined: 12/9/2011 Location: Planet Earth  User Profile |
You can put any formula in the Short box and will get the same result but if kept blank then results will be different. [Edited by Vinay on 9/18/2018 1:48 AM]  Attached file : ShortError.jpg (50KB - 1020 downloads) | ||
| Jim Dean Sage   Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
The reason for the difference in the Long trades is that if a strategy has a short trade active, then during that time (and possibly a short time thereafter based on your System Block settings), Long trades have to wait. Otoh, if a formula in the Short filter blocks some or all of the short trades, (“false” or “2=3” does this effectively), then all possible Long trades (given the other strategy settings) will be taken. | ||
| Vinay Elite Posts: 640 Joined: 12/9/2011 Location: Planet Earth User Profile |
I have configured the Systems block in the strategy to generate Long Signals only, so the question of Short Signals blocking Long Signals doesn't arise. | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
| ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
I'd expect to not see any, but if some slipped in for some reason, that might point to a problem in a different area. | ||
| Vinay Elite Posts: 640 Joined: 12/9/2011 Location: Planet Earth User Profile |
| ||
| Barry Cohen Sage  Posts: 6338 Joined: 1/19/2004 User Profile |
| ||
| Vinay Elite Posts: 640 Joined: 12/9/2011 Location: Planet Earth User Profile |
"(ATM) Deleting a short ranker & leaving it blank no longer results in an error running Portfolio Simulation analysis." I wanted to clarify whether it refers to the issue documented in this thread or it is different. My confusion is due to the fact that I reported this issue in the context of ATS whereas the change log refers to ATM. | ||
| Barry Cohen Sage Posts: 6338 Joined: 1/19/2004 User Profile |
| ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
To troubleshoot, I created a baseline 10% equity, 1 trade per day method and methods using VTY_PRICE ascending and descending. I fully expected the 3 methods to all get different results. But the two ranking methods are producing the exact same equity curves. I've tried recompiling everything, restarting OT, even moved the ITS files from OT2018 to OT2019 and nothing resolves the problem. At one point I somehow managed to get the three methods to give different results, but after trying to apply an RSI ranking, it didn't work. In fact I was unsuccessful in switching back and getting VTY_PRICE to work again. I've scrutinized every setting in each method to make sure they are the same (except the ranking). I've examined the ITS.txt files to make sure they reflect all the settings. And as mentioned previously in this thread, I do have formulas for the short conditions set even though I am allowing longs only. Here is the ranking tab for each of the methods. Any one else seeing or able to reproduce this? I can't take any result I am getting as meaningful. [Edited by SJ on 2/25/2019 3:39 PM] Attached file : ATM_trouble.JPG (52KB - 771 downloads) | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
I have been struggling with a similar problem concerning VTY_PRICE. If you look at the ATM Methods issued to date all of them sort both Long and Short in the Conditions Box Descending. And usually with the (10,5) Parameter set like you are using. In my situation using a custom Market State I developed I was trying to optimize the value of the Ranker Default. Every time I did so though whether sorting Long and Short both Descending or Long Ascending and Short Descending (which I thought to be the better choice and I'll show why in a second) the Optimization returned a value of zero (0). But that may have had to do with the trades taken in my Market State although I thought the result a bit strange. So, what is the description of VTY_PRICE? From the OT Help File: Note that it is true that high values of the Volatility Price Ratio does indeed seem to portend a reversal - but mainly a reversal that taking a Long position in might benefit from. Contrarily a Short position might be seen to benefit from a low value of the Volatility Price Ratio. Here it is on LLY - I just happened to be looking at it today for a trade. Note the similarity in respect to my analysis above: So, here is what I would suggest - try the Sort with Long Ascending and Short Descending with those boxes checked in the Conditions interface even though you are only trading Long only and see what happens. Note - I haven't tried this but it should be a simple test. Tom Helget [Edited by EYEGUY on 2/26/2019 9:54 AM] Attached file : Volatility Price Ratio Definition.jpg (56KB - 733 downloads) Attached file : VTY_PRICE.jpg (75KB - 709 downloads) | ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
Thanks for the reply. Appreciate your observations on the behavior of VTY_PRICE. AT this point, it is not the behavior of how any given ranker affects performance, it is simply that if applying a ranker that has a known effect and we throttle down the number of trades allowed to 1 per day, there should be a detectable difference in the equity curves when trades are based on ascending vs descending ranker value. They should not be exactly the same. I don't know if this means the ascending/descending radio buttons are not working or what is happening. I can see from the log files that the same trades are being taken and the others rejected. BTW, although the thread is for ATS, I am using ATM for this test with just a Default market state. Ran today, long only. Test 0 is baseline. Test is VTY_PRICE descending, Test2 is ascending. This looks correct! Switched to RSI(2) ranker. Strategy is a variation of XLS-19, so RSI level should be predictive varying between 0 and 10 (10 is the signal cutoff level). Test 0 is baseline. Test 3 is RSI(2) ascending, Test 4 is descending. There is no difference in Test 3 vs 4. Also, I would have expected a lower RSI value to be more profitable on average, but the baseline performed better. So what sort is happening? Edited Test 3 & 4 to switch back to VTY_PRICE like Test & Test2 were. Much to my surprise, the 3 curves presented as before. So I edited 3 & 4 so that 3 was VTY_PRICE ascending and 4 was descending. If ranking is working, 3 & 4 should have reversed, but they did not change at all. I even exited back to Real Mode and back to Port Sim to be sure – no change. So just to cover every base, I made sure to my ascending/descending were reversed for long/short as suggested by Tom. This made no difference, but when I put it back to original, my three curves became two! Now Test 3 & Test 4 produce the same equity curve despite having opposite direction rankings. I’ve broken it. WTF? Shut down OT and restarted. Port Sim still producing only two curves. This is madness! This is Sparta! Steve Johnson [Edited by SJ on 2/26/2019 4:23 PM] Attached file : Trouble1.png (27KB - 694 downloads) Attached file : Trouble2.png (26KB - 635 downloads) Attached file : Trouble3.png (25KB - 699 downloads) | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
I have to admit that playing with the Rankers is, for me too, at best a wonky experience. One other thing I noted reviewing this thread is that for your VTY_PRICE Rank Ascending and Rank Descending Tests you didn't have the "Scale Indicator Values" box checked. What this does is to scale the indicator you are investigating from 0 to 100 and, in the case of VTY_PRICE that would be helpful if you eventually decided to enable both RSI and VTY_PRICE in your testing paradigm. Early on working with ATM I was reluctant to do that since I didn't know what effect checking the box would have, but found that it was helpful when I did check it. Just a thought.... Tom Helget | ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
In my file, here are the max and min values for VTY_PRICE: RangeMaxLong="0.1029571" RangeMinLong="0.0018801" RangeMaxShort="0.1029571" RangeMinShort="0.0018801" And here are the max and min values for RSI2: RangeMaxLong="100" RangeMinLong="0" RangeMaxShort="100" RangeMinShort="0" What is curious here is that I am a little surprised that RSI(2) hit both 0 and 100 in the data set. But here is the more important point. RSI varies between 0 and 100, but on any given signal day, by definition, RSI(2) was below 10 or there would be no signal generated. So to mix scaled VTY_PRICE and scaled RSI equally in a weighting scheme is not going to be effective as trades are more likely to be spread across the VTY_PRICE range, but will be concentrated in the lower 10% of the RSI range. If your goal is to give both an approximate equal contributing value, you need to scale up RSI by a factor of ten (IMHO). Not to say equal weight is ideal. That is where optimization can help solve the puzzle and I would guess it also resolves the imbalance discrepancy I just outlined. Unfortunately, all that is moot if I can't even get the most basic of ranking functions to act consistent and repeatable. It really looks like it sometimes gets stuck and can't properly distinguish which is the correct sort direction. I would appreciate if anyone else could attempt to recreate my scenario(s). | ||
| Jim Dean Sage Posts: 3022 Joined: 9/21/2006 Location: L'ville, GA User Profile |
Another sure fire solution is to create a custom OLang indicator that does everything transparently, creating and combining normalized versions of each indicator that’s to be used. The advantage of this is that there’s no mystery, and that you can do all kinds of interesting things re weighting the indicators or whatever. | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
You should not be surprised about the RSI(2) value being either 0 or 100 - as a matter of fact it is rarely anything but: If you are a Nirvana Club member you might also want to check out my woes with the ATM Ranker tab here: http://forum.thenirvanaclub.com/thread-view.asp?threadid=7955&posts=1 One thing I would suggest if you are not already doing it is to not just assume that once you have made a change in your ATM Method that you can just go on working with it in OT. I would advise you to shut OT down and then upon re-invoking it double check everything in the ATM Method for accuracy and then, and only then, do your run. Tom Helget [Edited by EYEGUY on 2/26/2019 5:32 PM] Attached file : RSI(2) on SPY.jpg (199KB - 658 downloads) | ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
Jim, good points on alternatives to testing combined rankers. Frankly, the weighting and multiple ranker discussion, while interesting, was straying from the original issue. Taking Tom's advise and some extra measures, I verified my method settings, shut down OT, purged the results folder and the VBA/Temp folder, restarted OT, downloaded data and then analysis. I go to Port Sim and find the two methods (Rank by VTY_PRICE ascending/descending) are still giving the exact same equity curve, both higher than with no sort. As we have generally observed that sort descending (long only) is the beneficial configuration, the conclusion is that both are sorting descending despite the settings. Why I was apparently getting "correct" results earlier today, but cannot replicate that now remains an utter mystery. Tom, I am in N-club and read your post. I will try to add my woes there as well. Our issues are not identical, but similar. | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
Thanks I appreciate that! I too believe there is something wrong with the ATM Ranker function. One other thing - I gave a friend the ATM Method I concocted and he couldn't load it up on his OT although I am using it on two of my computers with no problems - loading-wise, at least. I only hope that these threads will encourage Nirvana Support to take a look into the Ranker problem and suggest how one might take better advantage of this wonderful tool they have supplied us with. Tom Helget | ||
| jpb Veteran Posts: 168 Joined: 5/11/2005 Location: Brown Deer, WI User Profile |
What's the sequence of your actions after applying a change to your rankers? By this, I mean, are you just clicking Run Analysis in Port Sim or do you switch to real mode and run a full analysis or something else. And if you just click, Run Analysis, do you run it just once or twice between changes to the ranker? Reason I ask is I've had the equity curve and many of the statistics change between the first and second run when I made no change in between. Usually after the second run, the stats stay consistent. I've sent info to Barry along with strategies, profiles, ATM methods, and VBA files. I don't think he's been able to reproduce on his system yet after multiple examples. We keep working at it. It would be helpful if you could post the files and the process/actions you do so we could try and reproduce. One consistently reproducible example helps significantly if we can get it on a Nirvana desktop. I personally haven't seen the sort order fail with my testing and have seen results change when I flip the order. But I haven't run one of those tests recently in my methods. | ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
And in every case, I didn't have to leave the Port Sim page to see my changes process properly. So, I am trying to document as I play and I am now trying to break the thing and hopefully figure out what causes the trouble I outlined in previous posts. I tried using a file compare app to see if there were notable differences in the ITS files I was having the discrepancies in prior and did not see any red flags. Will probably pick this up over the weekend to test some more. I've been archiving off copies of the files as I test so that when things misbehave, I can compare the two ITS file versions for any discrepancies. | ||
| EYEGUY Icon Posts: 1543 Joined: 12/12/2003 Location: BALDWINSVILLE, NEW YORK User Profile |
Nice, scientific approach. Tom Helget | ||
| SJ Member Posts: 49 Joined: 9/28/2012 Location: Olathe, KS User Profile |
In diagnosing the problem, I suspect I had an error in my initial work in OT 2018, but I finally came to the conclusion there is a real issue in OT 2019. When you create a method (by New or Save As) a filename.ITS.txt (ITS) file is created. It is in XML format and contains all the parameter settings for the method. Here is the ATM\ folder. The “Test ascend (descend)” and “Test 2018” files (4) were created in OT2018. The “Test 2019” files (2) were created in OT 2019. I brought all files over to OT 2019 Compiler status in Omnilanguage window after a forced compile. The 4 methods ported over from OT 2018 are compiling. None of the 2019 created methods are compiling. After compilation, a folder is created with OL style code that represents the filters and rankers within the method. Here is the VBA\ITS\ folder in OT 2019. Note there are folders for methods from ported from OT 2018, none for the two created in OT 2019. Within these ITS folders are a folder for Filters (if any) and Ranks (if any). I was only playing with rankers, so I had no Filters folder. After compiling, subRanker and filter files are generated. For ranking, these represent the individual ranking functions. After you Run Analysis, OT determines the highest and lowest values for each ranking function and does two things. First, it creates a long and short Ranker file in the Ranks folder. These represent the composite ranking function inclusive of weights and direction. Without them, no ranking takes place. Second, after Run Analysis completes, the ITS file in the ATM folder (or ATS folder if you don’t have ATM) is modified with this information. Below is a comparison of the ITS files after they are compiled and run. The one on the left is on OT2018 and the difference is the subRanker min and max values are calculated. In the one on the right, OT2019 refused to calculate the values. Without those subRanker values, a composite Ranker formula is not generated and no ranking takes place. With that knowledge, I processed a method in OT2018 and ran it (Run Analysis) so that the ITS file was modified with ranking min/max information and then copied it into OT2019. When OT 2019 is launched, it seems to properly recognize it, does compile and does rank trades. I even modified this ported over method within OT 2019 by adding a second ranker function and it appears to properly integrate that second ranker. I verified the OT 2019 behavior on a second computer. One of the two is updated to 2L. So, specifically what I found after starting over was the following: - Methods created and or modified in OT 2018 are working (I can’t explain what was going wrong initially in my original post.) - Methods created in OT 2019 are not working for Ranker functions. - Methods created and executed in OT 2018 do work if copied to OT 2019. - Subsequent modifications to methods in OT 2019 appear to work. As a side note, I raised some thoughts on how the ranking function works in one of my earlier posts on this topic. I've expanded those thoughts, especially with respect to use of multiple rankers and posted them here. [Edited by SJ on 3/3/2019 9:03 PM] Attached file : ATM_Folder.png (27KB - 540 downloads) Attached file : Build_Window.png (7KB - 538 downloads) Attached file : ITS_Folder.png (15KB - 515 downloads) Attached file : Ranks_Folder.png (15KB - 553 downloads) Attached file : File_Compare.png (51KB - 532 downloads) | ||
| Vinay Elite Posts: 640 Joined: 12/9/2011 Location: Planet Earth User Profile |
The change-log for PR-2I mentions that "(ATM) Using Trade Rankings no longer requires entering formulas for both Long & Short conditions." Please clarify whether the Trade Ranking issue in ATS as reported in this post is also fixed? |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
