OmniTrader Professional Forum
OmniTrader Pro General Discussion
ATM3 Early Results
50 replies, 4006 viewings
|
OmniTrader Professional Forum
OmniTrader Pro General Discussion
ATM3 Early Results
|
Last Activity 2/23/2025 4:01 AM 50 replies, 4006 viewings |
|
|
| Printer friendly version |
mholstius Regular  Posts: 76 Joined: 6/15/2011  User Profile |
We all face the question of what is the most useful “target” / goal to give us an edge? Historically, we’ve used HR, PPT, CAR, CALMAR, etc. – with the inherent problem that each can be quite specific and have their own weaknesses. So, for about a year or two I’ve worked on developing a formula that would evaluate a number of different parameters and assign a single number that would correlate with the overall equity performance in the past. I hope I’ve come up with something useful, and Ed’s dedicated a lot of time and resources this year to incorporate it into the updated ATM3. This post is an attempt to show a bit of what it does. I’ve run numerous tests to confirm that the number derived from the formula accurately reflects past performance. One of my more extensive tests involved 500 symbols, and the following snags show the equity curves in the order of the ranking number produced by the formula… The equity curves produced when charting the top 3 of 500; Ranked #50, 100, and 200; Ranked #300, 400, and 500; The theory is that given a choice, we’d rather trade something tomorrow that had a high rank number vs a low one… As Ed said, moving forward on a monthly basis using excel (even with 1M cells) just wasn’t possible – but ATM3 gives us that capability. ATM 3’s only been out for the past week or so, but I’ve been able to run some tests that appear to validate the theory. The formula (rank “Mark II” in ATM3) uses the backtest period to generate the ranking number, and ATM3 uses that number to choose the symbols to trade the following month - calculating a new number each month and then walking forward the following month. I have a very diversified list of 365 symbols that I use for development work. I knew that VBX-3 hadn’t performed well with this list over the past few years, so I figured it would be a good test. I set up ATM3 to trade the top 15% of the ranked symbols (55 of 365), walking forward each month from 1/1/2017 to the present using 10% allocation / trade. It produced the significantly better results you can see in the snag below…. (ATM3 curve in green) Final equity rose from $90,000 to $123,000 Avg Ann MDD fell from 25.3% to 4.6% # of trades fell from 788 to 250 And the Avg % Invested fell from 40% to 12% (considerably less risk) The better, stable, and robust performance got me thinking… What if we allocated more to each trade to more efficiently utilize equity, rather than letting it sit on the sidelines? So… I ran the test again, adding 20% (blue line) and 30% (now the green line) allocations; The performance improvements are impressive, and the equity curves are very similar – they don’t improve just because a particular allocation cherry picked better trades. With 30% allocation; Final equity rises from $90,000 to $161,000 Avg Ann MDD falls from 25.3% to 7.4% # of trades decreases from 788 to 186 And Avg % Invested falls from 40% to 22.8% (again, less risk) While we might not trade at those allocations, it’s a concrete example of what’s possible with ATM3. Just take a minute to reflect on what this demonstrated… I gave ATM3 a big list – and it not only chose the symbols that traded the best with VBX-3 in the past, it repeated that process every month. And then, more importantly, it charted the walk forward, out of sample results of what would’ve happened if I’d been able to use ATM3 in the past. We’ve never had that before – and it’s simply incredible! There are currently 7 different ranking algorithms in ATM3: CAR, HR, Consistency, Stability, Smooth, Mark I, and Mark II. No surprise, I tried “Mark II” for this run - but I’m certainly going to try the others too. Maybe one of them will work even better…??? This was a limited test to simply see if ATM3 could find the best symbols to trade with VBX-3. It can also find the best Market State & Strategy pairings, and even combine those with the Symbol & Strategy pairing capability this test demonstrated. So many outstanding possibilities… I haven’t had time for all that yet – I just wanted to share these early results since they’re a concrete example of what ATM3 can do. Imagine what’ll happen when we pair the improved ARM6 AI capabilities with ATM3…. That thought puts a smile on my face. Mark [Edited by mholstius on 1/22/2020 6:42 AM]  Attached file : 01 Top 3.png (104KB - 2009 downloads) Attached file : 02 Middle 3.png (108KB - 1935 downloads) Attached file : 03 Bottom 3.png (116KB - 1938 downloads) Attached file : 04 VBX3 ATM3 Results.png (174KB - 1899 downloads) Attached file : 05 Higher Allocation vs Risk.png (241KB - 1888 downloads) | |||
| Jeff Brown Member Posts: 31 Joined: 1/29/2008 User Profile |
I have just a few questions and a request: 1) When you ran the test, did you have leverage defined? It looks like up to 300% (3x)? 2) Did you have Port Sim set to "Increase Number of Trades" or just the "Increase Trade Size"? 3) Where did you come up with such an awesome ranker? I'd love to hear more about your thinking process and what lead you to it. With results like you've shown, you've got me wanting to test a ton of strategies that haven't performed in the past and see what ATM3 can do with them just by giving them symbols that behave well for how it works. I've already got 2 VMs running since Friday night when I installed ATM3 -- both AI strategies and a Method I'm using in live trading to see how much better it could have been. But I didn't touch the ranker. Now I want to get another test running... | |||
| mholstius Regular Posts: 76 Joined: 6/15/2011 User Profile |
Glad to hear you’re experimenting with ATM3. Hope you’ll share what you find. I think ATM3 will “rank” right on up there as one of the premier pieces of software from Nirvana – and that’s saying a lot. The tests I posted above all used 2X leverage and “Increase Trade Size”. The only difference between the black, blue, and green plots is the allocation of equity / trade. Black used 10% / trade, blue used 20% / trade, and green used 30% / trade. ATM3 was set to trade using VBX-3 Long and the symbols that had Mark II ranking values in the top 15% as of end of the previous month; I’ve been working on the algorithm / formula for a couple of years now, trying to boil it down to one number that can be used to rank the “quality” of the equity curve in the past. It took a lot of work on the part of the Nirvana developers to incorporate it into ATM3 – and it’s a testament to their skills that it runs as fast as it does. (I just checked, and I’ve created 16,000 spreadsheets on this machine – 3,000 of them since 1/1/2018. That’s probably not a good thing… sigh.) Good luck in your testing – we have a lot to work with, and a lot to learn with 7 different ranking algorithms to choose from; Hope you enjoy the walk forward… Mark [Edited by mholstius on 1/22/2020 7:54 PM] Attached file : 01 ATM3 settings.png (53KB - 1823 downloads) Attached file : 02 ATM3 rankers.png (9KB - 1808 downloads) | |||
| BenQ Member Posts: 17 Joined: 8/15/2016 Location: Kent, WA User Profile |
| |||
| John W Regular Posts: 96 Joined: 6/18/2011 Location: Sydney, NSW, Australia User Profile |
| |||
| Keith Parsons Member Posts: 47 Joined: 6/5/2012 Location: Durban, RSA User Profile |
Really appreciated. | |||
| mholstius Regular Posts: 76 Joined: 6/15/2011 User Profile |
Just trying to share, since we can all move this forward a bit faster if we work together. There are a ton of new possibilities with ATM3 – if each of us shares something new we’ll all improve that much quicker. Ed was kind enough to share the latest build with me this afternoon, so I’d like to show what I was able to find in a few runs. I have a strategy that I was developing a year ago that wasn’t doing very well, so I’d “put it on a shelf” in 2018. I thought it would be an interesting challenge for ATM3, so I dug it up and ran it using the same 365 symbol list I used in my first post here. As I expected, it didn’t do very well – but ATM3 did a nice job of improving the results; I had time for another run before supper, so I did one that you can duplicate if you have ATM3… The list is simply the SP500 from Omniscan using X-MFI for the strategy with the following pretty generic settings in ATM3; No Market State (just C>0) Allocation 10% All Longs Symbol-Strategy using Top 30% Mark II Internal FT 30 Bars Monthly Cutoff of 0 Below is the result; The lower red plot is the normal 10% of Equity The blue line is the ATM3 result using allocation of 10% Since that result was only using 13% of equity (vs 51% for the “normal” run), I did it again and just increased the allocation (no other changes). The green line is the result after increasing the allocation to 20%. The improvements are quite obvious. I have to run for the evening but wanted to share this for all of you that now have ATM3 – and those that are still trying to decide. It’s definitely a winner... and did I forget to mention the walking forward part too? Mark [Edited by mholstius on 1/26/2020 8:04 PM] Attached file : 00 My Strategy.png (212KB - 1675 downloads) Attached file : 01 ATM3 SP500 Settings.png (210KB - 1675 downloads) Attached file : 02 X-MFI.png (244KB - 1664 downloads) | |||
| SteveL Veteran  Posts: 133 Joined: 2/15/2007 Location: Boulder, CO User Profile |
Thanks for sharing your thoughts, tests, etc. Early in the thread, you indicated you are running test with walk-forward testing starting in 2017. Does that apply to all the testing you are showing? Steve L. | |||
| mholstius Regular Posts: 76 Joined: 6/15/2011 User Profile |
Yup – it can be hard to read in the snags… They start on 1/1/17 and run for 3 years, except that the snag for “My Strategy” started on 1/1/16 for 4 years because I was doing something else when I grabbed that one. Also, I forgot to include some of the settings I used in OT and Port Sim for anyone that might want to replicate this – so here are those details… I’ve been concentrating on testing the ability of ATM3 to dynamically find the best symbols to use with strategies, and being able to verify the results of those automated choices walking forward using out of sample data is amazing. I haven’t begun to use the optimization capabilities or the ability to find the best strategies to use with Market States - choices also made using the BT period and validated by a walk forward in out of sample data. I thought ATM was a game changer when it came out, but this adds a whole new dimension. Mark [Edited by mholstius on 1/27/2020 7:30 AM] Attached file : 01 Settings.png (235KB - 1632 downloads) | |||
| Buffalo Elite  Posts: 603 Joined: 7/11/2007 Location: Braintree, MA User Profile |
You have any idea why in the auto trend ATM3 method there is no trade ranking used? Barry? | |||
| mholstius Regular Posts: 76 Joined: 6/15/2011 User Profile |
To be honest, I just haven’t had enough time to look at the Auto trend method, so I can’t comment. If there’s no trade ranking, that sure sounds like something that could be modified and improve the results. With ATM3, there are a lot of tools that can be used to advance things… so every little bit of collaboration (like your observation) can go a long way toward helping everyone become more profitable. Here’s hoping you can find a ranker that helps… Mark | |||
| Barry Cohen Idol  Posts: 2305 Joined: 1/1/1900 User Profile |
Jeff says, "The ATM3 automation is essentially ranking the combinations based on recent performance. While we could add another step and rank the trades based on an indicator, this particular method doesn't generate a large number of trades a day and we found the indicator ranking to be an unnecessary step." | |||
| Buffalo Elite Posts: 603 Joined: 7/11/2007 Location: Braintree, MA User Profile |
Adding in strats for more trades so I may need to add some trade ranking ATM3 can build the symbol-strat and MS-strat combos, then if still multiple trades available this maybe good to have. Anyone found a good trade ranker beyond VTY-PR against SPY? | |||
| james Member Posts: 32 Joined: 5/9/2006 Location: victoria bc User Profile |
How do I deal with this ? | |||
| Barry Cohen Idol Posts: 2305 Joined: 1/1/1900 User Profile |
| |||
| james Member Posts: 32 Joined: 5/9/2006 Location: victoria bc User Profile |
It appears that the files, even if sent individually are greater then 5,000k Is there another way of sending them ? James | |||
| mholstius Regular Posts: 76 Joined: 6/15/2011 User Profile |
I'm pretty much out of it with the flu at the moment, but oftentimes choosing a different Focus list in the Port Sim will help with the problem of not finding data: 'There were no orders obtained to run a simulation, try a different source' Mark [Edited by mholstius on 2/14/2020 10:33 AM] Attached file : list.png (53KB - 1149 downloads) | |||
| Barry Cohen Idol Posts: 2305 Joined: 1/1/1900 User Profile |
| |||
| james Member Posts: 32 Joined: 5/9/2006 Location: victoria bc User Profile |
Files attached James Attached file : ATM Auto Trend Copy MH.its.zip (1KB - 192 downloads) | |||
| james Member Posts: 32 Joined: 5/9/2006 Location: victoria bc User Profile |
Trying again to send profile Attached file : ATM Auto Trend Copy MH.zip (664KB - 215 downloads) | |||
| james Member Posts: 32 Joined: 5/9/2006 Location: victoria bc User Profile |
Sorry to hear about the flu bug. Just finished a case of one myself I tried your solution and no go Hope Barry is able to solve | |||
| Barry Cohen Idol Posts: 2305 Joined: 1/1/1900 User Profile |
In your method, for each market state, you have "Use These Strategies" enabled, but there are no strategies enabled. This is resulting in an error & I'll report it. But you can fix it by enabling at least one strategy in each market state or by selecting the "Use All Strategies in Current Profile" setting in each market state. | |||
| gbarber Member Posts: 28 Joined: 2/26/2018 Location: PEARLAND, TX User Profile |
So I want to use something else that does not give crazy large orders as equity grows. I am now trying the fixed size option in port sim. However, if I give it a fixed size of 10 or 20, it will issue orders varying from 1 up to 60 or so contracts. I am not sure why that is happening. For values larger than 10 or 20 it could have something to do with using signals from multiple market states. I would like to understand better what the port sim allocation methods do. The ATM manual is extremely brief on that topic. I have not found a port sim specific manual. Does anyone know of a manual that explains the methods like fixed size, NN trade size, Kelley criteria, and so on? If so, please point me to it. | |||
| Barry Cohen Idol Posts: 2305 Joined: 1/1/1900 User Profile |
The reason you're seeing varying sized contracts is due to this Blend Allocation setting. ATM Smart Calls was designed to use that setting, but you can certainly disable it if you like, & your Fixed Trade Size will work as you would expect. Attached file : Blend.jpg (36KB - 999 downloads) | |||
| gbarber Member Posts: 28 Joined: 2/26/2018 Location: PEARLAND, TX User Profile |
Yes I am trying some method allocation changes. There are various available. but I have not found documentation that describes what they do. I need to know that to make my experimentation meaningful. Where can I find that documentation? It is not obvious from reading the name of the allocation. For example when I look up Kelly criteria on the web it gives me the following: There are two basic components to the Kelly Criterion. The first is the win probability, or the probability that any given trade will return a positive amount. The second is the win/loss ratio. This ratio is the total positive trade amounts divided by the total negative trade amounts. These two factors are then put into Kelly's equation which is: K% = W - (1-W)/R where: K%=The Kelly percentage W=Winning probability R=Win/loss ratio But when selecting Kelly criteria in portsim it asks for the following: initial amount, multiplier and warmup trades. That has no recognizable relation to what the web says about Kelly criteria. So how do I know what those are and how they are used in our specific portsim? I need the documentation that describes those allocation methods. |
|
|
|
|
Administrator
Forum Moderator |
Registered User
Unregistered User |
 E-Mail this thread to a friend |
Toggle e-mail notification |
