There are plenty of shops making tons of money with HFT who do not have deal flow at all - it's got nothing to do with luck.
Survivorship bias would mean I simply got lucky. If you're going to say that you're at least going to need to look at my P&L charts and say how I could possibly achieve that much success with luck alone.
Finally, machine learning has everything to do with my success. There were hundreds of variables in my algorithm that were ALL optimized using ML. If you read the article you would know that I built an accurate model for backtesting that I used to optimize variables as well as confirm that I was going to make money before I even started live trading.
I'm pretty confident that whatever you were doing in 2004 has nothing to do with what I was doing.
agreed. jspaulding got it right although I can see nashequilibrium's skepticism as the US futures market is incredibly crowded (read: competitive, no free cookies).
It's easy to fall into that mindset. And in fact, that mindset is right back where I am now. The only reason I had the gall to attempt this in the first place was the the simple fact that I was making money at the time (in 2008) 'manually' day trading the Russell 2000. I thought this 'should not be possible' so I figured there's no reason not to try an automated program.
People will tell you that you were just a lucky monkey. But you could have run your algorithm on past data, for hundreds or thousands of fake portfolios, to tell, statistically, what the odds of your algorithm being simply lucky are.
In early 2000s I wrote a machine learning algorithm that beat the S&P 100 with over 1 trillion to 1 odds against it being luck. It predicted a full trading day in advance. But that was all on paper at trading firms' puny costs; unlike you I couldn't beat retail costs. It's amazing that you could do that. For that reason alone I think it's highly likely that you were a skilled monkey.
Also like you, nobody in the industry was interested in my code, even after an industry magazine watched it for 3 months and found it gave "stellar" performance. The few people I was able to discuss it with told me point blank that it was impossible to do it skillfully (efficient market theory), so they assumed it was a hoax or the algorithm was just lucky.
The code sits in one of my archive folders. I ran it for a few years, perhaps to 2004, and saw the market steadily becoming more efficient, lowering my results (like the OP did). It may well be that it no longer predicts skillfully or profitably. As I recall, to beat the market the costs had to be very low, like pennies per trade, with no bid/ask spread, which I understood to be possible for large trading firms.
OK, cool. There are some places that offer equity trading for ~$0.005/share, but that says nothing about overcoming bid/ask. Looks like the OP did that by throwing a bit of market making into the mix.
I don't know the exact definition of HFT but I did run my algorithm from a server collocated with my broker close to the exchange. I modelled lag time in simulation and not having it collocated certainly would have hurt.
Thank you. I'm looking back through my code and there are really a lot of indicators. It would get pretty technical to explain them. They are all explainable it's just that each one corresponds to slightly different market conditions and I just didn't want to get into it.
Fair enough. Most people considering trying this probably have a few ideas for indicators.
But for anyone coming to HFT from a coding background instead of a trading background, an explanation of one of your indicators would have been fascinating.
I basically just brute forced it. I came up with a cost function which would measure the difference between a possible curve and each data point. I think you're supposed to do the squared difference but I can't remember if I did that. With a cost function in place it's just a matter of zooming in on variables that minimize the cost function.
CourseTalk. CourseTalk! http://coursetalk.org .. You are right kind of :) But I've made a decision to start reaching out generally so I can attract cool people to work with on whatever projects I may be interested in in the future.
By the way, no offense meant by the advertising thing. I enjoyed the article. Although, as a technical person, would've enjoyed more details on the code and algorithms.
Survivorship bias would mean I simply got lucky. If you're going to say that you're at least going to need to look at my P&L charts and say how I could possibly achieve that much success with luck alone.
Finally, machine learning has everything to do with my success. There were hundreds of variables in my algorithm that were ALL optimized using ML. If you read the article you would know that I built an accurate model for backtesting that I used to optimize variables as well as confirm that I was going to make money before I even started live trading.
I'm pretty confident that whatever you were doing in 2004 has nothing to do with what I was doing.