Machine Learning for Stock Price Forecasting (1/3)
If you know me, I rarely talk outside of two things: travelling and startups. I follow quite a few travel blogs and Anthony Bourdain was one of them. His lessons remind me how small I am in a big world and how much further I have to go. Just thought I’d share 4 quotes from Bourdain that help define my ‘why’. A little late to this post, but here it goes...
Emphasize the moment
“Your body is not a temple: It’s an amusement park. Enjoy the ride.”
Live in the present as much as you plan for the future. Always seek out experiences that define who you are. At the end of it all, what’s stopping us from expressing our lives as a polynomial equations. As nerdy as it sounds, it’s a million times better than living in robot mode.
"If I'm an advocate for anything, it's to move. As far as you can, as much as you can. Across the ocean, or simply across the river. Walk in someone else's shoes or at least eat their food. It's a plus for everybody."
My curiosity drives me to challenge myself physically and mentally. I’ve learned too soon the price of not taking advantage of opportunity. I remember a night I camped at the bottom of the Colca Canyon in a small village only to learn that in the midst of all the noise, there’s happiness to be found in a simple life that is worth living.
'New' should be the norm
"Maybe that's enlightenment enough: to know that there is no final resting place of the mind; no moment of smug clarity. Perhaps wisdom... is realizing how small I am, and unwise, and how far I have yet to go."
In 2017, I spent 4 months on exchange living in France and travelling around Europe. After visiting close to 24 cities, every one of them had something valuable to teach me. There’s a whole world out there filled with people who think similarly and differently than all of us. To travel and meet them is so worth it.
"Travel isn’t always pretty. It isn’t always comfortable. Sometimes it hurts, it even breaks your heart. But that’s OK. The journey changes you; it should change you... You take something with you. Hopefully, you leave something good behind."
What motivates me about building companies is the ability to add value from day 0. When you’re starting a new venture you’re always putting yourself in uncomfortable situations. Take the smallest things you do in your life, and break habits, you might discover a nugget or two.
Follow my travel journey on Instagram and my startup journey on LinkedIn.
Machine Learning for Stock Price Forecasting (1/3)
This post will take you inside the works of a 4 month project on developing a machine learning algorithm for stock predictions under the supervision of Schulich Professor Zhepong (Lionel) Li. This is part 1 of a 3 part series. All code is written in python. Data science and model construct performed using Scikit-learn, numpy, and pandas packages.
- Part 1 - Overview: Using Machine Learning to make data-driven decisions
- Part 2 - The Math: Applying Supervised Machine Learning (code attachment here)
- Part 3 - The Finance: Inferences in Stock Behavior
Why I did this?
Since my late high school years, I’ve reaped a deep interest in financial markets. I’ve tried many different strategies and put a lot of thought trying to come up with an adequate strategy to consistently make money by investing in the stock market. I’m a believer that over the short term (under 1 year) stock prices move in wave patterns – understanding these, can help us understand stock price movements.
The efficient market hypothesis tells us that all relevant information is already factored into a stock’s price, meaning that neither fundamental or technical analysis can be used to achieve superior gains in the short and long-term. Lots of research has already gone into figuring out how stock prices move like here and here. This experiment challenges this. My hypothesis is simple: By mining for patterns in data using supervised machine learning techniques, I can construct a model and trading strategy that beats the market. After all I am only following the plethora of algo and speculative traders that continue to exploit the market in the short term. This has always been a passion topic for me, so here goes nothing…
The training data used in my project was collected from Quandl Database. I have used one of my favorite stocks, Amazon (NYSE: AMZN) to model the algorithm. The data contains daily stock information from 12/1/2000 - current (because I'm on the free plan, the data is always, at least, one week behind). I’ve used 80% of the data for training and 20% for testing.
In essence, I will apply binary classification to produce a bull or bear signal at time t (where t = 1,2,3...t). To make things short, I used daily labeling as follows: label “1” if the next day closing price is higher than that of the previous day, otherwise label “0”. Finally, I will apply this sequence to multiple models to evaluate performance. The most important metric for measuring performance is accuracy which I have defined below.
Accuracy = Number of days model correctly classified the testing data / Total number of testing days
It’s easy to observe that some models are steadier than others. Keep in mind, only 9 features were used to construct this model, and adding more information-rich features is one way to cancel out the noise and separate the good models from the bad.
The longer the forecasted horizon, the more accurate our predictions become. It’s to no surprise that next day predictions are not much better than the odds of correctly predicting a coin toss (observed by tracking the null accuracy). The Null accuracy is inversely related to the forecast period. The spread between the null model and the other models help describe how efficient each model is.
Professional quant traders on Wall Street and Bay Street achieve up to 55% accuracy in predicting next day stock prices, and up to 80% accuracy in predicting stock prices 30-days out. The addition of more features can help me filter the noise to reach these levels. For the time being, I have devised a trading strategy based on the current model to analyze how well I would do in the stock. Since the SVC had the highest accuracy levels, I’ve used it to model this strategy. The model will tell me to sell the stock if next day prices are going to decrease, buy if next day prices will appreciate. My model was able to produce an average ROI for 1.3% per month from 2008-2010, equal to 31.2%. Similarily a buy and hold strategy in the S&P500 during the same timeframe would have resulted in -6.9% in the same timeframe (recession) and Amazon's stock would have performed 152%. My model does not produce the best results, but I do believe it’s a strong start. Stock prices during this time frame were greatly influenced by macroeconomic factors which my model does not factor in. Better returns will come with more diverse and information-rich features.
Challenges and Shortcomings
- As stock prices behave differently, stock prediction accuracy will also vary. As much as you challenge it, some datasets are better to work with than others.
- The current model is narrow is scope and does not factor in implications of macroeconomic indicators such as Real GDP, inflation, or interest rates which can have drastic effects on some stocks. 9 features are certainly not a representative of a stock’s price (but it did allow us to draw some conclusions).
- Special events like quarterly earnings and not accounted for. The use of NLP to measure analyst report sentiment can be used to provide an indication on the stock’s direction.
* * *
My model only brings us a step closer to understanding stock price behavior. By improving the model we can achieve superior accuracy levels, invest more confidently, all backed by data-driven insight.
If you’re a financier who is passionate about financial markets or a tech talker/data diver/machine learning maniac/#whateveryoucallyourself, feel free to get in touch. The opportunity to collaborate on advancing the model further can get us a step closer to reaching true alpha in understanding the market.
Connect with me on linkedin/follow me on medium/fork me on github
"So who do you bank with"
How the value prop narrative of banks is changing
Financial services is evolving. The industry is catching on the trend of relying on technology to deliver quality customer experiences fueled by more efficient back-office environments after threats from new market entrants, known as FinTech startups, who offer crisp value prop narratives that compete with legacy bank offerings. As a result, banks are no longer monopolies; the supply chain has evolved into a much more complicated macro structure, consumer demographics have changed as the focus shifts to the millennial market, and designing friction-free user experiences have become the new gateway to innovation. In the large scope of things, the underlying value proposition of a bank is changing to one that is perhaps less compelling to the banks, but more inclusive and meaningful to the consumer. This post presents my latest thinking on the future role of the bank in an evolving digital world.
When Finance Meets Technology
It takes 2–3 days for your account balance to settle a transaction, 5–10 days to send money overseas, and 15–30 days for a bank to approve or decline your mortgage application. Living in 2017, you may wonder why these transactions aren’t yet instant.
While technology has developed in speed, accuracy, and capacity, banks have yet to catch up to emerging technology trends. Part of this has to do with the regulatory environment they must operate within, but also, with the way their IT systems are designed. In more technical terms, it’s the comparison of a bank’s mainframe computing system versus a FinTech startup’s cloud infrastructure — it’s about close-looped (current bank) vs open source architecture (future bank). Today, only one large scale initiative by the name of PSD2, has been launched to transition, suggesting that we are in the very early innings of this major transformation, but I do believe that if successful, it’s an example the world will learn from. PSD2 isthe first regulatory directive of its kind lead by the European Commission for the purpose of scaling bank architecture to something that is more meaningful, powerful, and inclusive to all stakeholders.
The open source banking movement will fundamentally change how we think about the banking ecosystem, all the way from customer interaction to the economics behind back-office operations. Banks are not going anywhere anytime soon; they will still power the same financial products we use everyday, but the delivery and implementation strategies of those products will change. Open source architecture will lower the barrier to entry, encourage innovation, and motivate entrepreneurs to get involved. Being the backbone of Financial Services, banks will focus their attention on what they do best and act as an infrastructure layer - and the data they have on consumers will serve as the biggest driver. As their attention shifts to this new role, they will lose sight of direct interaction with the consumer, causing them to lose their client-facing brand. In the end, banks will be data companies that just happen to be in the banking business. And from there, the global perception of what banking is and which players are involved will change.
Drivers of the Open Bank Environment
Banks as infrastructure companies will only make up 1/3 of the future open banking ecosystem. The relationship dynamics within the supply chain must have some sort of merit, and consumers still need a way to interact. Here is how all three come together:
- Infrastructure: The new bank infrastructure will be built on open source and widely decentralized technologies. Today, Blockchain is the best example. It will allow banks to redesign each of their main functions as a decentralized application (known as Dapps to the blockchain community) — this includes capital markets, mortgages, insurance, payments, etc. Already, Ripple and Nuco are two fintechs leading this space.
- Supplier-side: As the supply chain evolves, strategic partnerships will be formed to offer the user a friction-free experience. Great communication protocols must be in place to reduce friction down the supply chain. The concept of open source application programming interfaces (APIs) have been around for a long time, but never in the banking industry for many reasons beyond this post. The future will involve APIs as the main form of communication between different players in the banking supply chain and will be the foundation for building great IT communication protocols for the banking environment. For more information, Token is a great example capitalizing on the PSD2 directive in Europe.
- Client-Side: Chatbot powered AI will be one of the most powerful ways for consumers to interact with the banking environment. Today, most chatbots are built on social media platforms like messenger, slack and sms, and so, it leads us to think that banking will become more social and also user-friendly. Social media won’t be the only way to reach consumers, but it does represent a large identity gap in the making. The future of client-side banking will be social banking. Finn and TalkBank are great examples who have already established strategic partnerships with banks.
So much will change in the next five years alone, and as a consumer, you may start to forget who you bank with. Not because you want to, or because you will be unbanked, but because you will be interacting with the banking environment in a whole new way. You’ll be checking your bank balance and paying your bills through social media-enabled chatbots (Kasisto and TalkBank are messenger-enabled bots) or sending money to family and friends using third party loyalty-enabled digital wallets (Venmo or Circle). Before you know it, you might just be lucky enough to bank with Facebook, Amazon, or Google. And greatest of all, it won’t be because they’re regulated to manage your chequing or savings account, but because the open source banking environment will de-regulate the industry providing outsiders with enough incentive to get involved, as it's already started...
Banks have always been and will remain as the foundation for global capitalism, but the next time you’re asked, “Who do you bank with?” you may have very good reason to say, “I bank with Facebook.”
UX should be taught in B-School
Successful startups and corporate conglomerates all have something in common: they design the product around the consumer. What does this even mean? Well, to start off, it’s easier said than done. Think of the 90% of startups that fail within their first year of launch, or the multi-millions a year wasted on failed product launches or merger deals by Microsoft, Google, or Wal-Mart.
User experience has grown in popularity among the community of business developers, technologists, and designers. Design thinking is taking the center stage for companies who want to reap big rewards out of the products they offer. This doesn’t just mean creating beautiful images, aesthetically pleasing art-boards, or awesome-looking products, but it’s really designing frictionless user experiences that customers love and appreciate — now that’s a challenge.
“Design is how it works” — Steve Jobs
So let’s cut right to it: business strategy is design at work. A huge part of product innovation comes from designing good experiences that will attract and engage users. Here, business schools have failed to equip its future leaders with the tools necessary to innovate with a design-centric methodology — let’s call it the new way of product innovation. Since business developers play a huge role in product teams, they need to be well-equipped to speak the language of design and UX in order to succeed in their roles.
Every industry has been disrupted in some way by technology. Business strategy (especially to the tech savvy firm) is constantly changing courses. Traditional models are no longer enough to remain competitive — creativity is becoming a strategic lever, UX design is taking over traditional hypothesis testing (data-driven testing), and design is becoming a gateway for better innovation. Since business strategy to the digital firm is changing, so should the way it is learnt. The curriculum needs to be strategic in how it is structured to meet the evolving trend of companies who value design as part of their commitment to constant innovation. According to the 2016 DesignInTech Report, all of the top 10 U.S business schools have design clubs led by students. Design-thinking teaches us to think differently, while prioritizing certain stakeholders that business thinking doesn’t focus on.
We live our lives telling our story our own way
It really has been a wild past four months. Travelling across Europe, studying in France, and meeting extraordinary and ambitious people — both young and old. We live by telling our own stories, and tell it differently by the way we act, and behave. We should learn to embrace different cultures, be brave enough to take risks, and live in the moment just as much as we plan for the future. No two stories are ever the same, so here’s a little bit about mine.
From when I was a little boy, I’ve always been extremely ambitious and free-spirited (more than my other two siblings by a mile), so travelling the world exotically had always been at the top of my to do’s. I remember the day I told mama of my study abroad acceptance, and how happy I was (not because I was leaving home) but to be given the opportunity to explore, relive, and embrace change. To give you some perspective, I’ve never crossed borders without my parents, never lived away from home, and never set foot on European soil before — so yes I was worried — but that only made the hunger in me yearn for more.
Having visited nine countries and over twenty cities, I never felt as if I was away from home. After all I was just six thousand miles away from where I started. I won’t sit here and tell you it was all sunshine and rainbows, because it wasn’t. I got my fair share of ups and downs, but a truly adventurous trip can never go exactly as planned. It had been about cultivating my identity as a young, ambitious, and curious wanderer. The last four months have been great to me, and every city I visited had something valuable to teach me.
* * *
I would love to share some of the hundreds of adventures from my trip, but that would also have me writing a novel. In the meantime, I also wanted this to be my opening story to a series of other stories. With an interest in code, UX, entrepreneurship, and fintech, I’ll be blogging around these subjects, and only hope to entertain you in the process. If hearing any of these words excites you as much as it makes me jump on the couch like a two-year-old girl when she sees a piñata, then I’m sure we’ll meet again.