Week 1159
It’s my first week unemployed!
Why are healthcare and retirement accounts so complicated?
I spent an entire day trying to understand HSA and 401k migrations and I still don’t have a good grasp of how it all works. I apparently need to send physical mail to migrate my HSA account? Who still uses physical mail???
--------
Other than that this week has been very chill. I hung out with my parents for a few days and spent time reading and catching up on the AI world.
I reread the Attention is All You Need paper to refresh myself on how transformers work. Another great resource is Lilian Weng’s Transformer Family V2 blog post; it provides a comprehensive overview of the latest transformer optimizations. I’ve also found ChatGPT to be an amazing tutor for explaining complex concepts.
One of my friends suggested that I look into some of the recent work by Stanford's Hazy Research Lab. Specifically, their work on increasing transformer context length is pretty interesting and relevant to LLM applications. We spent a day reading through some of their papers. There’s still a ton for us to dive into but my initial takeaways are:
- It’s fun to feel like a noob again, but I really do need to brush up on my linear algebra knowledge.
- LLM capabilities will continue to improve as we iterate on model and hardware improvements.
- As a product engineer, I’m likely not going to contribute to these types of improvements, but I think it’s important to have a pulse on the latest research and understand what is currently possible.
- Ultimately, the most important thing for me is to be competent in using these models and find good problems to apply them to.
--------
Justin and I were chatting about whether we are people who “hate to lose” or “love to win.” I’m very much in the “love to win” camp. I’ve never felt terribly bad about any losses or failures in my life. While they do hurt in the moment, I always look back on them as learning experiences and am very willing to share them with others. There’s a saying in the gambling community that you only lose when you quit. This is terrible advice for gambling, but I feel this mindset is very applicable to anything where success behaves on a power law curve. Things like career, friendships, and love, all behave like this if you graph them with volume as the x-axis and happiness as the y-axis. If we only need a few or even just one success to achieve our goals, then why let yourself get emotionally dragged down by the countless failures it will take to get there?
We also posted our goodbye tweets this week. [Justin][Michael]
--------
Content:
I read the first half of Wonder Boy by Angel Au-Yeung and David Jeans. It’s a biography about Tony Hsieh, best known for being the CEO of Zappos. I enjoyed the storytelling about his entrepreneurial exploits that started in his childhood and continued into his professional career. It was inspiring to learn about his success stories with LinkExchange, VentureFrogs, and Zappos. It then started talking about his downward spiral due to drugs and alcohol so I stopped reading. I might pick it up again in the future but it’s honestly quite sad to read about and not super interesting to me at this point in time.
I finished Part 2 of The Founders by Jimmy Soni. It’s astonishing to see how hectic things are behind the scenes, even at a very successful startup like PayPal. I read to the part about Elon getting ousted from his CEO position by an internal coup, which was funny because we just had the modern day version of this with Sam Altman and OpenAI. I can’t wait to see how the OpenAI situation plays out and I hope we get a book or documentary about what really happened.
I came across Jared Hecht’s blog while riding BART and just started binging through his posts. He founded GroupMe and Fundera and shares a ton of insightful advice on his blog. These were some of my favorites: