Audio Version on YouTube: https://youtu.be/QVTdepLy12Y
I can't help but feel like we're stuck in a loop of human folly again, as history has shown us.
Humans have this tendency to think they've got it all figured out. Even way back in the dark ages, they fell into this trap, firmly believing they had all the answers about the world, like insisting the Earth was flat as a pancake.
During the time of Isaac Newton, scientists were pretty convinced they had cracked the code of physics, certain there was nothing more to uncover. But then along came Einstein, turning the whole game upside down. This pattern continued with discoveries such as quantum mechanics, where folks like Bohr and Heisenberg shook up the status quo, altering the course of science. And guess what? Niels Bohr did the same thing Einstein did when his theories were questioned.
Now, it's hard not to see history repeating itself with AI. It feels like we're stuck in a never-ending loop of human folly.
Pardon my French, but I always remind myself, "You don't know sh*t." That's how I keep learning and growing. The moment you think you know something well enough, you stop learning. That's when you get trapped in the loop of human folly. 🤗
A Path to Perfection - AI vs. Human https://github.com/neobundy/Deep-Dive-Into-AI-With-MLX-PyTorch/blob/master/essays/AI/A-Path-to-Perfection-AI-vs-Human.md
Here are my two cents on datasets for training, especially for fine-tuning, as exemplified by the Apple MLX LoRa example.
This concern might have prompted some of the godfathers of AI to worry about the field. The inner workings of deep neural networks remain somewhat of a mystery. We simply don't know how they really work, at least not yet. Perhaps we never will.
When we dismiss certain datasets because we presume they may not meet our standards without actually testing them, we risk missing something important. The model might uncover insights that aren't apparent to us.
You might not see any patterns, dependencies, or correlations in the dataset, but the model might. That's precisely why we need AI in the first place. It can see things we can't — things we don't even know exist. That prompts us to ask, "How did it know that?" We don't know. We can't know. We can only guess. This is what excites some and worries others about AI.
My MLX docstrings example serves as a case in point. I had my reservations as well. On the surface, it seemed unlikely that the model would gain any knowledge from it, as all the rows were independent, and there were no discernible patterns to the human eye. However, it did learn something, although not perfectly or even very well. This learning could be attributed to fine-tuning on top of its existing knowledge.
The same approach with docstrings and MLX code examples (along with their PyTorch equivalents) has proven effective with custom GPT-4, as I have previously posted. What seemed to be a silly idea for a dataset turned out to be a good one, prompting me to explore further and discover something new.
Let's not rule out any possibilities and instead begin with the assumption that we might not know everything about AI. This embodies the spirit of experimentation, doesn't it?
Moreover, as stated in my essay "A Path to Perfection - AI vs. Human", AI models are made by us to learn from mistakes. So why can't we do the same? I always wonder.