Why I'm Learning Neural Networks in 2026

Why start learning neural networks in 2026 when Codex or Claude Code can write the code for one?

I've been interested in neural networks throughout my software engineering career. I took machine learning courses on Coursera, including courses by Andrew Ng. But I kept returning to the skills that seemed more immediately useful for my work: software architecture, databases, and building applications.

Then LLMs made it much easier to build AI-powered products. I started integrating them into applications and building AI agents. Learning how the models themselves worked moved into the background.

Now I'm returning to it. I want to understand the technology behind the agents I build, and eventually build a small language model of my own.

I want to understand the models behind my agents

When I build an AI agent, I work with prompts, tools, evaluation, and application infrastructure. I can get useful results without knowing every detail of the model underneath.

But I want a better explanation of why a particular approach works and where its limitations come from. With a database, understanding its internals helps me reason about the application around it. I want to develop more of that understanding for neural networks and language models too.

Coding tools are also changing how I think about learning. They can handle more of the implementation work, but I still need to understand the systems I'm asking them to build. For me, that makes learning the underlying technology worth the effort.

I don't know exactly how this will affect my career. I'm treating it as a practical learning project, rather than a prediction about which engineering jobs will disappear.

My goal is to build a small LLM

Building language models used to feel like work that was far beyond my reach. Seeing other engineers build them made me question that assumption.

My goal is modest: learn how to build and train neural networks, then work toward a small language model that I can understand and explain. I'm not trying to compete with a frontier-model lab. Hardware and training costs are real constraints, and I can start at a much smaller scale.

I also want to gain experience integrating neural networks into applications. I've built software and infrastructure around AI systems; now I want more hands-on practice with the models themselves.

Starting with micrograd and makemore

I'm following Andrej Karpathy's Neural Networks: Zero to Hero.

At the time of this recording, I'd watched the first two lessons: micrograd and the first makemore video. My next step is to work through the makemore exercises.

The biggest insight so far came from micrograd. Derivatives had mostly been formulas I used to solve exercises at school and university. Working through the code connected that mathematics to something concrete: how changes propagate through a computation and how gradients are used during training.

I had studied some of these ideas before, but implementing them made them easier to reason about. There's still a lot for me to learn.

Sharing the learning process

I'll share my exercises and code in my neural-networks-zero-to-hero repository, along with videos about what I'm learning.

The target is to understand neural networks well enough to build them, experiment with them, and eventually build that small LLM. This video is the beginning of that process, not a finished model or a complete neural-network tutorial.

Watch the video

Watch Why I'm Learning Neural Networks in 2026 on YouTube