Neural Network

A neural network is a function built from layers of simple units, each computing a weighted sum of its inputs and passing the result through a non-linear activation. Stack a few of those layers and the composed function can approximate relationships far too tangled to write by hand; that is the entire mathematical premise. The biological name is a historical accident and actively misleading — nothing in a network models a synapse, and reasoning about them by analogy to brains produces worse intuitions than reasoning about them as differentiable functions with a great many knobs. Training is where the interesting part happens. You define a loss — a number saying how wrong the current output is — then use backpropagation to compute how each weight contributed to that error, and nudge every weight slightly in the direction that reduces it. Repeat over millions of examples and the network settles into weights that produce useful outputs. The non-linearity matters more than it looks: without it, any stack of layers collapses algebraically into a single linear transformation, and depth buys you nothing at all. Architecture is the choice of how those layers are wired, and it encodes an assumption about the data. Convolutional networks share weights across spatial positions because a cat is a cat wherever it sits in the frame. Recurrent networks pass a hidden state along a sequence because language arrives in order. Transformers replace that recurrence with attention, letting every position look directly at every other, which is what made training parallelisable and therefore made today's model scale possible. For a builder, the practical implications are mundane and worth internalising. A network's output is a number that has been shaped to correlate with the right answer, not a derivation of it — so it can be confidently wrong, it will reflect whatever bias is in the training data, and it degrades quietly on inputs unlike anything it saw. None of that is fixable by prompting more carefully; it is a property of the function, and it is why monitoring and evaluation are part of shipping a model rather than optional extras.