Submissions from idlemachines.co.uk

		Adam and AdamW: adaptive optimisation and weight decay (idlemachines.co.uk)
		1 point by smaddrellmander 2 days ago \| past \| discuss
		Thinking Machines dropped RoPE, and it's a good idea (idlemachines.co.uk)
		6 points by smaddrellmander 11 days ago \| past \| discuss
		The annotated PyTorch training loop (idlemachines.co.uk)
		81 points by smaddrellmander 41 days ago \| past \| 19 comments
		Mae vs. MSE: more than just the mean vs. median debate (idlemachines.co.uk)
		1 point by smaddrellmander 45 days ago \| past
		DiffusionGemma: Discrete diffusion in a large language model (idlemachines.co.uk)
		3 points by smaddrellmander 52 days ago \| past \| 1 comment
		Heaven knows I'm perplexed now (idlemachines.co.uk)
		2 points by smaddrellmander 57 days ago \| past
		Reading MAI's efficiency gain. How to pick architectures like serious people (idlemachines.co.uk)
		9 points by smaddrellmander 60 days ago \| past
		Are contrastive losses just cross entropy all along? (idlemachines.co.uk)
		2 points by smaddrellmander 62 days ago \| past
		Every token, everywhere, all at once (idlemachines.co.uk)
		2 points by smaddrellmander 69 days ago \| past
		The cut in the Mixture of Experts compute graph (idlemachines.co.uk)
		1 point by smaddrellmander 76 days ago \| past
		DeepSeek V4 from the Inside (idlemachines.co.uk)
		2 points by smaddrellmander 83 days ago \| past
		Softmax, can you derive the Jacobian? And should you care? (idlemachines.co.uk)
		134 points by smaddrellmander 3 months ago \| past \| 47 comments
		Gemma 4 is not your standard transformer (idlemachines.co.uk)
		2 points by smaddrellmander 3 months ago \| past

HN For You