Giles' Blog

# Giles' Blog

> Giles' Blog is the personal technical blog for Giles Thomas, a software engineer
> and entrepreneur.  Current information about Giles Thomas can be found on the
> [about page](https://www.gilesthomas.com/about.md).

This page lists the 20 most recent posts, and then all categorised posts by their
category (many posts have multiple categories).

## Recent posts

* [Writing an LLM from scratch, part 20 -- starting training, and cross entropy loss](https://www.gilesthomas.com/2025/10/llm-from-scratch-20-starting-training-cross-entropy-loss.md) posted on 2025-10-02T22:10:00+00:00
* [How do LLMs work?](https://www.gilesthomas.com/2025/09/how-do-llms-work.md) posted on 2025-09-15T23:20:00+00:00
* [An addendum to 'the maths you need to start understanding LLMs'](https://www.gilesthomas.com/2025/09/maths-for-llms-addendum.md) posted on 2025-09-08T18:15:00+00:00
* [The maths you need to start understanding LLMs](https://www.gilesthomas.com/2025/09/maths-for-llms.md) posted on 2025-09-02T23:30:00+00:00
* [What AI chatbots are actually doing under the hood](https://www.gilesthomas.com/2025/08/what-ai-chatbots-are-doing-under-the-hood.md) posted on 2025-08-29T20:00:00+00:00
* [Writing an LLM from scratch, part 19 -- wrapping up Chapter 4](https://www.gilesthomas.com/2025/08/llm-from-scratch-19-wrapping-up-chapter-4.md) posted on 2025-08-29T17:00:00+00:00
* [Writing an LLM from scratch, part 18 -- residuals, shortcut connections, and the Talmud](https://www.gilesthomas.com/2025/08/llm-from-scratch-18-residuals-shortcut-connections-and-the-talmud.md) posted on 2025-08-18T20:20:00+00:00
* [The fixed length bottleneck and the feed forward network](https://www.gilesthomas.com/2025/08/the-fixed-length-bottleneck-and-the-feed-forward-network.md) posted on 2025-08-14T23:00:00+00:00
* [Writing an LLM from scratch, part 17 -- the feed-forward network](https://www.gilesthomas.com/2025/08/llm-from-scratch-17-the-feed-forward-network.md) posted on 2025-08-12T23:00:00+00:00
* [Writing an LLM from scratch, part 16 -- layer normalisation](https://www.gilesthomas.com/2025/07/llm-from-scratch-16-layer-normalisation.md) posted on 2025-07-08T18:50:00+00:00
* [Moving from Fabric3 to Fabric](https://www.gilesthomas.com/2025/06/fabric3-to-fabric.md) posted on 2025-06-15T01:30:00+00:00
* [Leaving PythonAnywhere](https://www.gilesthomas.com/2025/06/leaving-pythonanywhere.md) posted on 2025-06-05T19:30:00+00:00
* [Writing an LLM from scratch, part 15 -- from context vectors to logits; or, can it really be that simple?!](https://www.gilesthomas.com/2025/05/llm-from-scratch-15-from-context-vectors-to-logits.md) posted on 2025-05-31T23:55:00+00:00
* [Writing an LLM from scratch, part 14 -- the complexity of self-attention at scale](https://www.gilesthomas.com/2025/05/llm-from-scratch-14-taking-stock-part-2-the-complexity-of-self-attention-at-scale.md) posted on 2025-05-14T21:00:00+00:00
* [Writing an LLM from scratch, part 13 -- the 'why' of attention, or: attention heads are dumb](https://www.gilesthomas.com/2025/05/llm-from-scratch-13-taking-stock-part-1-attention-heads-are-dumb.md) posted on 2025-05-08T22:00:00+00:00
* [Writing an LLM from scratch, part 12 -- multi-head attention](https://www.gilesthomas.com/2025/04/llm-from-scratch-12-multi-head-attention.md) posted on 2025-04-21T23:00:00+00:00
* [Writing an LLM from scratch, part 11 -- batches](https://www.gilesthomas.com/2025/04/llm-from-scratch-11-batches.md) posted on 2025-04-19T23:00:00+00:00
* [Dropout and mandatory vacation](https://www.gilesthomas.com/2025/03/dropout-and-mandatory-vacation.md) posted on 2025-03-24T23:45:00+00:00
* [Writing an LLM from scratch, part 10 -- dropout](https://www.gilesthomas.com/2025/03/llm-from-scratch-10-dropout.md) posted on 2025-03-19T23:30:00+00:00
* [Adding /llms.txt](https://www.gilesthomas.com/2025/03/llmstxt.md) posted on 2025-03-18T22:30:00+00:00


## Posts in category Python

* [Resolver One as a Python Success Story](https://www.gilesthomas.com/2008/08/resolver-one-as-a-python-success-story.md) posted on 2008-08-01T10:41:47+00:00
* [A bit of fun](https://www.gilesthomas.com/2008/09/a-bit-of-fun.md) posted on 2008-09-29T22:52:13+00:00
* [Why use IronPython?](https://www.gilesthomas.com/2008/10/why-use-ironpython.md) posted on 2008-10-07T11:00:35+00:00
* [Ironclad 0.7 released](https://www.gilesthomas.com/2008/11/ironclad-07-released.md) posted on 2008-11-27T22:16:36+00:00
* [Getting phpBB to accept Django sessions](https://www.gilesthomas.com/2008/12/getting-phpbb-to-accept-django-sessions.md) posted on 2008-12-10T16:44:46+00:00
* [xmlrpc](https://www.gilesthomas.com/2009/02/xmlrpc.md) posted on 2009-02-13T14:07:16+00:00
* [R in Resolver One (and perhaps IronPython generally)](https://www.gilesthomas.com/2009/03/r-in-resolver-one-and-probably-ironpython-generally.md) posted on 2009-03-02T19:19:04+00:00
* [Fix for pygame/PyOpenGL/NeHe tutorial windows not disappearing when run from IDLE](https://www.gilesthomas.com/2009/08/fix-for-pygamepyopenglnehe-tutorial-windows-not-disappearing.md) posted on 2009-08-30T20:44:04+00:00
* [3D graphics in Resolver One using OpenGL and Tao, part II: an orrery](https://www.gilesthomas.com/2009/09/3d-graphics-in-resolver-one-using-opengl-and-tao-part-ii-an-orrery.md) posted on 2009-09-17T15:06:10+00:00
* [London Financial Python Users Group](https://www.gilesthomas.com/2009/11/london-financial-python-users-group.md) posted on 2009-11-11T18:19:30+00:00
* [New York Financial Users Group](https://www.gilesthomas.com/2009/11/new-york-financial-users-group.md) posted on 2009-11-13T14:50:28+00:00
* [A website for LFPUG](https://www.gilesthomas.com/2009/12/a-website-for-lfpug.md) posted on 2009-12-07T18:39:50+00:00
* [Next London Financial Python Users Group meeting](https://www.gilesthomas.com/2010/01/next-london-financial-python-users-group-meeting.md) posted on 2010-01-28T15:57:15+00:00
* [London Financial Python Users' Group](https://www.gilesthomas.com/2010/02/london-financial-python-users-group-2.md) posted on 2010-02-16T12:01:54+00:00
* [Playing with NLTK](https://www.gilesthomas.com/2010/02/playing-with-nltk.md) posted on 2010-02-18T18:21:19+00:00
* [OpenCL: .NET, C# and Resolver One integration -- the very beginnings](https://www.gilesthomas.com/2010/03/opencl-net-c-and-resolver-one-integration-the-very-beginnings.md) posted on 2010-03-18T20:16:47+00:00
* [Regular expressions and Resolver One column-level formulae](https://www.gilesthomas.com/2010/04/regular-expressions-and-resolver-one-column-level-formulae.md) posted on 2010-04-26T17:26:29+00:00
* [Generating political news using NLTK](https://www.gilesthomas.com/2010/05/generating-political-news-using-nltk.md) posted on 2010-05-04T17:16:25+00:00
* [London Financial User Group Meeting: September 15](https://www.gilesthomas.com/2010/08/london-financial-user-group-meeting-september-15.md) posted on 2010-08-24T15:06:05+00:00
* [A big announcement from Resolver Systems](https://www.gilesthomas.com/2010/10/a-big-announcement-from-resolver.md) posted on 2010-10-01T18:39:24+00:00
* [London Financial User Group Meeting: 17 January](https://www.gilesthomas.com/2011/01/london-financial-user-group-meeting-17-january.md) posted on 2011-01-10T19:52:19+00:00
* [Busy, busy, busy](https://www.gilesthomas.com/2011/04/busy-busy-busy.md) posted on 2011-04-27T14:37:28+00:00
* [Teaching programming](https://www.gilesthomas.com/2011/10/teaching-programming.md) posted on 2011-10-14T14:16:41+00:00
* [PythonAnywhereAnywhere](https://www.gilesthomas.com/2012/02/pythonanywhereanywhere.md) posted on 2012-02-27T15:31:40+00:00
* [Running Django unit tests on PythonAnywhere](https://www.gilesthomas.com/2012/05/running-django-unit-tests-on-pythonanywhere.md) posted on 2012-05-21T19:35:19+00:00
* [How many Python programmers are there in the world?](https://www.gilesthomas.com/2013/06/how-many-python-programmers-are-there-in-the-world.md) posted on 2013-06-24T18:13:05+00:00
* [An HTTP request's journey through a platform-as-a-service](https://www.gilesthomas.com/2014/08/an-http-requests-journey-through-a-platform-as-a-service.md) posted on 2014-08-20T12:32:33+00:00
* [Parsing website SSL certificates in Python](https://www.gilesthomas.com/2016/12/parsing-website-ssl-certificates-in-python.md) posted on 2016-12-09T17:31:52+00:00
* [Creating a time series from existing data in pandas](https://www.gilesthomas.com/2017/05/creating-a-time-series-from-existing-data-in-pandas.md) posted on 2017-05-09T12:31:40+00:00
* [Python code to generate Let's Encrypt certificates](https://www.gilesthomas.com/2018/11/python-code-to-generate-lets-encrypt-certificates.md) posted on 2018-11-16T19:55:00+00:00
* [Building an AI chatbot for beginners: part 0](https://www.gilesthomas.com/2023/03/ai-llm-bot-beginners-tutorial-00.md) posted on 2023-03-19T20:45:00+00:00
* [Building an AI chatbot for beginners: part 1](https://www.gilesthomas.com/2023/03/ai-llm-bot-beginners-tutorial-01.md) posted on 2023-03-19T21:45:00+00:00
* [Building an AI chatbot for beginners: part 2](https://www.gilesthomas.com/2023/04/ai-llm-bot-beginners-tutorial-02.md) posted on 2023-04-04T19:45:00+00:00
* [Messing around with fine-tuning LLMs](https://www.gilesthomas.com/2024/04/fine-tuning.md) posted on 2024-04-27T22:45:00+00:00
* [Messing around with fine-tuning LLMs, part 2 -- to the cloud!](https://www.gilesthomas.com/2024/04/fine-tuning-2.md) posted on 2024-04-28T22:45:00+00:00
* [Messing around with fine-tuning LLMs, part 3 -- moar GPUs](https://www.gilesthomas.com/2024/05/fine-tuning-3.md) posted on 2024-05-15T23:45:00+00:00
* [Messing around with fine-tuning LLMs, part 4 -- training cross-GPU.](https://www.gilesthomas.com/2024/05/fine-tuning-4.md) posted on 2024-05-21T21:45:00+00:00
* [Messing around with fine-tuning LLMs, part 5 -- exploring memory usage](https://www.gilesthomas.com/2024/07/fine-tuning-5.md) posted on 2024-07-05T17:45:00+00:00
* [Messing around with fine-tuning LLMs, part 6 -- measuring memory usage more systematically](https://www.gilesthomas.com/2024/07/fine-tuning-6.md) posted on 2024-07-10T23:45:00+00:00
* [Messing around with fine-tuning LLMs, part 7 -- detailed memory usage across sequence lengths for an 8B model](https://www.gilesthomas.com/2024/08/fine-tuning-7.md) posted on 2024-08-16T23:45:00+00:00
* [Messing around with fine-tuning LLMs, part 8 -- detailed memory usage across batch sizes](https://www.gilesthomas.com/2024/08/fine-tuning-8.md) posted on 2024-08-25T23:00:00+00:00
* [Messing around with fine-tuning LLMs, part 9 -- gradient checkpointing](https://www.gilesthomas.com/2024/09/fine-tuning-9.md) posted on 2024-09-03T23:00:00+00:00
* [Messing around with fine-tuning LLMs, part 10 -- finally training the model!](https://www.gilesthomas.com/2024/12/fine-tuning-10.md) posted on 2024-12-22T19:00:00+00:00
* [Writing an LLM from scratch, part 2](https://www.gilesthomas.com/2024/12/llm-from-scratch-2.md) posted on 2024-12-23T21:00:00+00:00
* [Writing an LLM from scratch, part 3](https://www.gilesthomas.com/2024/12/llm-from-scratch-3.md) posted on 2024-12-26T22:30:00+00:00
* [An AI chatroom (beginnings)](https://www.gilesthomas.com/2024/12/ai-chatroom-1.md) posted on 2024-12-29T23:15:00+00:00
* [An AI chatroom (a few steps further)](https://www.gilesthomas.com/2024/12/ai-chatroom-2.md) posted on 2024-12-30T23:15:00+00:00
* [Michael Foord: RIP](https://www.gilesthomas.com/2025/01/michael-foord-rip.md) posted on 2025-01-26T20:30:00+00:00
* [Writing an LLM from scratch, part 9 -- causal attention](https://www.gilesthomas.com/2025/03/llm-from-scratch-9-causal-attention.md) posted on 2025-03-09T23:30:00+00:00
* [Writing an LLM from scratch, part 11 -- batches](https://www.gilesthomas.com/2025/04/llm-from-scratch-11-batches.md) posted on 2025-04-19T23:00:00+00:00
* [Writing an LLM from scratch, part 12 -- multi-head attention](https://www.gilesthomas.com/2025/04/llm-from-scratch-12-multi-head-attention.md) posted on 2025-04-21T23:00:00+00:00
* [Writing an LLM from scratch, part 15 -- from context vectors to logits; or, can it really be that simple?!](https://www.gilesthomas.com/2025/05/llm-from-scratch-15-from-context-vectors-to-logits.md) posted on 2025-05-31T23:55:00+00:00
* [Moving from Fabric3 to Fabric](https://www.gilesthomas.com/2025/06/fabric3-to-fabric.md) posted on 2025-06-15T01:30:00+00:00
* [The fixed length bottleneck and the feed forward network](https://www.gilesthomas.com/2025/08/the-fixed-length-bottleneck-and-the-feed-forward-network.md) posted on 2025-08-14T23:00:00+00:00

## Posts in category AI

* [Evolution in action](https://www.gilesthomas.com/2008/10/evolution-in-action.md) posted on 2008-10-03T17:52:56+00:00
* [Building an AI chatbot for beginners: part 0](https://www.gilesthomas.com/2023/03/ai-llm-bot-beginners-tutorial-00.md) posted on 2023-03-19T20:45:00+00:00
* [Building an AI chatbot for beginners: part 1](https://www.gilesthomas.com/2023/03/ai-llm-bot-beginners-tutorial-01.md) posted on 2023-03-19T21:45:00+00:00
* [Building an AI chatbot for beginners: part 2](https://www.gilesthomas.com/2023/04/ai-llm-bot-beginners-tutorial-02.md) posted on 2023-04-04T19:45:00+00:00
* [Giving up on the AI chatbot tutorial (for now)](https://www.gilesthomas.com/2024/02/giving-up-on-tutorial-and-link-to-new-pythonanywhere-blog-post.md) posted on 2024-02-27T20:45:00+00:00
* [LLM Quantisation Weirdness](https://www.gilesthomas.com/2024/02/llm-quantisation-weirdness.md) posted on 2024-02-27T22:45:00+00:00
* [Messing around with fine-tuning LLMs](https://www.gilesthomas.com/2024/04/fine-tuning.md) posted on 2024-04-27T22:45:00+00:00
* [Messing around with fine-tuning LLMs, part 2 -- to the cloud!](https://www.gilesthomas.com/2024/04/fine-tuning-2.md) posted on 2024-04-28T22:45:00+00:00