seangoedecke.com RSS feed
Description
Feed Activity
Latest Posts
Strengths and limitations of diffusion language models
Google recently released Gemini Diffusion, which is impressing everyone with its speed. Supposedly they even had to slow down the demo so people could see what was happening. What’s special...
Practical AI techniques for daily engineering work
Popular advice about using AI is obsessed with tools and “magic prompts”. To me, this is like if software engineering advice was all about memorizing the right vim commands: useful,...
Diffusion models explained simply
Transformer-based large language models are relatively easy to understand. You break language down into a finite set of “tokens” (words or sub-word components), then train a neural network on millions...
Practical advice for engineers in these troubled times
Since 2023, the rise of interest rates has caused a sea change in how software companies relate to their engineers. It’s harder to be a software engineer now than at...
Why strong engineers are rarely blocked
Earlier in my career I used to get blocked1 constantly, but these days it’s rare. What changed? At first glance being blocked sounds like it’s outside your control, since it...
For many, patience is the killer LLM feature
Sam Altman, CEO of OpenAI, famously said that his goal was to make intelligence “too cheap to meter”. Right now, buoyed by venture capital, we’re living in that world. Every...
Why can't language models come up with new ideas?
Why can’t large language models come up with new ideas? Human polymaths routinely come up with interesting new ideas by combining insights from different areas. But language models have not...
The importance of virtue in software engineering
Software engineers care a lot about being smart and knowledgeable. Conversations about how to become a better software engineer often center around learning more facts: programming language syntax, design patterns,...
How projects fail at large tech companies
How do projects fail at large tech companies? As I’ve said many times, failure means executives aren’t happy with how the project turned out. At healthy companies, that typically means...
Getting things "done" in large tech companies
What does it mean to get things done? In the abstract, you can complete a mathematical proof or a problem set, but the real world is much fuzzier. Suppose I...
I don't care about your magic prompts
There’s a brand of tech influencer now that’s all about sharing the perfect prompt for any situation. The tweets in question typically read something like “this prompt will make you...
The valley of engineering despair
I have delivered a lot of successful engineering projects. When I start on a project, I’m now very (perhaps unreasonably) confident that I will ship it successfully. Even so, in...
Sycophancy is the first LLM "dark pattern"
People have been making fun of OpenAI models for being overly sycophantic for months now. I even wrote a post advising users to pretend that their work was written by...
Debugging, emotional resilience, and mental models
Being good at debugging is more useful than being good at writing code - you only write a piece of code once, but you may end up debugging it hundreds...
Senior engineers should make side bets
When you’re a junior, you should work on what you’re given. There are two reasons for this. First, your work needs to be supervised and checked by a more experienced...
The OpenAI house style is exhausting
I was reading this Reddit post when I noticed a pattern: a few times now I’ve seen a negative Reddit comment that to me just screamed “written by ChatGPT”. Here...
Anarchy in the East India Company
I recently read (well, listened to the audiobook of) The Anarchy: The Relentless Rise of the East India Company by William Dalrymple1. Before reading The Anarchy, my vague pop-culture understanding...
When you should lie to the language model
Here’s an unreasonably effective trick for working with AIs: always pretend that your work was produced by someone else. The problem is that current-generation AIs are too agreeable. They’re trained...
Is using AI wrong? A review of six popular anti-AI arguments
Some people really, really don’t like AI. Broadly speaking, being anti-AI is a popular left-wing position: AI1 is cringe, it’s plagiarism, it’s stunting real growth, it’s killing the environment, it’s...
Principles for coding securely with LLMs
Writing code with LLMs is fundamentally different from other ways of programming. LLMs are often non-deterministic and always unpredictable. They have a capability that no other technology can match: the...