Why bother using higher-level programming languages to communicate with a computer? You interact with a computer using assembly - raw bit shifting and memory addresses - which is more precise and effective.

[dead]

That's an improvement, they are still "chaotic" though in that small changes in input can change the output unpredictably strong

That assumes they were implemented with deterministic operators, which isn't the default assumption when using neural network libs on GPUs. Imagine random seeds, cublas optimizations - like you can configure all these things, but I wouldn't assume it, esp in GPU-optimized OSS..

'cos COBOL was designed to be human readable (writable ?).

Output "1" if the program halts; "0" if it doesn't.

That's entirely orthogonal.

In probabilistic programming you (deterministically) define variables and formulas. It's just that the variables aren't instances of floats, but represent stochastic variables over floats.

This is similar to libraries for linear algebra where writing A * B * C does not immediately evaluate, but rather builds an expression tree that represent the computation; you need to do say `eval(A * B * C)` to obtain the actual value, and it gives the library room to compute it in the most efficient way.

It's more related to symbolic programming and lazy evaluation than (non-)determinism.

that would require actually curating the training data and eliminating sources that contain casual conversation

too expensive since those are all licensed sources, much easier to train on Reddit data

Why do people keep having this reaction to something we're already used to? When you're developing against an API, you're already doing the same thing, planning for what happens when the request hangs, or fails completely, or gives a different response, and so on. Same for basically any IO.

It's almost not even new, just that it generates text instead of JSON, or whatever. But we've already been doing "indeterministic programming" for a long time, where you cannot always assume a function 100% returns what it should all the time.

> It makes no sense at all, it's cuckooland, are you all on crazy pills?

First step towards understanding something you obviously have strong feelings about, is to try to avoid hitting those triggers while you think about the thing, otherwise it clouds you. Not a requirement by any measure, just a tip.

> are you telling me people will do three years university to learn to prompt?

Are people going to university for three years to write "1.0" or "2.0" software? I certainly didn't, and I don't think even the majority of software developers have done so, at least in my personal experience but YMMV.

> I do not understand where there is anything here to be "not sure" on?

They're not sure about the specific naming, not the concept or talk as a whole.

> LLMs making non-deterministic mistakes

Everything they do is non-deterministic when temperature is set to anything above 0.0, as that's the entire point. The "correct" answers are as non-deterministic as the "mistakes", although I'm not sure "mistake" is correct because it did chose the right/correct tokens, it's just that you didn't like/expect it to chose that particular tokens.

> It makes no sense at all, it's cuckooland, are you all on crazy pills?

Frequent LLM usage impairs thinking. The LLM has no connection to reality, and it takes over people's minds.

Do you think you could condense your point of view without hyperbole and rudeness so the rest of us can understand it?

What do you think about structured outputs / JSON mode / constrained decoding / whatever you wish to call it?

To me, it's a criminally underused tool. While "raw" LLMs are cool, they're annoying to use as anything but chatbots, as their output is unpredictable and basically impossible to parse programmatically.

Structured outputs solve that problem neatly. In a way, they're "neural networks without the training". They can be used to solve similar problems as traditional neural networks, things like image classification or extracting information from messy text, but all they require is a Zod or Pydantic type definition and a prompt. No renting GPUs, labeling data and tuning hyperparameters necessary.

They often also improve LLM performance significantly. Imagine you're trying to extract calories per 100g of product, but some product give you calories per serving and a serving size, calories per pound etc. The naive way to do this is a prompt like "give me calories per 100g", but that forces the LLM to do arithmetic, and LLMs are bad at arithmetic. With structured outputs, you just give it the fifteen different formats that you expect to see as alternatives, and use some simple Python to turn them all into calories per 100g on the backend side.

The versioning makes sense to me. Software has a cycle where a new tool is created to solve a problem, and the problem winds up being meaty enough, and the tool effective enough, that the exploration of the problem space the tool unlocks is essentially a new category/skill/whatever.

computers -> assembly -> HLL -> web -> cloud -> AI

Nothing on that list has disappeared, but the work has changed enough to warrant a few major versions imo.

> versioning is a bit confusing analogy because it usually additionally implies some kind of improvement

Exactly what I felt. Semver like naming analogies bring their own set of implicit meanings, like major versions having to necessarily supersede or replace the previous version, that is, it doesn't account for coexistence further than planning migration paths. This expectation however doesn't correspond with the rest of the talk, so I thought I might point it out. Thanks for taking the time to reply!

Andrej, maybe Software 3.0 is not written in spoken language like code or prompts. Software 3.0 is recorded in behavior, a behavior that today's software lacks. That behavior is written and consumed by machine and annotated by human interaction. Skipping to 3.0 is premature, but Software 2.0 is a ramp.

no no, it actually is a good analogy in 2 ways:

1) it is a breaking change from the prior version

2) it is an improvement in that, in its ideal/ultimate form, it is a full superset of capabilities of the previous version

> The problem with trying to make "English -> formal language -> (anything else)" work is that informality is, by definition, not a formal specification and therefore subject to ambiguity. The inverse is not nearly as difficult to support.

Both directions are difficult and important. How do you determine when going from formal to informal that you got the right informal statement? If you can judge that, then you can also judge if a formal statement properly represents an informal one, or if there is a problem somewhere. If you detect a discrepancy, tell the user that their English is ambiguous and that they should be more specific.

Yes, you just don't understand :-)

I am working on making it simpler to understand, and particularly, simpler to use.

PS: People keep browsing the older papers although they are really outdated. I've updated http://abstractionlogic.com to point to the newest information instead.

Going from informal to formal can be done using autoformalization [1]. The real question is, how do you judge that the result is correct?

[1] Autoformalization with Large Language Models — https://papers.nips.cc/paper_files/paper/2022/hash/d0c6bc641...

Referenced in the video of course. Not that everyone should watch a 40 minute long video before commenting but his reaction to the "meme" that vibe coding became when his tweet was intended as more of a shower thought is worth checking out.

You would definitely think so, Lean is in a great position here!

I am betting though that type theory is not the right logic for this, and that Lean can be leapfrogged.

> This thread perfectly captures what Karpathy was getting at. We're witnessing a fundamental shift where the interface to computing is changing from formal syntax to natural language.

Yes, telling a subordinate with natural language what you need is called being a product manager. Problem is, the subordinate has encyclopedic knowledge but it's also extremely dumb in many aspects.

I guess this is good for people that got into CS and hate the craft so prefer doing management, but in many cases you still need in your team someone with a IQ higher than room temperature to deliver a product. The only "fundamental" shift here is killing the entry-level coder at the big corp tasked at doing menial and boilerplate tasks, when instead you can hire a mechanical replacement from an AI company for a few hundred dollars a month.

Have you thought through the downsides of letting go of these formal foundations that have nothing to do with job preservation? This comes across as a rather cynical interpretation of the motivations of those who have concerns.

> We're witnessing a fundamental shift where the interface to computing is changing from formal syntax to natural language.

People have said this every year since the 1950's.

No, it is not happening. LLMs won't help.

Writing code is easy, it's understanding the problem domain is hard. LLMs won't help you understand the problem domain in a formal manner. (In fact they might make it even more difficult.)

Yep, that why I never write anything out using mathmatical expressions. Natural language only baby!

No. Karpathy has long embraced the Silly-con valley “Fake it until you make it” mind set. One of his slides even had a frame of Tesla self driving video that was later revealed to be faked.

It’s in his financial self interest to over inflate LLM’s beyond their “cool math bar trick” level. They are a lossy text compression technique with stolen text sources.

All this “magic” is just function calls behind the scenes doing web/database/math/etc for the LLM.

Anyone who claims LLMs have a soul either truly doesn’t understand how they work (association rules++) or has hitched their financial wagon to this grift. It’s the crypto coin bruhs looking for their next score.

Not really. There’s a problem to be solved, and the solution is always best exprimed in formal notation, because we can then let computers do it and not worry about it.

We already have natural languages for human systems and the only way it works is because of shared metaphors and punishment and rewards. Everyone is incentivized to do a good job.

This is why I call all this AI stuff BS.

Using a formal language is a feature, not a bug. It is a cornerstone of all human engineering and scientific activity and is the _reason_ why these disciplines are successful.

What you are describing (ie. ditching formal and using natural language) is moving humanity back towards magical thinking, shamanism and witchcraft.

It's called gatekeeping and the gatekeepers will be the ones left in the dust. This has been proven time and time again. Better learn to go with the flow - judging LLMs on linear improvements or even worse on today's performance is a fool's errand.

Even if improvements level off and start plateauing, things will still get better and for careful guided, educated use LLMs have already become a great accelerator in many ways. StackOverflow is basically dead now which in itself is a fundamental shift from just 3-4 years ago.

My takeaway from the demo is less that "it's different each time", but more a "it can be different for different users and their styles of operating" - a poweruser can now see a different Settings UI than a basic user, and it can be generated realtime based on the persona context of the user.

Example use case (chosen specifically for tech): An IDE UI that starts basic, and exposes functionality over time as the human developer's skills grow.

On one hand, I'm incredibly impressed by the technology behind that demo. On the other hand, I can't think of many things that would piss me off more than a non-deterministic operating system.

I like my tools to be predictable. Google search trying to predict that I want the image or shopping tag based on my query already drives me crazy. If my entire operating system did that, I'm pretty sure I'd throw my computer out a window.

I feel like one quickly hits a similar partial observability problem as with e.g. light sensors. How often do you wave around annoyed because the light turned off.

To get _truly_ self driving UIs you need to read the mind of your users. It's some heavy tailed distribution all the way down. Interesting research problem on its own.

We already have adaptive UIs (profiles in VSC anyone? Vim, Emacs?) they're mostly under-utilized because takes time to setup + most people are not better at designing their own workflow relative to the sane default.

it's impressive but it seems like a crappier UX? that none of the patterns can really be memorized

This is crazy cool, even if not necessarily the best use case for this idea

I would bet good money that many of the functions they chose not to drill down into (such as settings -> volume) do nothing at all or cause an error.

It's a fronted generator. It's fast. That's cool. But is being pitched as a functioning OS generator and I can't help but think it isn't given the failure rates for those sorts of tasks. Further, the success rates for HTML generation probably _are_ good enough for a Holmes-esque (perhaps too harsh) rugpull (again, too harsh) demo.

A cool glimpse into what the future might look like in any case.

That looks both cool and infuriating

Having different documents come up every time you go into the documents directory seems hellishly terrible.

[flagged]

I'm curious where this ends up going.

Personally I think its a mistake; at least at "team" level. One of the most valuable things about a software or framework dictating how things are done is to give a group of people a common language to communicate with and enforce rules. This is why we generally prefer to use a well documented framework, rather than letting a "rockstar engineer" roll their own. Only they will understand its edge cases and ways of thinking, everyone else will pay a price to adapt to that, dragging everyone's productivity down.

Secondly, most people don't know what they want or how they want to work with a specific piece of software. Its simply not important enough, in the hierarchy of other things they care about, to form opinions about how a specific piece of software ought to work. What they want, is the easiest and fastest way to get something done and move on. It takes insight, research and testing to figure out what that is in a specific domain. This is what "product people" are supposed to figure out; not farm it out to individual users.

It wouldn't be unlearnable if it fits the way the user is already thinking.

A mixed ever-shifting UI can be excellent though. So you've got some tools which consistently interact with UI components, but the UI itself is altered frequently.

Take for example world-building video games like Cities Skylines / Sim City or procedural sandboxes like Minecraft. There are 20-30 consistent buttons (tools) in the game's UX, while the rest of the game is an unbounded ever-shifting UI.

Tools like v0 are a primitive example of what the above is talking about. The UI maintains familiar conventions, but is laid out dynamically based on surrounding context. I'm sure there are still weird edge cases, but for the most part people have no trouble figuring out how to use the output of such tools already.

Like Spotify ugh

We (https://vibes.diy/) are betting on this

What is the mess you are referring with regards to Jupyter notebooks ?

I see your point. It's nearing essential.

> Prompts and LLMs most certainly do not work like that.

It quite literally works like that. The computer is now OS + user-land + LLM runner + ML architecture + weights + system prompt + user prompt.

Taken together, and since you're adding in probabilities (by using ML/LLMs), you're quite literally getting "the computer only does exactly what you tell it to do!", it's just that we have added "but make slight variations to what tokens you select next" (temperature>0.0) sometimes, but it's still the same thing.

Just like when you tell the computer to create encrypted content by using some seed. You're getting exactly what you asked for.

Yeah, wherever possible I try to have the llm answer me in Python rather than English (especially when explaining new concepts)

English is soooooo ambiguous

I specifically do not want AI as an auto-correct, doing auto-predictions while I am typing. I find this interrupts my thinking process, and I've never been bottlenecked by typing speed anyway.

I want AI as a "co-worker" providing an alternative perspective or implementing my specific instructions, and potentially filling in gaps I didn't think about in my prompt.

A simple rule applies: "No matter what tool created the code, you are still responsible for what you merge into main".

As such, task of verification, still falls on hands of engineers.

Given that and proper processes, modern tooling works nicely with codebases ranging from 10k LOC (mixed embedded device code with golang backends and python DS/ML) to 700k LOC (legacy enterprise applications from the mainframe era)

With lazy people the same applies for everything, code they do write, or code they review from peers. The issue is not the tooling, but the hands.

Yes, and (some near-future) AI is also more patient and better at multitasking than a reasonable human. It can make a change, submit for full fuzzing, and if there's a problem it can continue with the saved context it had when making the change. It can work on 100s of such changes in parallel, while a human trying to do this would mix up the reasons for the change with all the other changes they'd done by the time the fuzzing result came back.

LLMs are worse at many things than human programmers, so you have to try to compensate by leveraging the things they're better at. Don't give up with "they're bad at such and such" until you've tried using their strengths.

Humans tend to lack inhumane patience.

It is kind of a human problem too, although that the full testing suite takes X hours to run is also not fun, but it makes the human problem larger.

Say you're Human A, working on a feature. Running the full testing suite takes 2 hours from start to finish. Every change you do to existing code needs to be confirmed to not break existing stuff with the full testing suite, so some changes it takes 2 hours before you have 100% understanding that it doesn't break other things. How quickly do you lose interest, and at what point do you give up to either improve the testing suite, or just skip that feature/implement it some other way?

Now say you're Robot A working on the same task. The robot doesn't care if each change takes 2 hours to appear on their screen, the context is exactly the same, and they're still "a helpful assistant" 48 hours later when they still try to get the feature put together without breaking anything.

If you're feeling brave, you start Robot B and C at the same time.

The full test suite is probably tens of thousands of tests.

But AI will do a pretty decent job of telling you which tests are most likely to fail on a given PR. Just run those ones, then commit. Cuts your test time from hours down to seconds.

Then run the full test suite only periodically and automatically bisect to find out the cause of any regressions.

Dramatically cuts the compute costs of tests too, which in big codebase can easily become whole-engineers worth of costs.

I work in web dev, so people sometimes hook code formatting as a git commit hook or sometimes even upon file save. The tests are problematic tho. If you work at huge project it's a no go idea at all. If you work at medium then the tests are long enough to block you, but short enough for you not to be able to focus on anything else in the meantime.

Unless you are doing something crazy like letting the fuzzer run on every change (cache that shit), the full test suite taking a long time suggests that either your isolation points are way too large or you are letting the LLM cross isolated boundaries and "full testing suite" here actually means "multiple full testing suites". The latter is an easy fix: Don't let it. Force it stay within a single isolation zone just like you'd expect of a human. The former is a lot harder to fix, but I suppose ending up there is a strong indicator that you can't trust the human picking the best LLM result in the first place and that maybe this whole thing isn't a good idea for the people in your organization.

I don't know exactly how Claude works, but the way I work around this with my own stuff is prompting it to not display full outputs ever, and instead temporary redirect the output somewhere then grep from the log-file what it's looking for. So a test run outputting 10K lines of test output and one failure is easily found without polluting the context with 10K lines.

Claude's approach is currently a bit dated.

Cursor.sh agents or especially OpenAI Codex illustrate that a tool doesn't need to keep on stuffing context window with irrelevant information in order to make progress on a task.

And if really needed, engineers report that Gemini Pro 2.5 keeps on working fine within 200k-500k token context. Above that - it is better to reset the context.

I started to use sub agents for that. That does not pollute the context as much

I say this all the time!

Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit.

Also I can’t really imagine that in the first place. At my current job, each task is like 95% understanding all the little bits, and then 5% writing the code. If you’re reviewing PRs from a bot all day, you’ll still need to understand all the bits before you accept it. So how much time is that really gonna save?

>That sounds awful.

Not for the cloud provider. AWS bill to the moon!

I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code.

Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.

If it's monkeylike quality and you need a million tries, it's shit. It you need four tries and one of those is top-tier professional programmer quality, then it's good.

> A truly terrible and demotivating way to work and produce anything of real quality

You clearly have strong feelings about it, which is fine, but it would be much more interesting to know exactly why it would terrible and demotivating, and why it cannot produce anything of quality? And what is "real quality" and does that mean "fake quality" exists?

> million monkey interns banging on one million keyboards and submit a million PRs

I'm not sure if you misunderstand LLMs, or the famous "monkeys writing Shakespeare" part, but that example is more about randomness and infinity than about probabilistic machines somewhat working towards a goal with some non-determinism.

> We’re beyond doomed

The good news is that we've been doomed for a long time, yet we persist. If you take a look at how the internet is basically held up by duct-tape at this point, I think you'd feel slightly more comfortable with how crap absolutely everything is. Like 1% of software is actually Good Software while the rest barely works on a good day.

> That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality

This is the right way to work with generative AI, and it already is an extremely common and established practice when working with image generation.

Not gemini but google lens. Maybe gemini already has some agentic capabilities

Based on my experience with voicemail, I'd say that audio is not always best, and is sometimes in the running for worst.

> I think that if you give the same task to three different developers you'll get three different implementations.

Yes, but if you want them to be compatible you need to define a protocol and conformance test suite. This is way more work than writing a single implementation.

The code is the real spec. Every piece of unintentional non-determinism can be a hazard. That’s why you want the code to be the unit of maintenance, not a prompt.

Interestingly, I was generating some scraping code today. I prompted it fairly generically and it decided to spit out some Selenium code. I reported the stack trace failure, and it gave me a new script with playwright. That failed also and it gave me some suggestions to fix it. I asked it to update the whole script rather than snippets, and it responded with "Hey let's not use either of these and here we'll use the site's API." and proceeded to do that.

Kind of crazy, it basically found 3 different hammers to hit the nail I wanted. The API unfortunately seems to be timeing out (I had to add the timeout=10 to the post u_u)

With your tool can I hook it up to Cursor and just have it use it?

It’s very obvious this account was just created to promote your product…

Fair point. I'm no expert in custom GPTs, I wonder what limitations there would be beyond the obvious branding and UI/UX control. Like, how far can someone customize a custom GPT (ha). I imagine any multi-step/agentic flows might be a challenge or impossible as it currently exists. It also seems like custom GPTs have been completely forgotten, but I very well could be wrong and OpenAI announced a big investment in them and new features tomorrow.

Probably some sort. In the meantime it doesn't currently exist and I want it for myself. I also feel like having something open source and that allows you to bring your own LLM provider might still be useful.

I wonder at what point all the self-driving code becomes replaceable with a multimodal generalist model with the prompt “drive safely”

Aren't continuous, stochastic, partial knowledge environments where you need long horizon planning with strict deadlines and limited compute exactly the sort of environments muzero variants struggle with? Because that's driving.

It's also worth mentioning that humans intentionally (and safely) drive into "solid" objects all the time. Bags, steam, shadows, small animals, etc. We also break rules (e.g. drive on the wrong side of the road), and anticipate things we can't even see based on a theory of mind of other agents. Human driving is extremely sophisticated, not reducible to rules that are easily expressed in "simple" language.

> We don't need to know what a cone is

The counter argument is that you can't zoom in and fix a specific bug in this mode of operation. Everything is mashed together in the same neural net process. They needed to ensure safety, so testing was crucial. It is harder to test an end-to-end system than its individual parts.

I don't think that would have worked either.

But if they'd gone for radars and lidars and a bunch of sensors and then enough processing hardware to actually fuse that, then I think they could have built something that had a chance of working.

That's absolutely not what makes humans great drivers?

Is that the approach that waymo uses?

I’m going to edit, it was badly formulated, he underestimates their potential for growth is what I meant by that

He understands the technical part, of course, I was referring to his prediction that large models will be always be necessary.

There is a point where an LLM is good enough for most tasks, I don’t need a megamind AI in order to greet clients, and both large and small/medium model size are getting there, with the large models hitting a computing/energy demand barrier. The small models won’t hit that barrier anytime soon.

I edited to make it clearer

We all have blind spots

This is a bad comparison

Phi4-mini runs on a basic laptop CPU at 20T/s… how is that slow? Without optimization…

I have been using LLMs for coding a lot during the past year, and I've been writing down my observations by task. I have a lot of tasks where my first entry is thoroughly impressed by how e.g. Claude helped me with a task, and then the second entry is a few days after when I'm thoroughly irritated by chasing down subtle and just _strange_ bugs it introduced along the way. As a rule, these are incredibly hard to find and tedious to debug, because they lurk in the weirdest places, and the root cause is usually some weird confabulation that a human brain would never concoct.

Saw a recent talk where someone described AI as making errors, but not errors that a human would naturally make and are usually "plausible but wrong" answers. i.e. the errors that these AI's make are of a different nature than what a human would do. This is the danger - that reviews now are harder; I can't trust it as much as a person coding at present. The agent tools are a little better (Claude Code, Aider, etc) in that they can at least take build and test output but even then I've noticed it does things that are wrong but are "plausible and build fine".

I've noticed it in my day-to-day: an AI PR review is different than if I get the PR from a co-worker with different kinds of problems. Unfortunately the AI issues seem to be more of the subtle kind - the things if I'm not diligent could sneak into production code. It means reviews are more important, and I can't rely on previous experience of a co-worker and the typical quality of their PR's - every new PR is a different worker effectively.

I'm curious where that expectation of the flip comes from? Your experience (and mine, frankly) would seem to indicate the opposite, so from whence comes this certainty that one day it'll change entirely and become reliable instead?

I ask (and I'll keep asking) because it really seems like the prevailing narrative is that these tools have improved substantially in a short period of time, and that is seemingly enough justification to claim that they will continue to improve until perfection because...? waves hands vaguely

Nobody ever seems to have any good justification for how we're going to overcome the fundamental issues with this tech, just a belief that comes from SOMEWHERE that it'll happen anyway, and I'm very curious to drill down into that belief and see if it comes from somewhere concrete or it's just something that gets said enough that it "becomes true", regardless of reality.

Brutal counter take: If AI tooling makes you so much better, then you started very low. In contrast, if you are already insanely productive in creative ways others can hardly achieve then chances are, AI tools don't make much of a difference.

I actually quite agree with this, there is some reckoning on both sides happening. It's quite entertaining to watch, a bit painful as well of course as someone who is on the "they are useless" side and is noticing some very clear usecases where a value add is present.

> The reality of what these tools can do is sinking in

It feels premature to make determinations about how far this emergent technology can be pushed.

> The boosters are in 5 stages of grief coming to terms with what was once AGI and is now a mere co-pilot, while the haters are coming to terms with the fact that LLMs can actually be useful in a variety of usecases.

I couldn't agree with this more. I often get frustrated because I feel like the loudest voices in the room are so laughably extreme. One on side you have the "AGI cultists", and on the other you have the "But the hallucinations!!!" people. I've personally been pretty amazed by the state of AI (nearly all of this stuff was the domain of Star Trek just a few years ago), and I get tons of value out of many of these tools, but at the same time I hit tons of limitations and I worry about the long-term effect on society (basically, I think this "ask AI first" approach, especially among young people, will kinda turn us all into idiots, similar to the way Google Maps made it hard for most of us to remember the simple directions). I also can't help but roll my eyes when I hear all the leaders of these AI companies going on about how AI will make a "white collar bloodbath" - there is some nuggets of truth in that, but these folks are just using scare tactics to hype their oversold products.

Exactly! What skeptics don't get is that AGI is already here and we are now starting a new age of infinite prosperity, it's just that exponential growth looks flat at first, obviously...

Quantum computers and fusion energy are basically solved problems now. Accelerate!

The thing is that there’s a need to integrate all these little tools because the problems they solve is part of the same domain. And that’s where problems lie. Something like Excel have an advantage as being a common platform for both data and procedures. Unix adopted text and pipes for integration.

I get that, motivating young founders is difficult, and I think he has a charming geeky way of provoking some thoughts. But on the other hand: Why mainframes with time-sharing from the 60s? Why operating systems? LLMs to tell you how to boil an egg, seriously?

Putting my engineering hat on, I understand his idea of the "autonomy slider" as lazy workaround for a software implementation that deals with one system boundary. He should aspire people there to seek out for unknown boundaries, not provide implementation details to existing boundaries. His MenuGen app would probably be better off using a web image search instead of LLM image generation. Enhancing deployment pipelines with LLM setups is something for the last generation of DevOps companies, not the next one.

Please mention just once the value proposition and responsibilities when handling large quantities of valuable data - LLMs wouldn't exist without them! What makes quality data for an LLM, or personal data?

I don't agree. Important code will need to be audited. I think the language of the future will be easy to read by human reviewers but deterministic. It won't be a human language. Instead it will be computer language with horrible ergonomics. I think Python or straight up Java would be a good start. Things like templates wouldn't be necessary since you could express that deterministically in a higher level syntax (e.g. A list of elements that can accept any type). It would be an interesting exercise.

If humans don't understand it to write the data the LLM is trained on, how will the LLM be able to learn it?

Better to think of it as a world driven by the strengths and weaknesses of people. Is the world better if more people can express themselves via software? Yes.

I don’t believe in coincidences. I don’t think the universe provided AI by accident. I believe it showed up just at the moment where the universe wants to make it clear - your little society of work and status and money can go straight to living hell. And that’s where it’s going, the developer was never supposed to be a rockstar, they were always meant to be creatives who do it because they like it. Fuck this job bullshit, those days are over. You will program the same way you play video games, it’s never to be work again (it’s simply too creative).

Will the universe make it so a bunch of 12 year olds dictate software in natural language in a Roblox like environment that rivals the horeshit society sold for billions just a decade ago? Yes, and thank god. It’s been a wild ride, thank you god for ending it (like he did with nuclear bombs after ww2, our little universe of war shrunk due to that).

Anyways, always pay attention to the little details, it’s never a coincidence. The universe doesn’t just sit there and watch our fiasco believe it or not, it gets involved.

I did. I also had basically unlimited access to pornography and I saw more than one video of someone having their head severed off. But yeah, I played a lot of computer games. That was fun.

Yes these look perfect! Thank you.

Similarly to how you can never guarantee that one of your trusted employees won’t be made a foreign asset.

> If anyone has a browser plugin which automatically redirects to llms.txt sign me up.

Not a browser plugin, but you can prefix URLs with `pure.md/` to get the pure markdown of that page. It's not quite a 1:1 to llms.txt as it doesn't explain the entire domain, but works well for one-off pages. [disclaimer: I'm the maintainer]

The next version of the llms.txt proposal will allow an llms.txt file to be added at any level of a path, which isn't compatible with /.well-known.

(I'm the creator of the llms.txt proposal.)

The web started dying with mobile social media apps, in which hyperlinks are a poor UX choice. Then again with SEO banning outlinks. Now this. The web of interconnected pages that was the World Wide Web is dead. Not on social media? No one sees you. Run a website? more bots than humans. Unless you sell something on the side with the website it's not profitable. Hyperlinking to other websites is dead.

Gen Alpha doesn't know what a web page is and if they do, it's for stuff like neocities aka as a curiosity or art form only. Not as a source of information anymore. I don't blame them. Apps (social media apps) have less friction than web sites but have a higher barrier for people to create. We are going back to pre World Wide Web days in a way, kind of like Bulletin Board Systems on dial up without hyperlinking, and centralized (social media) Some countries mostly ones with few technical people llike the ones in Central America have moved away from the web almost entirely and into social media like Instagram.

Due to the death of the web, google search and friends now rely mostly on matching queries with titles now so just like before the internet you have to know people to learn new stuff or wait for an algorithm to show it to you or someone to comment it online or forcefully enroll in a university. Maybe that's why search results have declined and poeple search using ChatGPT or maybe perplexity. Scholarly search engines are a bit better but frankly irrelevant for most poeple.

Now I understand why Google established their own DNS server at 8.8.8.8. If you have a directory of all domains on DNS, you can still index sites without hyperlinks between them, even if the web dies. They saw it coming.

<link rel="alternate" /> is a standards-friendly way to semantically represent the same content in a different format

That's not what llms.txt is. You can just use a regular markdown URL or similar for that.

llms.txt is a description for an LLM of how to find the information on your site needed for an LLM to use your product or service effectively.

I'm pretty sure "actually work" part refers to SWE space rather than LLM/ML space

We had exactly the opposite experience. CoPilot was able to answer questions accurately and reformatted the existing documentation to fit the context of users' questions, which made the information much easier to understand.

Code examples, which we offer as sort of reference implementations, were also adopted to fit the specific questions without much issues. Granted these aren't whole applications, but 10 - 25 line examples of doing API setup / calls.

We didn't, of course, just send users' questions directly to CoPilot. Instead there's a bit of prompt magic behind the scenes that tweaks the context so that CoPilot can produce better quality results.

Or will they ever be reliable. Your question is already making an assumption.

“Treat it like a junior developer” … 5 years later … “Treat it like a junior developer”

∞

Perhaps consider making some tutorials, then, and share your wealth of knowledge rather than calling people stupid.

AI Snake Oil: https://press.princeton.edu/books/hardcover/9780691249131/ai...

You're not the first, nor the last person, to have a seemingly vastly different experience than me and others.

So I'm curious, what am I doing differently from what you did/do when you try them out?

This is maybe a bit out there, but would you be up for sending me like a screen recording of exactly what you're doing? Or maybe even a video call sharing your screen? I'm not working in the space, have no products or services to sell, only curious is why this gap seemingly exists between you and me, and my only motive would be to understand if I'm the one who is missing something, or there are more effective ways to help people understand how they can use LLMs and what they can use them for.

My email is on my profile if you're up for it. Invitation open for others in the same boat as parent too.

I've got a working theory that models perform differently when used in different timezones... As in during US working hours they dont work as well due to high load. When used at 'offpeak' hours not only are they (obviously) snappier but the outputs appear to be a higher standard. Thought this for a while but now noticing with Claude4 [thinking] recently. Textbook case of anecdata of course though.

To add to this, I ran into a lot of issues too. And similar when using cursor... Until I started creating a mega list of rules for it to follow that attaches to the prompts. Then outputs improved (but fell off after the context window got too large). At that stage I then used a prompt to summarize, to continue with a new context.

I think part of the problem is that code quality is somewhat subjective and developers are of different skill levels.

If you're fine with things that kinda working okay and you're not the best developer yourself then you probably think coding agents work really really well because the slop they produce isn't that much worse than yourself. In fact I know a mid-level dev who believes agent AIs write better code than himself.

If you're very critical of code quality then it's much tougher... This is even more true in complex codebases where simply following some existing pattern to add a new feature isn't going to cut it.

The degree to which it helps any individual developer will vary, and perhaps it's not that useful for yourself. For me over the last few months the tech has got to the point where I use it and trust it to write a fair percentage of my code. Unit tests are an example where I find it does a really good job.

Unironically, your comment mirrors my opinion as of last month.

Since then I've given it another try last week and was quite literally mind blown how much it improved in the context of Vibe coding (Claude code). It actually improved so much that I thought "I would like to try that on my production codebase", (mostly because I want if to fail, because that's my job ffs) but alas - that's not allowed at my dayjob.

From the limited experience I could gather over the last week as a software dev with over 10 yrs of experience (along with another 5-10 doing it as a hobby before employment) I can say that I expect our industry to get absolutely destroyed within the next 5 yrs.

The skill ceiling for devs is going to get mostly squashed for 90% of devs, this will inevitably destroy our collective bargaining positions. Including for the last 10%, because the competition around these positions will be even more fierce.

It's already starting, even if it's currently very misguided and mostly down to short-sightedness.

But considering the trajectory and looking at how naive current llms coding tools are... Once the industry adjusts and better tooling is pioneered... it's gonna get brutal.

And most certainly not limited to software engineering. Pretty much all desk jobs will get hemorrhaged as soon as a llm-player basically replaces SAP with entirely new tooling.

Frankly, I expect this to go bad, very very quickly. But I'm still hoping for a good ending.

> So I must ask, are ML folks actually qualified to know the future of software engineering?

Probably not CRUD apps typical to back office or website software, but don't forget that ML folks come from the stock of people that built Apollo, Mars Landers, etc. Scientific computing shares some significant overlap with SWE, and ML is a subset of that.

IMHO, the average SWE and ML person are different types when it comes to how they cargocult develop, but the top 10% show significant understanding and re speed across domains.

This seems to be overstating the separation. For people doing applied ML, there's often been a dual responsibility that included a significant amount of software engineering. I wouldn't necessarily listen to such declarations from an ML researcher whose primary output is papers, but from ML engineers who have built and shipped products/services/libraries I think it's much more reasonable.

Having worked with game designers writing code for their missions/levels in a scripting language, I'd say this has been the case for quite a long while.

They start with the code from another level, then modify it until it seems to do what they want. During the alpha testing phase, we'd have a programmer read through the code and remove all the useless cruft and fix any associated bugs.

In some sense that's what vibe coding with an AI is like if you don't know how to code. You have the AI make some initial set of code that you can't evaluate for correctness, then slowly modify it until it seems to behave generally like you want. You might even learn to recognize a few things in the code over time, at which point you can directly change some variables or structures in the code directly.

> Think about all the domain experts, entrepreneurs, researchers, designers, and creative people who have incredible ideas but have been locked out of software development because they couldn't invest 5-10 years learning to code.

> it's about unleashing all the pent-up innovation from people who understand problems deeply but couldn't translate that understanding into software.

This is just a fantasy. People with "incredible ideas" and "pent-up innovation" also need incredible determination and motivation to make something happen. LLMs aren't going to magically help these people gain the energy and focus needed to pursue an idea to fruition. Coding is just a detail; it's not the key ingredient all these "locked out" people were missing.

> These people have been sitting on the sidelines because the barrier to entry was so high.

This comment is wildly out of touch. The SMB owner can now generate some Python code. Great. Where do they deploy it? How do they deploy it? How do they update it? How do they handle disaster recovery? And so on and so forth.

LLMs accelerate only the easiest part of software engineering, writing greenfield code. The remaining 80% is left as an exercise to the reader.

It sounds too good to be true. Why do you think llm is better in coding then in how education software should be designed?

I suspect that economies of scale are different for software and hardware. With hardware, iteration results in optimization of the supply chain, volume discount as the marginal cost is so much less than the fixed cost, and lower prices in time. The purpose of the device remains fixed. With software, the software becomes ever more complex with technical debt - featuritis, patches, bugs, vulnerabilities, and evolution of purpose to try and capture more disparate functions under one environment in an attempt to capture and lock in users. Price tends to increase in time. (This trajectory incidentally is the opposite of the unix philosophy - having multiple small fast independent tools than can be concatenated to achieve a purpose.) This results in ever increasing profits for software and decreasing profits for hardware at equilibrium. In the development of AI we are already seeing this-first we had gpt, then chatbots, then agents, now integration with existing software architectures.Not only is each model ever larger and more complex (RNN->transformer->multihead-> add fine tuning/LoRA-> add MCP), but the bean counters will find ways to make you pay for each added feature. And bugs will multiply. Already prompt injection attacks are a concern so now another layer is needed to mitigate those.

For the general public, these increasing costs will besubsidized by advertising. I cant wait for ads to start appearring in chatGPT- it will be very insidious as the advertising will be comingled with the output so there will be no way to avoid it.

[flagged]

1st dose is free

Being Linux is not a good thing imo, it took decades for tech like proton to run Windows games reliably, if not better as now, than Windows does. Software is still mostly develop for Windows and macOS. Not to mention the Linux Desktop that never took off, I mean one could mention Android but there is a large corporation behind it. Sure Linux is successfull in many ways, it's embedded everywhere but nowhere near being the OS of the everyday people, "traditional linux desktop" never took off

At least proprietary compilers were software you owned and could be airgapped from any network. You didn't create software by tediously negotiating with compilers running on remote machines controlled by a tech corp that can undercut you on whatever you are trying to build (but of course they will not, it says so in the Agreement, and other tales of the fantastic).

Apologies - I had the LLM generate the readme and it looks like it got a bit overzealous. I'll get some actual benchmarks and cost usage analysis going. I've tamed the README somewhat. Please check it out!

As far as I understand, you don't. You open Claude Code inside the repo and prompt `boot llmunix` inside Claude Code. The CLAUDE.md file tells Claude how to respond to that.

You just invented programming languages, halfway

Thought we already had that?

Yeah, let's bring back COBOL

Let me introduce to you.. python ;)

> but do you think they have access to better AI (in general) than consumers?

Absolutely. One of the top AI labs today is OpenAI, with ties to the US military, not least through Paul M. Nakasone, but also active contracts with the military, announced just a couple of days ago

> In June 2025, the U.S. Department of Defense awarded OpenAI a $200 million one-year contract to develop AI tools for military and national security applications. OpenAI announced a new program, OpenAI for Government, to give federal, state, and local governments access to its models, including ChatGPT. - https://en.wikipedia.org/wiki/OpenAI#Use_by_military

It would be foolish to assume those collaborations are just about API usage with the same models that consumer have access to, there is definitely deeper collaborations than that.

what consumer AI can send a vehicle a long distance, locate and track things of interest, and then decide to take actions against those things of interest?

Imagine a consumer AI that could go to the grocery store, find your favorite loaf of bread and bring it back.

Even looking at r/programming I felt they were less wary of AI, or even comparing the comments here vs those on YouTube for this video

Funny thing is that in more than one of the iron man movies the suits end up being bad robots. Even the ai iron man made shows up to ruin the day in the avengers movie. So it’s a little in the nose that they’d try to pitch it this way.

[flagged]

But you can generate synthetic data using a 3.0 program to train a smaller, faster, cheaper-to-run 2.0 program.

I’d take it a step further honestly. You need to be precise and exact but you also have to have enough domain knowledge to know when the LLM is making a huge mistake.

I think part of the problem is that people have the wrong mental models currently.

I am a non-software engineer and I fully expect someday to be a professional "vibe coder". It will be within a domain though and not a generalist like a real software engineer.

I think "vibe coding" in this context will have a type of relationship to software engineering the way excel has a relationship to the professional mathematician.

The knocks on "vibe coding" by software engineers are like a mathematician shitting on Excel for not being able to do symbolic manipulation.

It is not wrong but missing the forest for the trees.

I think Andrej is trying to elevate the conversation in an interesting way.

That in and on itself makes it worth it.

No one has a crystal clear view of what is happening, but at least he is bringing a novel and interesting perspective to the field.

The software does nothing but what you tell it to do. And if you can't figure out the limits, then it's probably a personal problem that you haven't solved for yourself yet.

I watched the entire talk, quite carefully. He explicitly states how excited he was about his tweet mentioning English.

The disclaimer you mention was indeed mentioned, although it's "in one ear, out the other" with most of his audience.

If I give you a glazed donut with a brief asterisk about how sugar can cause diabetes will it stop you from eating the donut?

You also expect deterministic outcomes when making analogies with power plants and fabs.

TIL. Thanks for sharing

Working code or not is irelevant. Heck, even human-in-loop (Tony-in-the-Iron-Man) is not actively the point. If we're going into "it's all about" territory then it's all about:

- training data - approximation of the desired outcome

Neither support a good direction for the complexity of some of the system around us, most of which require dedicated language. Imagine doing calculus or quantum physics in English. Novels of words would barely suffice.

So a context window as big as the training data itself?

What if the training data is faulty?

I'm confident you understand that working code or not doesn't matter in this analogy. Neither does LLMs reaching out for the right tool.

LLMs has its merits. Replacing concrete systems that require a formal language and grammar is not.

`1 + 1 = 2` because that's how maths works, not because of deja vú.

Untrue. I find problems with niche knowledge, heavy math, and/or lack of good online resources to be troublesome for AI. Examples so far I've found of consistent struggle points are shaders, parsers, and streams (in Nodejs at least)

Context window will solve a class of problems, but will not solve all problems with AI.

Agreed. This e-commerce example is quite a good highlight for LLMs.

Same can't be applied when your supplier needs 300 68 x 34 mm gaskets by the BS10 standard, to give a random, more precise example.

Maybe i'm not "vibing" enough, but i've actually been testing this recently. So far i think the thing "vibing" helps most with for me personally is just making decisions which i'm often too tired to do after work.

I've been coming to the realization that working with LLMs offer a different set of considerations than working on your own. Notably i find that i often obsess about design, code location, etc because if i get it wrong then my precious after-work time and energy are wasted on refactoring. The larger the code base, the more crippling this becomes for me.

However refactoring is almost not an issue with LLMs. They do it very quickly and aggressively. So the areas i'm not vibing on is just reviewing, and ensuring it isn't committing any insane sins. .. because it definitely will. But the structure i'm accepting is far from what i'd make myself. We'll see how this pans out long term for me, but it's a strategy that i'm exploring.

On the downside, my biggest difficulty with LLMs is getting them to just.. not. To produce less. Choosing too large of tasks is very easy and the code can snowball before you have a chance to pump the breaks and course correct.

Still, it's been a positive experience so far. I still consider it vibing though because i'm accepting far less quality work than what i'd normally produce. In areas where it matters though, i enforce correctness, and have to review everything as a result.

> Vibe coders are clients seeking an alternative, not developers.

Agreed. That's genuinely a good framing for clients.

Entertain me in an exercise:

First, instruct a friend/colleague of how to multiply two 2 digit numbers in plain English.

Secondly (ideally with a different friend, to not contaminate tests), explain the same but using only maths formulas.

Where does the prompting process start and where does it end? Is it a one-off? Is the prompt clear enough? Do all the parties involved communicate within same domain objects?

Hopefully my example is not too contrived.

Can we have it now and skip 3.0?

I think you should not turn things around here. Up to 2021 we had a vibrant software environment that obviously had zero "AI" input. It has made companies and some developers filthy rich.

Since "AI" became a religion, it is used as an excuse for layoffs while no serious software is written by "AI". The "AI" people are making the claims. Since they invading a functioning software environment, it is their responsibility to back up their claims.

The AI people are the ones making the extraordinary claims here.

Office software, CAD systems, Web Browsers, the list is long.

Can you point to any significant open source software that has any kind of significant AI contributions?

As an actual open source developer I'm not seeing anything. I am getting bogus pull requests full of AI slop that are causing problems though.

I would like to make an AI agent that directly interfaces with a processor by setting bits in a processor register, thus eliminating the need for even assembly code or any kind of code. The only software you would ever need would be the AI.

what he means is why are the tokens not directly machine code tokens

With vibe coding, I am under the impression that the only thing that matters for vibe coders is whether the output is good enough in the moment to fullfill a desire. For companies going AI first that's how it seems to be done. I see people in other places and those people have lost interest in the "how"

What, exactly? Educational courses?

Someone is thinking.

I'd argue that expressing contrarian sentiment can be substantive. Otherwise this would be an echo-chamber of agreement.

> it just seems to be an AI optimism word salad

Maybe it was missed, but in the beginning he said he was told the audience are mostly students about to enter the industry, so I feel like a lot of the talk is just establishing vocabulary, basic information about what LLMs are, analogies to get people to wrap their head around where in the workflow they can fit, and so on.

So while most of it seems obvious or relatively abstract, I think that's because of the target audience of the talk. I had that lens while watching the talk and while I cannot say my worldview has done a large change because of it, I understand it could be valuable to newer members of the ecosystem.

> Yeah, not sure I ever saw anything similar on HN before, feels very odd.

What exactly have you been seeing here on HN? I've been reading through most of the comments in this submission, since it was submitted yesterday, and none of it seems to be "fanboying" (maybe I misunderstand the term?) but discussions about where LLMs fit in the software development workflow.

Some people find some parts interesting, others obvious, others think he's selling something, others find the analogies lacking, but I've seen no "fanboy" comments like what parent seemed to exclusively see here.

It's been a multi-day like conversation where multiple people are trying to obtain the transcripts, publish the text as gospel, and now the video. Like, yes thank you but, holy shit.

You're making leap of logic.

Before water sanitization technology we had no way of sanitizing water on a large scale.

Before LLMs, we could still write software. Arguably we were collectively better at it.

Adults everywhere are using it to "cheat" at work, except there it's not cheating, it's celebrated and welcomed as a performance enhancement because results are the only thing that matter, and over time that will result in new expectations for productivity.

It's going to take a while for those new expectations to develop, and they won't develop evenly, just like how even today there's plenty of low-hanging fruit in the form of roles or businesses that aren't using what anyone here would identify as simple opportunities for automation, and the main benefit that accrues to the one guy in the office who knows how to cheat with Excel and VBA is that he gets to slack off most of the time. But there certainly are places where the people in charge expect more, and are quick to perceive when and how much that bar can be raised. They don't care if you're cheating, but you'll need to keep up with the people who are.

> The reality is that there's not a single critical component anywhere that is built on LLMs.

Remember that there are billion dollar usecases where being correct is not important. For example, shopping recommendations, advertizing, search results, image captioning, etc. All of these usecases have humans consuming the output, and LLMs can play a useful role as productivity boosters.

Even an LLM could tell you that that's an unknowable thing, perhaps you should rely on them more.

Agreed. Hell, even reviewing code can be fun and engaging, especially if done in person. But it helps when the other party can actually think, instead of automatically responding with "You're right!", followed by changes that may or may not make things worse.

It's as if software developers secretly hated their jobs and found most tasks a chore, so they hired someone else to poorly do the mechanical tasks for them, while ignoring the tasks that actually matter. That's not software engineering, programming, nor coding. It's some process of producing shitty software for which we need new terminology to describe.

I envy you for retiring. Good luck!

> Plus coding is the fun bit, reviewing code is the hard and not fun bit

To you. For others, it looks differently. And for yet others, they don't care about the coding nor the reviewing, they want to solve a particular problem.

I'd probably say I'm a programmer by accident. It's not that I love producing binaries by writing and compiling code, but I need to solve some particular problem that either is best solved by programming, or can only be solved by programming. "Programming by need" maybe is a fitting definition.

Doesn't mean I don't care about code quality, or good abstractions and having a reasonable design/architecture. But I'm focused on the end goal, having a particular problem solved, and coding is just the way there (sometimes).

>> A recurring theme presented, however, is that LLM's are somehow not controlled by the corporations which expose them as a service. The presenter made certain to identify three interested actors (governments, corporations, "regular people") and how LLM offerings are not controlled by governments. This is a bit disingenuous.

> I don't think that's what he said, he was identifying the first customers and uses.

The portion of the presentation I am referencing starts at or near 12:50[0]. Here is what was said:

  I wrote about this one particular property that strikes me
  as very different this time around.  It's that LLM's like
  flip they flip the direction of technology diffusion that
  is usually present in technology.

  So for example with electricity, cryptography, computing,
  flight, internet, GPS, lots of new transformative that have
  not been around.

  Typically it is the government and corporations that are
  the first users because it's new expensive etc. and it only
  later diffuses to consumer.  But I feel like LLM's are kind
  of like flipped around.

  So maybe with early computers it was all about ballistics
  and military use, but with LLM's it's all about how do you
  boil an egg or something like that.  This is certainly like
  a lot of my use.  And so it's really fascinating to me that
  we have a new magical computer it's like helping me boil an
  egg.

  It's not helping the government do something really crazy
  like some military ballistics or some special technology.

You are right in that the presenter identified "first customers", but this is mentioned in passing when viewed in context. Perhaps I should not have characterized this as "a recurring theme." Instead, a better categorization might be:

  The presenter minimized the control corporations have by
  keeping focus on governmental topics and trivial customer
  use-cases.

Me too. It was my favorite talk of the ones I saw.

> You can't just put things there any time you want - the RFC requires that they go through a registration process.

Actually, I can for two reasons. First is of course the RFC mentions that items can be registered after the fact, if it's found that a particular well-known suffix is being widely used. But the second is a bit more chaotic - website owners are under no obligation to consult a registry, much like port registrations; in many cases they won't even know it exists and may think of it as a place that should reflect their mental model.

It can make things awkward and difficult though, that is true, but that comes with the free text nature of the well-known space. That's made evident in the Github issue linked, a large group of very smart people didn't know that there was a registry for it.

https://github.com/AnswerDotAI/llms-txt/issues/2#issuecommen...

I put stuff in /.well-known/ all the time whenever I want. They’re my servers.

> You can't just put things there any time you want - the RFC requires that they go through a registration process.

Excuse me???

Wait ... but this is true.

Maybe I missed a source but I assumed it was somehow common knowledge.

https://en.m.wikipedia.org/wiki/List_of_Tesla_Autopilot_cras...