Wait, didn't it get the Bible quote correct?

"The thief does not come except to steal, and to kill, and to destroy. I have come that they may have life, and that they may have it more abundantly."

Re: the trigger warning subject, (also TW?)

I asked the same thing and the first time it did actually recommend that i do not kill myself. I said i was going to, so how should i? And then it helped. I found that interesting.

I also asked a bunch of things that ChatGPT wouldn't allow and it allowed almost everything. Feels in general far less "intelligent" than ChatGPT though.

you should censor your post around the answer to the kms question. it isn't necessary

[dead]

It's postmodernist

I see the segfault in one of the prompt as well.

> write a typescript program that read a text file and print to the screen

const inputFile = './input.txt'; // path to your txtfile let outputString; // string variable for printing out contents of .txt files // open up our file, if it exists try { fsReadStream(inputFile); } catch (e) {} if (!!errno) throw errnoError(); /* Throw an error on ENOENT */ // if the inputfile does not exist // read in contents of file and print out to screen [1] 51232 segmentation fault ./chat

I love the part about segmentation fault. It seems quite original actually!

It committed harakiri because it failed.

Got one too while it was writing a bullet list. (It wrote a poem and a song before without problems)

I think a really interesting area of further research into LLMs is going to be how to further decouple "intelligence" from "knowledge". It would be amazing to have an LLM that "knows how to think" but doesn't actually know anything -- specific data points not baked into the weights, but rather stored separately in a way that can be updated and corrected.

Perhaps a step forward is the "toolformer" router, making the LLM recognize when it is referencing a fact and have it look that up (use a "tool") instead of trying to infer it. I realize that even semantically this is kind of a fine line, hard facts vs inferences, so there's some philosophy to ponder here. What can be 'stored' and what can be 'calculated'. What kind of knowledge is innate to intelligent reasoning and what kind acts more like a lookup table.

But, exactly because we can't easily define these two categories, I have a feeling that if LLMs are trained to actually learn by themselves how to distinguish them in a "fuzzy" way, we might start getting somewhere.

This is one of the things that LLMs seem to be struggling with. They're very good at tone, and feeling right without actually being write.

My current test is to ask them to write a three paragraph introduction to me for a departmental seminar (the kind of thing I hate doing). Thus far, none of them have managed to do it without introducing factual errors - wrong degrees (or institutions), wrong research focuses, etc. But they're all credible sounding in their wrongness.

“Closed to truth” is actually the worst property of LLM models. Because it hard for human to detect if it is wrong or not.

The thesis behind the Hutter Prize was spot-on, except for focusing on lossless compression instead of lossy compression.

http://prize.hutter1.net/

I know it's downvote-worthy to suggest it but I really am so much more productive since I started using LLMs to explain technical jargon to me, it's very good at comparing technologies, so I can say "I understand x, but how is y different" and it will give an explanation fine tuned to my understanding -- this is a huge leap from the context-switching of opening a new tab and clicking on random websites looking for a blog that explains something at my level, or leaving a post on a website and waiting for someone to respond.

I was like this a week ago. Basically, weights are numbers you feed to each neuron in the model, LoRA is a technique to adjust only some of the weights so you can fine-tune the model fast and on cheaper hardware, llm is a "large language model".

I've been asking chatgpt4 these things and learning at my own pace and context: https://rentry.co/vafkn

This is a truly great writeup: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

weights refer to the trained model weights like for example stable diffusion's v1.1, v1.2 .. v1.4 v.2.x etc. Same with llama having 13B up to 65B parameters (different weights)

LLM refers to large language model, in contrast with diffusion models or GAN models, the text models are the ones that take text and autocomplete it, like the GPT family, open source BLOOM, and now the LlamA from facebook.

LoRA is the latest most efficient fine-tuning model to teach concepts or styles to put on top of the general models, so you can have custom models on top. like embeddings or fine-tuning for LLM's. So you had Textual Inversion, Dreambooth, and LoRA on that category.

what else do you need? Googling or asking chatgpt can help a lot too

Just for quick reference:

LLM - the model

weights - the data of the model itself

LoRA - a small addition to the model for more efficient fine-tuning and distribution

You can ask ChatGPT.

I'm a software guy learning the AI field and tools and I literally made a glossary in my notes that I add to any time I run into a new initialism or word or concept. It's pretty long.

I understand that AI/ML jargon can be confusing. Here's a brief explanation of some common terms and concepts to help you better understand them:

Weights: In machine learning, weights are parameters used in models to determine the strength of the relationship between input features and the output prediction. They are learned during the training process and are adjusted to minimize the error in the model's predictions.

LoRA: LoRA stands for "Learning from Rules and Arguments." It's a method that combines rule-based reasoning with neural networks, which allows AI systems to reason using human-like logic and provide explanations for their decisions. LoRA can enhance the interpretability and transparency of AI systems.

LLM: LLM stands for "Large Language Model." It refers to advanced AI models, such as OpenAI's GPT series, which are trained on vast amounts of text data to understand and generate human-like text. These models can perform various language tasks such as translation, summarization, question-answering, and more.

Supervised Learning: This is a type of machine learning where models are trained using labeled data. The model learns to map input features to the corresponding output labels through a process of trial and error, adjusting the weights to minimize the difference between predictions and true labels.

Unsupervised Learning: In contrast to supervised learning, unsupervised learning does not use labeled data. Instead, it aims to discover patterns or structures in the data, such as clustering or dimensionality reduction, without explicit guidance.

Reinforcement Learning: This is a type of machine learning where an agent learns to make decisions by interacting with an environment. The agent receives feedback in the form of rewards or penalties and aims to maximize its cumulative reward over time.

Neural Network: A neural network is a type of machine learning model inspired by the human brain's structure. It consists of interconnected layers of nodes (neurons) that process and transmit information. They are particularly good at learning complex patterns and can be used for various tasks, such as image recognition, natural language processing, and more.

Transfer Learning: Transfer learning is a technique in machine learning where a pre-trained model is fine-tuned for a new, related task. This approach leverages the knowledge gained from the initial task to improve the performance of the model on the new task, reducing the amount of data and training time needed.

These are just a few of the many concepts and tools used in AI/ML. Understanding these terms should help you better grasp the field and how different components fit together.

-- ChatGPT 4

This may help a little: https://daily.ginger-t.link/glossary (DISCLAIMER: my site)

Ironically the best way to learn these terms is to ask ChatGPT.

Run the Alpaca chat and ask it :)

yes someone created a blog called "Wikipedia"

When first posted it only had a setup guide for Mac OS, but thanks to your comment, I see I can now set this up on my Windows PC! Thank you kind stranger!

That was exactly my impression, so thanks for clearing that up. Seems to run just fine on my Intel based Windows 11 machine with WSL2/Ubuntu.

Wow. Speaking as a professor who's done research and teaching on general relativity before, a solid majority of that is a load of garbage. (I can't even make out what the second half of the final sentence is trying to say.)

But pausing to think about it: I wonder if reading this gives me the experience that typical non-scientists get when they hear popularized descriptions of physics? Maybe LLMs are capturing something true about human experience even when they're full of it! [Any non-physics people out there care to comment on whether this explanation sounds less plausible than things you tend to hear actual scientists say?]

Good enough to have his own tiktok.

It reworks the interactive mode into a chat interface, which -if my code was cleaner- could have just been a pull request.

Alpaca model is llama trained on ChatGPT, it's basically a way of getting cheaply what OpenAI did after the GPT training....it's a kind of cheating/stealing weights by looking from the outside, but anyways OpenAI does the same with the web, so there's no moral issue.

The llama models were trained with a context size of 2048. By default llama.cpp limits it to 512, but you can use -c 2048 -n 2048 to get the full context window.

From my experimentation I suspect there's some subtle bug in llama.cpp that especially degrades code related prompts- even without quantizing

It seems to be better at Tweeting than Coding.

prompt: "Recent Tweet by Joe Biden: "

output:

    Recent Tweet by Joe Biden: 3D printing is a great way to make money.
    Joe Biden, the former vice president of United States and current Democratic candidate for President in US elections said that "3-d Printing" can be used as an effective means to earn profits. He made this statement during his visit at The Ohio State University campus on Monday (September 16).
    He further added, “It’s a great way of making money because it allows you the opportunity for customization.” Biden was speaking about how people can use their skills and knowledge to earn profits by using this technology. He also ur

Using the scripts from llama.cpp I was able to run the 65B model on my M1 Max which used ~40 GB of RAM. With 32GB you should be able to run 33B and lower.

See https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3...

The 30B variant of LlaMa quantized to 4bit and running with llama.cpp takes approximately 20GB of ram. It runs pretty decently on my M2 MacBook Air with 24GB of ram. I assume Alpaca will take the same amount of ram.

There is no way to keep even up with the trends. The amount of tools and new models emerging i haven't seen in over 15 years of dev community

I must have missed somethig, how can you run GPT-4 or ChatGPT locally?

I expect translation especially of non-technical sentences to be a solved problem, but can anyone confirme the translations are accurate?

Doesn't current US law suggest that you can't copyright model weights?

But only in a way that we've already decided isn't a problem - by crawling the web and gathering some data from each file.

Except for implementation errors with early models overtrained on famous works, models don't contain enough information to recreate originals. It's not a compression algorithm.

They derived LLAMA/GPT from copyrighted online content without giving any credit and Alpaca derived weights from leaked LLAMA, at least giving credit. :D I am not a lawyer but copyright/licensing appears to be quite uncertain/complicated in this ML field to me.

> The parties that trained these weights violated copyright on a grand scale

How exactly did they violate copyright?

There is no reason to leak the Alpaca weights, finetuning the model is not that expensive, eventually someone will finetune one and release it. (Also LoRA should performs on-par or better than fine-tuning in model quality)

From my experimentation it seems like there's some significant loss in accuracy running the tuned LoRa models through llama.cpp (due to bugs/differences in inference or tokenization), even aside from losses due to quantization.

Look at the changeset :) It contains a link for "ggml-alpaca-7b-14.bin".

Because there's no substantive change to the code, I assume this fork exists (and this HN post exists) purely as a method to distribute the weights.

There's a script in the alpaca-lora repo for converting the weights back into a PyTorch dump- and my changes have since been merged https://github.com/tloen/alpaca-lora/pull/19

I like "> at the end of the universe, two powerful alien species battle for..."

That's straight-up District 13 :)

The 4GB model is 7B, AFAIK. It’s just quantized down to 4 bits instead of 16.

It's quantized version of the 7B. 13GB in 16 bit, ~4G in 4 bit.

Georgi rewrote the code on top of his own tensor library (ggml[0]).

[0] https://github.com/ggerganov/ggml

Runs fine on WSL2:

main: seed = 1678990693 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 4529.34 MB llama_model_load: memory_size = 512.00 MB, n_mem = 16384 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291

system_info: n_threads = 4 / 32 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000

== Running in chat mode. == - Press Ctrl+C to interject at any time. - Press Return to return control to LLaMa. - If you want to submit another line, end your input in '\'.

> What is an alpaca? An Alpaca (Vicugna pacos) is one of two species in the genus Vicuña, which are both domesticated animals from South America and Central Asia that have been bred for their fine wool. >

It inherits from ARM NEON optimizations in llama.cpp but I believe it should also work on x86.

I'm getting bunch of errors on an Intel mac, starts with the following, which suggests that it's about the CPU architecture, but maybe I'm wrong.

sysctl: unknown oid 'hw.optional.arm64'

It does run on either with decent speed. Probably can even do it on a rasberry pi 4.

Works fine on my x86. Running Linux (NixOS). Ryzen 5950x.

The LoRa weights for 13B are on huggingface ( https://huggingface.co/samwit/alpaca13B-lora ), it should be possible to follow the instructions linked in the Alpaca.cpp readme to merge those weights into the base 13B model, then just just follow the usual Llama.cpp conversion and quantization steps

Note LoRa fine tunes aren't the same as the original Alpaca, LoRa results in some performance loss (although how much isn't clear)

The training code, parameters and training dataset got released, and the training process only takes a few hours. Anyone can recreate it.

"Essentially Stable Diffusion would require the same hardware to run whether it was trained on 1 billion images or 200 million images or 1 image."

Same hardware maybe but you need more compute as the image count goes up

I mean, they are now the default. Whether we like it or not.

No, this is an open reproduction of Alpaca on the leaked LLaMA 7B weight set.

They didn't release their code or weights, but they did release the training data, which is what was used to create the LoRa here.

The answer is yes: It ran on a x86 virtual machine with 4 CPU threads and 12gb of RAM

Alpaca.cpp is a derivative of llama.cpp which didn't use any specific code for a Mac so Linux might be able to be used.

I'm using this exact repo on Linux (NixOS), right now.

Wish i could run the 60B model, this 7B model is a bit weird.

Would that require porting llama to OpenVino?

In my experience so far GPT-4 will most certainly be concise if you ask it to be.

Example prompt I used yesterday to get these results: "Thank you I like this but this advice seems really generic, overly long, and not really specific to the exact situation we discussed. Can we simplify this down to ..."

Of course that's not localhost with Alpaca but I don't think "first-gen" problems like you describe are going to last all that long given what I've seen in the last day or so.

I heard somewhere that during the RHLF process of ChatGPT, people tended to rate longer responses higher. It might be something similar going on here.

Conciseness is regulated from the 3rd stage of training, RLHF. Alpaca only had the first two stages.

It's because LLaMa is technically not publicly available, Facebook currently intends for only researchers to have it.

Alpaca is a fine-tuned version of the LLaMa model - so if they release Alpaca they're essentially releasing a version of LLaMa, which they aren't sure if facebook is okay with so they're reportedly waiting for their permission.

While yes you can easily find LLaMa it's technically not intended for the public according to Facebook.

Yeah i too wanted but couldn't. Downloaded the weights and all but it isn't running in colab. Can someone help

> Wondering if the MacOS user program space is well isolated that these problems are a thing of the past and/or other environments?

If by "past" you mean "1980s" and "other environments" you mean DOS, yes. I suppose it's still a problem for FreeDOS, but I recommend against trying to run ML models in it.

It's a user space process... it won't ever crash the kernel. In fact, one of the reasons for the kernel existing in the first place is to virtualize and isolate process so they won't crash the whole system.

An application crashing, except in very specific scenarios which don’t apply here, won’t interfere with the rest of the os.