Hosted open models vs download to your machine

Ken watched a progress bar for 41 minutes while qwen3:8b landed as a 4.7 GB file. He told legal the vendor PDFs would stay on the laptop. Then the local reply took 28 seconds per sentence, so he flipped a control in the same app that offered a cloud model. The 86-page NDA packet went out on the next prompt. Ollama’s public site currently advertises both “on your computer” and “in the cloud.” Llama in the name of a weight file is not a location. Legal heard “local.” The packet heard a host.

This is OS2 of Open-source AI explained. OS1 split open weights, open source AI, and free-to-use chat. This page splits where those weights run: a host you send prompts to, or a download you run yourself. Groq is not Grok. A playground that serves Llama is not your fan. Re-check product names the week you toggle anything that says Cloud.

What you’ll learn

Hosted open models vs a file on your disk, in one picture
Who Groq, Together, Fireworks, and OpenRouter are (and who they are not)
What Ollama, LM Studio, and llama.cpp are for, including Ollama’s cloud path
A location picker you run before you pick a model family
How Ken’s 4.7 GB download still leaked an 86-page packet

Location is the noun

Hosted open models send the prompt off-device. Local downloads can stay on disk if you skip the cloud toggle. Groq is not Grok. A Llama n…

Open weights are a file (or a pile of shards) plus a license. You can send that file to a company that already has GPUs, or you can load it on the machine in front of you. The license does not change when the fan spins. The privacy story does. Hosted means the prompt leaves. Local means the prompt can stay, if you do not enable a cloud mode and if the disk is yours.

Closed products (ChatGPT, Claude, Gemini, Grok) are also hosted. They are just not this series’ download story. If Ken only needed a summary of a public blog post, the chooser’s writing door was enough. He used open-weight theater because the packet felt “too sensitive for ChatGPT” and then sent it to a different host with a friendlier logo.

Place	What you run	Where the prompt goes	You pay with
Hosted open model	Groq, Together, Fireworks, OpenRouter, a vendor chat that lists Llama/Qwen/DeepSeek	Their building	Tokens, credits, or a seat
On your machine	Ollama / LM Studio / llama.cpp, a GGUF or similar	Your RAM, if you stay offline	Hardware, heat, your time
Closed chat	ChatGPT, Claude, Gemini, Grok apps	Their building	Free caps or ~$20 plans (see P7)
Same app, cloud toggle	Ollama cloud models, “run on our servers”	Not your laptop, even if you already downloaded 4.7 GB	Whatever that toggle’s plan is

Hosted: fast GPUs, still a vendor

A handful of companies rent inference. Groq (the LPU company, spelling with a q, not xAI’s Grok) is known for speed on a curated set of open models. Together and Fireworks rent GPUs and tend to list more families, fine-tunes, and sometimes dedicated deployments. OpenRouter is a switchboard: one key, many backends. Prices as of recent public pages sit in a wide band per million tokens. Do not copy a blog’s $0.09 into a budget. Open the host’s pricing the week you start.

Hosted is the right door when you want a 70B-class model you cannot fit in a laptop, or you want an API without buying a GPU. It is the wrong door when the sentence you told legal was “nothing leaves.” Read retention and training policies. Fireworks has advertised zero-data-retention options; others differ. Assume logs exist until a contract says otherwise. P4 is still the paste test: would you email this PDF to that vendor.

Playground chats on those sites feel like ChatGPT with a Llama badge. They are not local. They are also not Groq-the-chatbot. If someone says “we moved to Groq,” ask whether they mean the inference cloud or they misspelled Grok. That mix-up is how a slide gets a wrong vendor and a wrong privacy story in one breath.

On your machine: the 4.7 GB is the beginning

Ollama is the one-command path a lot of people actually finish. LM Studio is a friendly window. llama.cpp is the engine under many of those apps. You download a quantized file (GGUF and friends), you load it, you chat. OS3 will do size and RAM. Today: a 7B-class or 8B-class file is a laptop animal. A 70B file is a different animal. Ken’s 4.7 GB pull was the easy half. The 28-second sentences were the hardware talking.

Local wins when the file must not leave and you will live with quality and heat. It loses when you expected ChatGPT-Plus prose from a small quantized model, or when the same app offers a cloud shortcut and you take it. Ollama’s homepage currently sells computer and cloud. That is useful. It is also how Ken’s toggle happened. Look at the model line. If it says cloud, the 41-minute download did not apply.

Pick a location first

Pick a location before you pick a model: allowed to leave, need a big model, need offline, or just writing

May this text leave the building. If no, local or nothing.
If yes, do you need a large open model today. Hosted. Read the host’s policy.
If you need offline or idle-time cheap, download a size your machine can hold. Feel the fan on a toy prompt first.
If the job is a thank-you, use the chooser and a closed chat plan. Do not invent a local stack for a paragraph.

A check you can run in the terminal after you think you are local. This is a shape, not a promise that every runner uses the same flag.

# After a local pull, stay explicit.
# Ollama-style (flags and names move; read `ollama --help` this week):
ollama list
# If a model line says cloud / remote, do not paste NDAs into it.

# Toy prompt only:
# "Summarize this public sentence: The warehouse delay was ours."
# Time it. If you then switch to a cloud id because it was slow,
# you changed location. Tell legal the new location or do not paste.

What that check is for: Ken listed models after the fact and found a cloud id sitting next to the 4.7 GB file. The list is the map. The download progress bar is not.

Worked example: 41 minutes, 86 pages, one toggle

Claim Ken made	What was true	What legal needed
“We run Llama locally”	He downloaded a Qwen 8B-class file, then used a cloud id	The host name and the retention policy
“The 4.7 GB file is the privacy control”	The file sat unused after the toggle	Whether the prompt left
“Open source so we’re fine”	Weights plus a license; see OS1	The PDF of that license, plus location
28 seconds per sentence	Hardware tax of local 8B on his laptop	A decision: wait, smaller model, or a contracted host

The fix is boring. Turn cloud off. Summarize a two-page public PDF first. If 28 seconds is unacceptable, either keep the packet off models or sign a host with a real agreement. Do not split the difference with a toggle you do not mention in the slide.

When hosted is the honest cheaper path

Builders sometimes rent Together or Fireworks because a weekend GPU quote is worse than a token bill under 100 million tokens a month. That math is a later ops post. For an analyst who wanted a private summary, hosted open models are still vendors. They can be cheaper than Claude API. They are not “free because Qwen.” Local is cheaper at idle if the machine already exists and you accept the quality. P7’s $20 closed plan is cheaper than a GPU if the job is writing.

OS3 is size and quantization. OS8 is “easiest hosted chat UI.” Do not skip this location split to chase a model card. Family maps (Llama, DeepSeek, Qwen) come after you can say hosted or local without blushing.

Common mistakes

Saying Groq when you mean Grok, or the reverse.
Calling a Groq/Together playground “local” because the model is Llama.
Downloading 4.7 GB, then using the cloud id in the same window.
Promising legal “open source local” without a license PDF or a location.
Building a local stack to write the thank-you from Part 1.
Ignoring that Ollama (and friends) may ship both modes.

How to practice this week

Write one sentence: hosted or local, and the app name. Run a toy prompt only. If you enable cloud, write the host on the same sticky note. Next: OS3, will your laptop run it. The privacy chooser remains P4. Setup and cards: P7.

Quick recap

Hosted open weights still leave the building. Local can stay, if you do not toggle cloud.
Groq ≠ Grok. Llama in the name ≠ on your disk.
Pick location, then model. The 4.7 GB bar is not a privacy control by itself.

Sources

Ollama (local and cloud; confirm which mode you are in)
Groq (inference cloud; not xAI Grok)
Together AI and Fireworks AI (hosted open models)
LM Studio
llama.cpp
OSI: Open Weights and AMS OS1

Keep going

Same lessons in your feed

Short diagrams and hooks on Instagram, X, and Facebook.

@analyticsmadesimple Follow @AnalyticsSimply Follow Analytics Made Simple Follow