Platforms

Virtual Environments and Dependency Management

LLM libraries have complex dependency trees, and version conflicts are a daily reality. Isolating each project in its own virtual environment is not optional; it is essential for reproducibility and sanity. Code Fragment G.2.1 below puts this into practice.

Option 1: venv (Built-in)

This snippet shows how to create and activate a virtual environment using Python's built-in venv module.

# Create a virtual environment
python -m venv llm-env

# Activate it
# On Linux/macOS:
source llm-env/bin/activate
# On Windows:
llm-env\Scripts\activate

# Install packages
pip install torch transformers datasets accelerate

# Freeze dependencies for reproducibility
pip freeze > requirements.txt

# Recreate environment elsewhere
pip install -r requirements.txt

Option 2: Conda (Recommended for GPU Work)

This snippet shows how to set up a Conda environment with GPU support for LLM projects.

# Create an environment with a specific Python version
conda create -n llm-project python=3.11
conda activate llm-project

# Install PyTorch with CUDA support (conda handles CUDA toolkit)
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

# Install Hugging Face libraries via pip (within conda env)
pip install transformers datasets peft trl

# Export environment
conda env export > environment.yml

# Install uv
pip install uv

# Create a virtual environment and install packages
uv venv llm-env
source llm-env/bin/activate
uv pip install torch transformers datasets

Option 3: uv (Fast Alternative)

uv is a modern, Rust-based package manager that is dramatically faster than pip. It is gaining traction in the ML community for its speed and compatibility. Code Fragment G.2.3 below puts this into practice.

# Create a virtual environment with uv (Rust-based, ~10x faster than pip)
# uv resolves and installs hundreds of packages in seconds.
# Install uv itself first:
curl -LsSf https://astral.sh/uv/install.sh | sh

# Then create + activate the venv
uv venv llm-env
source llm-env/bin/activate   # Linux/macOS
# llm-env\Scripts\activate    # Windows

# Install dependencies (uv reads pyproject.toml or requirements.txt)
uv pip install torch transformers datasets accelerate

Jupyter Notebooks and Google Colab

Notebooks are the preferred environment for exploration, prototyping, and learning. They let you run code cell by cell, inspect intermediate results, and mix code with documentation. Nearly every LLM tutorial and research artifact is published as a notebook. Code Fragment G.3.1 below puts this into practice.

Local Jupyter Setup

This snippet shows how to install and launch Jupyter for local notebook development.

# Install Jupyter in your virtual environment
pip install jupyterlab

# Launch the notebook server
jupyter lab

# Or use the classic interface
jupyter notebook

Google Colab Tips

• Free GPU access: Go to Runtime > Change runtime type > select T4 GPU. The free tier gives you a T4 with 15 GB VRAM, enough to run 7B parameter models in 4-bit quantization.
• Mounting Google Drive: Use from google.colab import drive; drive.mount('/content/drive') to access your files and save checkpoints persistently.
• Installing packages: Use !pip install transformers peft trl at the top of your notebook. The exclamation mark runs shell commands from within a notebook cell.
• Secrets management: Use Colab's Secrets panel (the key icon in the left sidebar) to store API keys instead of hardcoding them in cells.
• Session limits: Free Colab sessions disconnect after idle periods and have usage quotas. Save checkpoints frequently to Google Drive.

Hardware Requirements for LLM Work

Large language models earn their name. Even "small" open models have billions of parameters, and each parameter requires memory. Understanding hardware requirements before you begin will save you from frustrating out-of-memory errors.

GPU (Graphics Processing Unit)

A GPU is the most important piece of hardware for LLM work. NVIDIA GPUs with CUDA support are the standard; AMD and Intel GPUs have improving support but lag in ecosystem maturity.

RAM (System Memory)

You need sufficient system RAM to load model weights before transferring them to the GPU. A general rule: have at least as much system RAM as GPU VRAM, plus extra for your dataset and operating system. For most setups, 32 GB of system RAM is comfortable; 16 GB is workable but tight.

Disk Space

Model weights are large files. A 7B parameter model typically occupies 13 to 14 GB on disk in float16, or about 4 GB in 4-bit quantized GGUF format. If you plan to download several models, allocate at least 100 GB of free disk space. SSDs are strongly preferred over HDDs for loading speed.

CUDA and Driver Setup

CUDA is NVIDIA's parallel computing platform, and it is required for GPU-accelerated deep learning. Getting the right combination of driver version, CUDA toolkit version, and PyTorch version is one of the trickiest parts of setup.

Step 1: Install the NVIDIA Driver

Download the latest driver for your GPU from the NVIDIA Driver Downloads page. On Linux, you can also install via your package manager: Code Fragment H.2.1 below puts this into practice.

# Ubuntu/Debian
sudo apt update
sudo apt install nvidia-driver-550

# Verify installation
nvidia-smi

# Verify CUDA installation
nvcc --version
# Check that PyTorch sees CUDA
python -c "import torch; print(torch.cuda.is_available())"

The nvidia-smi command should display your GPU name, driver version, and the maximum CUDA version your driver supports.

Step 2: Install CUDA Toolkit (Optional if Using Conda)

If you install PyTorch via Conda, the CUDA toolkit is bundled automatically (see Section H.3). If you prefer a system-wide CUDA installation, download it from the NVIDIA CUDA Toolkit page. Code Fragment H.2.2 below puts this into practice.

# Verify the CUDA toolkit version and confirm PyTorch detects the GPU.
# nvcc is the CUDA compiler driver; its version must match the PyTorch build.
nvcc --version
# This one-liner imports PyTorch and prints True if a CUDA GPU is available
python -c "import torch; print(torch.cuda.is_available())"

Cloud Compute Options

Not everyone has a beefy GPU at home. Cloud platforms offer GPU access at various price points, from free to enterprise-grade. Here is a comparison of popular options.

Google Colab Quick Start

This snippet installs the required packages in a Google Colab notebook and verifies GPU availability.

# First cell: install libraries
!pip install -q transformers peft trl bitsandbytes accelerate datasets
# Second cell: check GPU
!nvidia-smi
# Third cell: mount Google Drive for persistent storage
from google.colab import drive
drive.mount('/content/drive')
# Fourth cell: set your Hugging Face token
from google.colab import userdata
hf_token = userdata.get('HF_TOKEN') # stored in Colab Secrets

IDE Setup and Editor Integrations

Once your CUDA stack and Python libraries are in place (sections I.2 through I.5), the next decision is which editor you will live inside. The choice determines how productive you are with the workflows LLM engineering imposes: notebook-plus-script hybrid development, remote GPU instances, type-checked autocompletion against Hugging Face and PyTorch, and AI-in-IDE assistants that have become table stakes.

This section covers the four editors that dominate as of 2026: VS Code (default), Cursor (AI-first fork), PyCharm (heavy refactoring), Neovim (keyboard-driven). Deep AI-in-IDE coverage is in Chapter 50.2 (Vibe-Coding); this section is the configuration counterpart. Notebook UX is in Section 5.2 (Libraries & Frameworks); API-key handling for AI assistants is in Section 14.1 (Platforms).

1. VS Code: The Default

VS Code (Microsoft, free, MIT-licensed core) is the most common editor for Python and LLM work as of 2026: ~70% of Stack Overflow's Python respondents use it. Defaults are good; the extension ecosystem is the largest; remote development (SSH, containers, WSL, Codespaces) works without configuration.

Python (Microsoft) provides interpreter selection, virtual-env detection, debugger, and test runner. Pylance powers type-checked autocompletion against Hugging Face Transformers, PyTorch, LangChain stubs (which landed across 2024). Set "python.analysis.typeCheckingMode": "basic" in .vscode/settings.json. Jupyter provides notebook UI with kernel selection, variable explorer, and cell debugger; with %load_ext autoreload you iterate on library code and re-run cells without kernel restart.

Ruff (Astral, Rust) is the 2025 consensus formatter/linter, 10-100x faster than flake8 + black + isort. Configure via pyproject.toml. GitLens adds inline blame and richer diffs, useful for tracing why a hyperparameter changed three commits ago when an eval regresses.

AI-in-IDE options: GitHub Copilot (paid, most polished), Continue (open source, BYO-model), Cody (Sourcegraph, code-search-aware), Codeium (free tier). All four are VS Code extensions; the deep comparison is in Chapter 50.2.

// .vscode/settings.json for LLM engineering
{
  "python.analysis.typeCheckingMode": "basic",
  "python.analysis.diagnosticMode": "workspace",
  "[python]": {
    "editor.defaultFormatter": "charliermarsh.ruff",
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {"source.fixAll.ruff": "explicit"}
  },
  "jupyter.askForKernelRestart": false,
  "notebook.output.textLineLimit": 200,
  "files.exclude": {"**/__pycache__": true, "**/.ipynb_checkpoints": true},
  "git.autofetch": true
}

2. Cursor: VS Code with Native AI

Cursor (Anysphere, freemium) is a VS Code fork integrating AI directly rather than via extension. Differences from VS Code + Copilot: multi-file edits (Cmd-K across N files), Composer (agent mode for multi-step changes), codebase chat against embedded code search, and tab completion with more code context. Supports hosted models and BYO-API-key for Claude / GPT / others.

Trade-off: closed-source vs VS Code's MIT core; you depend on Anysphere's cadence. Most VS Code extensions work in Cursor. 2025 production usage at AI-native companies (Vercel, Replit, Perplexity) is driven by Composer for multi-file refactors.

3. PyCharm: The Heavy IDE

PyCharm (JetBrains; Pro and Community editions) fits two situations: large refactoring projects where cross-file rename / inspection matter, and teams already in the JetBrains ecosystem. Scientific Mode (Pro only) provides tensor inspection and a plotting pane. JetBrains AI Assistant (paid) and the 2025 Junie agent provide a Copilot-comparable experience with deeper refactoring-engine integration.

Strongest feature: the remote interpreter. Point a remote SSH interpreter at a RunPod or Lambda GPU; the IDE runs code there with full debugger, profiler, and Scientific Mode while you edit locally. Heavier setup than VS Code Remote-SSH, but more cohesive for week-long fine-tuning projects.

4. Neovim: The Terminal Native

Neovim with LSP fits terminal natives with a Lua config. The 2024-2025 ecosystem matured: nvim-jupyter and jupytext for notebooks, nvim-dap for debugpy, copilot.lua and codecompanion.nvim for AI. Trade-off: configuration complexity in exchange for an editor that does exactly what you want over any SSH session.

5. Comparison: Which to Pick

6. Recommended Starter Setup

For 90% of readers, the recommended setup is:

1. Install VS Code (or Cursor if committing to AI-first from day one; switching later is easy).
2. Install Python, Pylance, Jupyter, Ruff, and GitLens extensions.
3. Pick one AI assistant: Copilot (paid), Continue (free, BYO-model), or Cursor's native AI.
4. Drop the .vscode/settings.json above into your project.
5. For remote GPU work, install Remote-SSH and verify you can open a folder on the remote machine.

A typical configured workspace: editor pane shows a Python file with type-checked autocompletion against the Hugging Face stack; notebook pane hosts a kernel where you iterate on training loops; bottom terminal runs nvidia-smi against the remote instance; an AI-assistant chat docks on the right. The whole thing is reproducible because .vscode/settings.json and .devcontainer/devcontainer.json are in Git.

7. AI-in-IDE Configuration Hygiene

AI assistants read your code, file paths, sometimes terminal output, and send them to a remote API. For LLM engineering this matters more than for typical Python work because your source tree contains proprietary prompts, embeddings, and eval-set examples. Three defenses: (1) configure workspace exclusions for data/, secrets/, anything with API keys; (2) use enterprise / business tiers (Copilot Business, Cursor Business) which disable training-on-your-code by default; (3) verify the assistant respects .gitignore.