Overview

INTLLM is a local-first AI runtime that extends compatible local models (served via Ollama) with layered memory, optional live web retrieval, controlled browser tools, an Agent mode for coding workflows, and an OpenAI-compatible local API — all executing directly on your machine.

Standard local models operate in isolation: model → prompt → response. They lack persistent recall, verified real-time sources, safe system capabilities, and client integration. INTLLM provides the surrounding operating environment while ensuring user responses always retain top execution priority.

• Local First: Orchestrator, PostgreSQL storage, pgvector index, and web UI bind to 127.0.0.1.
• Privacy by Default: Conversations, vector embeddings, and API keys remain on disk in local PostgreSQL; no telemetry or silent cloud synchronization.
• Model Agnostic: Dynamically detects and manages models from your local Ollama inventory.
• Honest State Reporting: Dependency states (Ollama, PostgreSQL, pgvector) are transparently reported as connected, degraded, unavailable, or offline.
• Licensing: INTLLM is distributed under the PolyForm Noncommercial License 1.0.0. Copyright © 2026 Al Shahriar Sowan.

Architecture

A unified FastAPI backend orchestrates all operations. User requests arriving via the React frontend, the CLI, or the /v1 OpenAI-compatible endpoint route through the same pipeline.

User / Web UI / OpenAI-compatible API (/v1) / intllm CLI
                       │
                       ▼
         INTLLM Runtime (FastAPI Orchestrator)
    ┌─────────────┬──────────────┬──────────────────┬──────────────┐
    │             │              │                  │              │
    ▼             ▼              ▼                  ▼              ▼
 L0 Flash      L1 Hot        L2 Secondary       Tool Gateway   Agent Mode
  Brain         Cache          Brain            · Playwright   · Workspace
 (Routing)     (30m TTL)     (pgvector)         · Web Search   · Sandbox
                                                · Governor     · Permissions
                       │
             ┌─────────┴─────────┐
             ▼                   ▼
       Ollama Daemon       PostgreSQL + pgvector
     (127.0.0.1:11434)   (Local Durable Storage)

A low-priority Background Learner maintains memory indexes and verifies freshness in idle cycles, yielding resources immediately when interactive user queries arrive.

Agent Mode

Agent Mode is designed primarily for coding and development workflows. It bridges the model with a user-selected project workspace on disk, subject to strict sandboxing and granular permissions.

Workspace Selection & Boundary

The user explicitly chooses a directory on the local machine. This directory establishes the root boundary for all Agent operations. The Agent cannot perform operations outside this designated folder.

Workspace Security & Sandbox

• Path Traversal Guards: Every path referenced in a tool call is canonicalized and resolved against the workspace root. Any attempt to access paths outside the boundary is blocked with a 403 Forbidden error.
• Root Protection: The workspace root directory itself cannot be deleted or relocated by the Agent.
• Permission Gateway: File writes, modifications, deletions, directory removals, and terminal commands must pass through the permission gateway.
• Approval Levels: The user can choose Allow Once, Allow Session (to suppress repetitive prompts during a continuous task), or Reject.

Terminal Integration

Terminal commands execute with the workspace directory set as the working directory. Commands are monitored with configurable execution timeouts, and commands that attempt to touch external filesystem paths are flagged for explicit approval.

Layered Memory System

INTLLM performs external knowledge indexing and retrieval, not model weight retraining. Relevant memory items are retrieved, scored, and injected into context before prompt transmission.

L0 Flash Brain

A fast routing index over the knowledge store backed by pgvector embeddings and keyword tables. Evaluates query confidence and freshness before performing durable vector lookups.

L1 Hot Cache

Short-lived cache storing recently verified memory matches. Highly confident and fresh results are promoted here with a default 30-minute TTL, avoiding redundant vector distance calculations during active conversations.

L2 Secondary Brain

The durable persistent knowledge store in PostgreSQL with pgvector. Houses curated facts, workflows, lessons, and user preferences. Every item records its source URL/citation, creation time, verification timestamp, and decay policy.

Model Runtime & Hardware Tiers

INTLLM integrates with your local Ollama daemon at http://127.0.0.1:11434. It lists, pulls, and deletes models, and streams token completions via Server-Sent Events (SSE).

Any model family supported by Ollama (Qwen, Llama, DeepSeek, Gemma, Mistral, Phi, etc.) functions out of the box. If Ollama is unreachable, the runtime returns an explicit 503 service_unavailable status code.

Hardware-Based Recommendation Tiers

INTLLM detects host hardware and categorizes models into practical parameter tiers:

Tool Gateway

The language model never possesses direct operating system privileges. All tool calls traverse a rigorous security barrier: Schema Validation → Policy Check → Permission Decision → Execution → Result Sanitization.

Browser & Live Information

Live web retrieval (via DuckDuckGo or SearXNG) is strictly opt-in per request. Pass intllm_use_web: true in the API payload or enable the Web toggle in the UI. Retrieved content is sanitized and tagged with domain trust scores; local addresses and private subnets are blocked by SSRF filters.

The browser agent uses Playwright. If Playwright is not installed, the tool gateway reports it as unavailable rather than interrupting the chat session.

OpenAI-Compatible Local API

Integrate third-party developer tools, editor extensions, or custom scripts by pointing them at http://127.0.0.1:8000/v1.

GET  /v1/models                 # Lists installed Ollama models
GET  /v1/models/{model}         # Detailed model metadata
POST /v1/chat/completions       # Supports stream: true (SSE) and stream: false

Example request using curl:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer intllm_your_local_key" \
  -d '{
    "model": "qwen2.5:7b",
    "messages": [{"role": "user", "content": "Explain vector memory"}],
    "stream": true,
    "intllm_use_brain": true,
    "intllm_use_web": false
  }'

API keys use salted scrypt hashing. The full key is displayed only once upon generation and can be revoked instantly.

CLI Reference

After installation, the intllm command is registered in your PATH:

intllm                 # Start the local runtime and open the web UI
intllm start           # Same as above
intllm serve           # Run the API server only (no browser window)
intllm doctor          # Check PostgreSQL, pgvector and Ollama readiness
intllm --version       # Print the version (INTLLM 1.0.2)
intllm --help          # Print command reference

Security & Privacy

• Localhost Binding: The API binds to 127.0.0.1 by default. Exposing it across a LAN requires explicit configuration.
• Zero Remote Telemetry: No analytical beacons, crash pings, or background metrics are transmitted to any cloud server.
• Secret Redaction: Auth tokens and API keys are automatically scrubbed from runtime logs and terminal output.
• SSRF Guards: Live web retrieval rejects loopback, RFC 1918 subnets, and non-routable targets.
• Sandbox Isolation: Workspace operations cannot escape the selected project root.

Release Notes — INTLLM v1.0.2

Release Date: 2026-10-02 · Status: Production Release

INTLLM v1.0.2 represents the official production release, introducing Agent Mode, complete workspace sandboxing, enhanced model tier classification, production Windows packaging, and PolyForm Noncommercial licensing:

• Agent Mode: Coding-focused Agent mode with workspace selection, file reading/creation/editing/deletion, and monitored terminal commands.
• Workspace Security: Filesystem operations sandboxed strictly within the selected workspace root; path escape attempts rejected with 403.
• PostgreSQL Health & Diagnostics: Improved status reporting (connected, degraded, unavailable, offline) without silent failures.
• Persistent Chat History: Full multi-turn conversation and session persistence in local PostgreSQL.
• OpenAI-Compatible Local API: Robust /v1/models and /v1/chat/completions supporting streaming SSE and single-request memory/web toggles.
• API Key Management: Salted scrypt hashing, one-time reveal, label tracking, and instant revocation.
• Model Tier Classification: Automatic categorization into Potato (< 3B), Medium (3B–< 8B), and High (>= 8B) tiers.
• CLI Integration: Unified intllm command with --version, --help, doctor, serve, and start.
• Windows Installer: Official Inno Setup installer (INTLLM-Setup.exe) with per-user PATH registration and shortcuts.
• Production Packaging & PyPI: Standalone PyInstaller executable (INTLLM.exe), universal Python wheel (intllm-1.0.2-py3-none-any.whl), and source tarball.
• PolyForm Noncommercial Licensing: Licensed under the PolyForm Noncommercial License 1.0.0.

License & Legal

Source Availability: The source code of INTLLM is publicly available on the GitHub Repository under the PolyForm Noncommercial License 1.0.0. Public source visibility does not constitute an open-source license.

Frequently Asked Questions