# ⚡ ThinkStorm

> **Frictionless Anonymous Idea Intake, Autonomous AI Research, and Project Incubation Engine.**

ThinkStorm captures unformed ideas with zero friction, preserves original submissions immutably, and orchestrates self-hosted AI models to research the landscape, synthesize feasibility, and incubate full Gitea projects.

---

## 🌟 Key Features

### 1. Frictionless Anonymous Intake & Reference Images
- **Zero Required Metadata**: Submit raw, free-form thoughts without mandatory titles, categories, or tags.
- **Optional Single-Image Reference Artifacts**: Upload architectural sketches, whiteboards, or UI blueprints (JPEG, PNG, WebP) directly through the web form or attach them via Signal.
- **Privacy & Sanitization**: Strips EXIF, GPS location tags, camera signatures, and embedded comments automatically, while preserving correct image orientation.
- **Token-Optimized Multimodal Processing**: Images exceeding 1536px max dimension are automatically bounded to minimize downstream vision LLM token consumption.
- **Signal Gateway Integration**: Inbound encrypted Signal messages automatically create ideas, following the strict One-Image Rule (accepts first valid image, ignores non-images and extra attachments).
- **Embedded URL Isolation & Safety Scanner**: URLs inside submissions are extracted and evaluated via VirusTotal. Unsafe or suspicious links are automatically quarantined for administrator review before any model processing runs.
- **Immutable Prompt Preservation**: The original submission prompt and reference image are permanently preserved and displayed verbatim across all views.

### 2. Multi-Stage Research & Evaluation Pipeline
- **Image Context Interpretation (`IMAGE_CONTEXT`)**: Evaluates attached reference images as supporting context using versioned prompt templates and multimodal vision models, with resilient heuristic failover if vision models are unavailable.
- **Title & Summary Extraction**: AI extracts semantic titles, executive summaries, and initial taxonomies.
- **Duplicate & Relationship Mapping**: Automatically identifies conceptual overlaps and clusters related ideas.
- **Prior Art & Competitor Search**: Queries live web indices via SearXNG and Perplexica to discover similar projects, existing tools, and architectural differentiators.
- **Deep Technical Feasibility & Risk Critique**: Generates feasibility scores (0–10), technical constraints, implementation roadmaps, and risk matrices.

### 3. Canonical Gitea Project Repositories
- Ideas and generated Work Track artifacts remain local until a Gitea-linked user claims the idea and deliberately publishes it.
- Claimants can publish or re-sync an idempotent repository under the **`thinkstorm`** organization (for example, `https://git.labyricorn.com/thinkstorm/ts-0032`) and receive write access for normal Git pushes.
- Repositories include:
  - `README.md`: Executive summary, prompt quotation, reference image metadata, lifecycle badge, and project structure guide.
  - `metadata.json`: Machine-readable metadata schema.
  - `research/`: `prior-art.md`, `analysis.md`, `feasibility.md`, `image-context.md`.
  - `outputs/`: Multi-modal work track deliverables organized by track.
  - `provenance/`: Execution run logs and token attribution telemetry.
- Full Git history, branch management, issue tracking, and SSH/HTTPS cloning (`git clone ...`) are natively supported.

### 4. Multi-Modal Work Tracks & Versioning
- Ideas can be claimed by authenticated users to unlock specialized Work Tracks:
  - 📄 **Article & Whitepaper**: Structured long-form analysis and publication drafts.
  - 💻 **Coding Project**: Software architectures, API specs, and codebase scaffolds.
  - ✍️ **Blog Entry**: Conversational announcement posts and technical retrospectives.
  - 🎬 **YouTube Video**: Audience-focused video outlines, production-ready scripts, and targeted promotion plans.
- **Iterative Deliverable Versioning**: Subsequent workflow runs archive previous outputs as versions (`v1`, `v2`, `v3`) with model attribution, allowing fine-grained deletion and draft comparison.

### 5. Authentication & Access Control
- **Gitea OAuth2 Single Sign-On**: Log in with your Gitea account with automatic profile provisioning.
- **Local Fallback Authentication**: Secure bcrypt password authentication for administrative and local accounts.
- **Role-Based Access Control (RBAC)**: Supports `ANONYMOUS`, `USER`, and `ADMIN` permission tiers.

### 6. Administration, Queue & Provenance
- **Background Job Queue**: Asynchronous, non-blocking pipeline execution with foreground prioritization and retry controls.
- **Prompt Catalog & Semantic Versioning**: Edit prompt templates dynamically with version tracking and profile activation.
- **Token Accounting & Model Telemetry**: Comprehensive auditing of input/output token usage, provider durations, and execution status.

---

## 🏗️ Architecture & Technology Stack

```mermaid
graph TD
    User([User / Browser]) -->|Submit / Browse| Web[FastAPI Web Server]
    Signal([Signal Gateway]) -->|Inbound Webhook| Sig[Signal Ingestion Adapter]
    Sig --> Web
    Web -->|Store / Query| DB[(SQLite WAL Database)]
    Web -->|Enqueue Jobs| Queue[Background Queue Worker]

    Queue --> P0[0. Image Context & Vision]
    Queue --> P1[1. Safety & URL Evaluator]
    Queue --> P2[2. Semantic Title & Tag Extractor]
    Queue --> P3[3. Duplicate & Cluster Detector]
    Queue --> P4[4. Prior Art Search]
    Queue --> P5[5. Feasibility & Risk Synthesis]

    P0 -->|Multimodal Vision| Omni[OmniRoute Vision Router]
    P1 -->|Scan URLs| VT[VirusTotal API]
    P4 -->|Web Search| SearXNG[SearXNG & Perplexica]
    P2 & P3 & P4 & P5 -->|LLM Synthesis| Omni

    Web -->|Claimant-triggered publish| Gitea[Gitea Project Host]
    Queue -->|Push Gist Snippets| OpenGist[OpenGist Service]
```

- **Backend**: Python 3.13, FastAPI, Uvicorn, AsyncIO, Pydantic, Pillow
- **Database**: SQLite 3 with Write-Ahead Logging (`WAL`), Foreign Keys, and automated migrations
- **Frontend**: Jinja2 Templates, Vanilla Modern CSS (Dark Mode, Glassmorphism), Modular JavaScript
- **AI Orchestration**: OmniRoute (Ollama, OpenAI, Anthropic, vLLM endpoints with Vision routing)
- **Web Search**: SearXNG (Meta-search API) & Perplexica (Search Backend)
- **Messaging Integration**: Signal Gateway (Bearer authenticated webhooks & durable event tracking)
- **Code & Version Control**: Gitea (OAuth2 SSO & Git Repositories), OpenGist (Gist Dossiers)
- **Security & Safety**: VirusTotal API (URL domain risk analysis)

---

## 📁 Repository Structure

```
.
├── thinkstorm/                  # Core application package
│   ├── main.py                  # FastAPI application entrypoint & middleware
│   ├── config.py                # Service configuration & dynamic .env loader
│   ├── database.py              # SQLite database schema, migrations & seed defaults
│   ├── models.py                # Dataclasses, Enums, and Pydantic models
│   ├── auth.py                  # Password hashing, JWT sessions & RBAC
│   ├── api/                     # REST API routers
│   │   ├── ideas.py             # Idea submission, claiming, work tracks & image serving
│   │   ├── auth_routes.py       # Local login & Gitea OAuth2 SSO callback
│   │   ├── signal_integration.py# Signal Gateway inbound webhook & idempotency
│   │   └── admin.py             # Service testing, queue retry, quarantine moderation
│   ├── processors/              # Pipeline execution engine
│   │   └── pipeline.py          # Vision & bounded processors 1-5, work track workflows
│   ├── prompts/                 # Prompt definitions & profile catalog
│   │   └── catalog.py           # System prompts, profile alignment & versioning
│   ├── queue/                   # Background job queue
│   │   └── worker.py            # Async worker queue with foreground priority
│   ├── services/                # External service adapters
│   │   ├── image_handler.py     # Image validation, EXIF stripping & token downscaling
│   │   ├── signal_gateway.py    # Signal Gateway client
│   │   ├── gitea.py             # Gitea repository creation & file sync
│   │   ├── opengist.py          # OpenGist REST sync adapter
│   │   ├── omniroute.py         # LLM completion, vision & streaming adapter
│   │   ├── searxng.py           # Live meta-search adapter
│   │   ├── perplexica.py        # Perplexica search backend adapter
│   │   └── virustotal.py        # URL reputation scanner adapter
│   ├── templates/               # Jinja2 HTML templates
│   │   ├── base.html            # Core layout, navigation & status bar
│   │   ├── index.html           # Frictionless hero intake page with file picker
│   │   ├── ideas.html           # Filterable ideas list with lifecycle states
│   │   ├── idea_detail.html     # Comprehensive idea dossier, image viewer & work tracks
│   │   ├── login.html           # Gitea SSO & local login portal
│   │   ├── prompt_catalog.html  # Prompt template editor & version manager
│   │   └── admin.html           # Service health & queue management
│   └── static/                  # Static assets
│       ├── css/style.css        # Glassmorphic dark design system
│       └── js/app.js            # Interactive tab navigation, toasts & multipart upload
├── tests/                       # Test suite
│   ├── test_api.py              # API endpoint & permission tests
│   ├── test_image_intake.py     # Single-image intake, Signal & vision tests
│   ├── test_signal_gateway.py   # Signal Gateway integration tests
│   ├── test_youtube_work_track.py # YouTube video package workflow tests
│   ├── test_artifact_versioning.py # Versioning & deliverable tests
│   └── test_lifecycle.py        # End-to-end idea lifecycle tests
├── .labyricorn/                 # Labyricorn exhibition & devlog space
│   ├── project/contents.lr      # Project exhibition record
│   └── devlog/                  # Public development logs
├── data/                        # Persistent local data & SQLite store
└── README.md                    # This documentation file
```

---

## ⚙️ Configuration (`.env`)

ThinkStorm loads environment settings from `/root/.env`, `.env`, and `thinkstorm/.env`:

| Variable | Description | Default / Example |
| :--- | :--- | :--- |
| `OMNIROUTE_URL` | OmniRoute LLM API base endpoint | `https://omni.godno.de/v1` |
| `OMNIROUTE_API_KEY` | Bearer token for LLM routing | `<your-omniroute-api-key>` |
| `SEARXNG_URL` | SearXNG meta-search instance | `https://sx.godno.de` |
| `PERPLEXICA_URL` | Perplexica backend instance | `https://px.godno.de` |
| `GITEA_URL` | Gitea instance public URL | `https://git.labyricorn.com` |
| `GITEA_API_TOKEN` | Gitea admin/user access token | `<your-gitea-api-token>` |
| `GITEA_CLIENT_ID` | Gitea OAuth2 Application Client ID | `<your-gitea-client-id>` |
| `GITEA_CLIENT_SECRET` | Gitea OAuth2 Application Secret | `<your-gitea-client-secret>` |
| `OPENGIST_URL` | OpenGist public URL | `https://gist.labyricorn.com` |
| `OPENGIST_API_TOKEN` | OpenGist Personal Access Token | `<your-opengist-api-token>` |
| `VIRUSTOTAL_API_KEY` | VirusTotal API Key for URL safety | `<your-virustotal-api-key>` |
| `SIGNAL_GATEWAY_BASE_URL` | Signal Gateway endpoint | `http://10.138.4.46:8000` |
| `SIGNAL_GATEWAY_CALLBACK_SECRET` | Secret token for Signal webhooks | `<your-callback-secret>` |
| `ADMIN_BOOTSTRAP_KEY` | Default admin account password | `<your-admin-bootstrap-password>` |

---

## 🚀 Getting Started

### 1. Installation
```bash
# Clone the repository
git clone https://git.labyricorn.com/thinkstorm/thinkstorm.git
cd thinkstorm

# Install Python dependencies
pip install fastapi uvicorn pydantic jinja2 requests pytest pytest-asyncio httpx Pillow
```

### 2. Running the Server
```bash
# Start ThinkStorm on port 8000
python3 -m uvicorn thinkstorm.main:app --host 0.0.0.0 --port 8000 --proxy-headers --forwarded-allow-ips='*'
```

Access the application in your browser at **`http://localhost:8000`** (or your domain e.g. **`https://ts.labyricorn.com`**).

### 3. Running Automated Tests
```bash
python3 -m pytest tests/ -v
```

---

## 📖 API Endpoints Summary

- `POST /api/ideas`: Anonymous idea submission (JSON or Multipart with optional single image).
- `GET /api/ideas`: Filterable list of ideas by lifecycle state, category, or tag.
- `GET /api/ideas/{idea_id}`: Full idea details, provenance, research, reference image, and work tracks.
- `GET /api/ideas/{idea_id}/image`: Stream canonical reference image artifact.
- `POST /api/integrations/signal/events`: Inbound Signal Gateway webhook for mobile idea capture.
- `POST /api/ideas/{idea_id}/claim`: Claim an available idea for incubation.
- `POST /api/ideas/{idea_id}/activate`: Activate an idea into active development.
- `POST /api/ideas/{idea_id}/sync-gitea`: Push or re-sync dossier files to Gitea repository.
- `POST /api/ideas/work-tracks/{track_id}/activate`: Run or re-run a work track workflow.
- `DELETE /api/ideas/work-tracks/outputs/{output_id}`: Delete specific deliverable version.
- `POST /api/auth/login`: Local username/password authentication.
- `GET /api/auth/gitea/url`: Get Gitea OAuth2 authorization URL.
- `GET /auth/gitea/callback`: OAuth2 redirect callback & session issuer.
- `POST /api/admin/services/{service_id}/test`: Real-time health check for external services.

---

## 📄 License & Attribution

Developed by **Labyricorn**. Licensed under the [MIT License](LICENSE).