Add project exhibition, development logs, and complete master README

This commit is contained in:
2026-08-21 13:58:15 -07:00
parent 2c95567e28
commit 94ff1e4408
8 changed files with 444 additions and 0 deletions
+25
View File
@@ -0,0 +1,25 @@
# Labyricorn Publishing Space for ThinkStorm
This directory contains the authoritative publishing records for ThinkStorm's project exhibition and public development log on Labyricorn.
## Directory Structure
```
.labyricorn/
├── agents.md # Top-level publishing and synchronization guidelines
├── README.md # Overview of the Labyricorn publishing space
├── project/ # Project exhibition page
│ ├── AGENTS.md # Project exhibition rules
│ └── contents.lr # Lektor project exhibition record
└── devlog/ # Public development log
├── AGENTS.md # Devlog authoring rules
├── contents.lr # Devlog index record
└── <slug>/ # Chronological development milestone entries
└── contents.lr
```
## Maintenance Rules
- **Authoritative Source**: Content defined here is mirrored by the Labyricorn publishing engine.
- **Lektor Records**: Content uses `schema_version: 1` and Lektor record format (`contents.lr`).
- **No Secrets**: Never store API tokens, passwords, or credentials in this directory.
+13
View File
@@ -0,0 +1,13 @@
_model: devlog
---
schema_version: 1
---
title: ThinkStorm Development Log
---
summary: Chronological development narrative, major architectural milestones, and platform engineering updates for ThinkStorm.
---
description: Follow the engineering journey of ThinkStorm, from initial frictionless intake pipelines to full Gitea project incubation.
---
body:
Welcome to the ThinkStorm public development log. Here we document major engineering milestones, architectural decisions, service integrations, and runtime improvements as ThinkStorm evolves into a comprehensive autonomous idea incubation platform.
@@ -0,0 +1,43 @@
_model: devlog-entry
---
schema_version: 1
---
title: Project Incubation: Canonical Gitea Repositories under the ThinkStorm Organization
---
date: 2026-08-20
---
author: Labyricorn
---
summary: Transitioned idea dossiers to full-featured Gitea Git repositories under the dedicated thinkstorm organization, featuring automated file tree synchronization, executive READMEs, and Git clone workflows.
---
tags: gitea, projects, git, idea-incubation, organization
---
source_commit: 2c95567e28b6f5e1694c5f46eb3007f90f29b368
---
body:
# Project Incubation: Canonical Gitea Repositories under the ThinkStorm Organization
While code snippets and gists provide quick sharing, fully incubating ideas requires the power of a complete Git repository with branches, Markdown rendering, file hierarchies, issue tracking, and clone capabilities. This milestone transformed ThinkStorm idea dossiers into native **Gitea Project Repositories** housed under a centralized **`thinkstorm`** organization.
## Architectural Architecture
### 1. `thinkstorm` Organization Namespacing
- All idea repositories are automatically provisioned under the `thinkstorm` organization on Gitea (`https://git.labyricorn.com/thinkstorm/ts-{id}`).
- Generates standard clone targets for both HTTPS (`git clone https://git.labyricorn.com/thinkstorm/ts-{id}.git`) and SSH (`git clone ssh://[email protected]:22/thinkstorm/ts-{id}.git`).
### 2. Structured Dossier File Hierarchy
Each repository is populated with a standardized file tree:
- `README.md`: Rendered project landing page with executive summary, lifecycle state badge, prompt quotation, taxonomy tags, and project structure overview.
- `metadata.json`: Machine-readable metadata schema for automated tools and CI/CD pipelines.
- `research/`:
- `prior-art.md`: Competitor discovery and alternative analysis.
- `analysis.md`: Deep technical synthesis and architecture evaluations.
- `feasibility.md`: Feasibility scoring, risk critique, and implementation constraints.
- `outputs/`: Multi-modal work track deliverables organized by track name (e.g. `article/`, `coding-project/`, `business-canvas/`).
- `provenance/`: Execution run telemetry, model policies, and token attribution records.
### 3. Asynchronous File Tree Synchronization
- Built a multi-file synchronization engine in `GiteaAdapter` utilizing Gitea's Contents API to commit or update file trees without wiping repository history.
- Integrated automated pushes into the idea intake pipeline and work track workflow executors.
- Added a one-click **`🔄 Publish / Re-sync to Gitea`** button on Tab 6 of the Idea Detail view.
@@ -0,0 +1,39 @@
_model: devlog-entry
---
schema_version: 1
---
title: Architectural Foundation: Frictionless Intake, Multi-Stage Pipelines, and Untrusted URL Boundaries
---
date: 2026-08-19
---
author: Labyricorn
---
summary: Established the core ThinkStorm architecture, featuring frictionless anonymous idea capture, VirusTotal URL scanning, bounded multi-stage research pipelines, prompt catalog versioning, and token usage accounting.
---
tags: architecture, intake-engine, pipeline, security, fast-api
---
source_commit: f57d5df0880a5c9a96e6a5deb381daf458a3a11a
---
body:
# Architectural Foundation: Frictionless Intake & Multi-Stage Pipelines
The core design objective of ThinkStorm is solving the cognitive barrier of initial idea capture. Most project management and idea tracking platforms burden users with mandatory fields, rigid taxonomies, and categorization upfront. ThinkStorm flips this model: users throw free-form thoughts into an anonymous intake box with zero friction, while autonomous background processors handle structuring, research, and technical evaluation.
## What Was Implemented
### 1. Bounded Intake Engine & Safety Boundaries
- Implemented single-textarea intake accepting raw text up to 10,000 characters.
- Built an automatic URL extraction engine that isolates embedded links from untrusted submission text.
- Integrated asynchronous VirusTotal scanning that evaluates link reputation, domain risk profiles, and malicious redirection vectors. Submissions containing flagged URLs are automatically routed to a secure quarantine state for administrator review before any model processing runs.
### 2. Multi-Stage Pipeline Orchestration
- **Processor 1 (Safety Evaluator)**: Validates link safety and determines automated processing eligibility.
- **Processor 2 (Semantic Extraction)**: Extracts concise project titles, executive summaries, categories, and tags while preserving the raw submission text immutably.
- **Processor 3 (Duplicate & Relationship Detector)**: Computes semantic overlaps and links related ideas in SQLite.
- **Processor 4 (Prior Art & Landscape Search)**: Interfaces with SearXNG and Perplexica to discover live competitors and open-source equivalents.
- **Processor 5 (Technical Feasibility & Critique)**: Generates rigorous feasibility scoring, architectural constraints, and risk matrices.
### 3. Prompt Catalog & Provenance Telemetry
- Modeled prompt definitions with semantic versioning (`prompt_definitions` and `prompt_versions`).
- Tracked complete execution provenance (`processor_runs`) capturing model policies, resolved backend providers, input/output token counts, durations, and output artifacts.
@@ -0,0 +1,33 @@
_model: devlog-entry
---
schema_version: 1
---
title: Ecosystem Integration: Gitea OAuth2 Single Sign-On and OpenGist Dossier Publishing
---
date: 2026-08-19
---
author: Labyricorn
---
summary: Connected ThinkStorm with self-hosted ecosystem services by implementing Gitea OAuth2 authentication, session cookie management, and automated dossier synchronization to OpenGist.
---
tags: authentication, gitea, sso, opengist, integrations
---
source_commit: f57d5df0880a5c9a96e6a5deb381daf458a3a11a
---
body:
# Ecosystem Integration: Gitea OAuth2 SSO & OpenGist Synchronization
To operate seamlessly within the self-hosted Labyricorn infrastructure, ThinkStorm required direct integration with authentication providers and publishing backends. This milestone unified user authentication with Gitea accounts and enabled automated snippet publishing to OpenGist.
## Key Changes
### 1. Gitea OAuth2 Single Sign-On Flow
- Configured Gitea OAuth application credentials and registered redirect callback routes at `/auth/gitea/callback`.
- Fixed authorization code exchange by enforcing `Accept: application/json` headers on token requests to Gitea.
- Implemented automated user provisioning: authenticating via Gitea retrieves profile information, associates local user records (`gitea_id`), and issues secure HTTP-only session cookies (`thinkstorm_session`).
### 2. OpenGist REST API Formatter Alignment
- Deployed OpenGist v1.15 with custom branding, SSH Git support, and Gitea OAuth links.
- Resolved REST API schema mismatches by updating the OpenGist adapter to target `POST /api/gists` using the array/map payload structures and Personal Access Token bearer authentication.
- Added direct local subnet communication to avoid external IPv6 connection latency.
@@ -0,0 +1,38 @@
_model: devlog-entry
---
schema_version: 1
---
title: Work Track Iteration: Deliverable Versioning, Soft Deletion, and UI Tab Persistence
---
date: 2026-08-19
---
author: Labyricorn
---
summary: Implemented comprehensive multi-version artifact management for generated work track deliverables, soft-delete and permanent purge lifecycle states, and seamless tab focus persistence.
---
tags: work-tracks, versioning, ui-ux, lifecycle
---
source_commit: f57d5df0880a5c9a96e6a5deb381daf458a3a11a
---
body:
# Work Track Iteration: Deliverable Versioning & Lifecycle Controls
As ideas progress from research into active incubation, users activate specialized Work Tracks (such as Articles, Coding Scaffolds, and Blog Entries). Because iterative AI generation produces successive drafts of documents, we introduced a structured versioning engine and fine-grained lifecycle management.
## Key Enhancements
### 1. Document Versioning Engine
- Extended `work_track_outputs` with `version`, `is_current`, and `model_used` attributes.
- Whenever a workflow re-runs, previous deliverables are automatically archived with incremented version numbers, ensuring historical drafts remain viewable and restorable without overwriting.
- Added version badges (`v1`, `v2`, `v3`) in the UI displaying generation timestamps and attributing the specific LLM model used for each version.
### 2. Fine-Grained Deletion & Soft-Delete Lifecycle
- Added single-document deletion (`DELETE /api/ideas/work-tracks/outputs/{output_id}`) allowing users to delete specific versions or entire deliverable series.
- Introduced two-stage idea lifecycle controls:
- **Soft Trash (`TRASHED`)**: Moves discarded ideas to the Trash section while preserving full provenance and allowing one-click restoration.
- **Permanent Purge (`DELETE_PERMANENT`)**: Cascades deletion across categories, tags, processor runs, work tracks, external resources, and on-disk files with audit logging.
### 3. UI Tab State Persistence
- Resolved UI focus jump regressions where triggering asynchronous workflow executions inadvertently reset the active tab to the first panel.
- Synchronized tab navigation with `window.location.hash`, `sessionStorage`, and `localStorage`, providing persistent tab focus across page refreshes and asynchronous job updates.
+53
View File
@@ -0,0 +1,53 @@
_model: project
---
schema_version: 1
---
title: ThinkStorm
---
tagline: Frictionless AI Idea Incubation & Autonomous Project Engine
---
status: active
---
author: Labyricorn
---
tags: ai-orchestration, self-hosted, idea-incubation, gitea, privacy-first, fast-api
---
technologies: Python, FastAPI, SQLite (WAL), Jinja2, OmniRoute, SearXNG, Perplexica, Gitea, OpenGist, VirusTotal
---
repository: https://git.labyricorn.com/thinkstorm
---
summary: ThinkStorm captures unformed ideas with zero friction, preserves original submissions, and orchestrates self-hosted AI to research prior art, synthesize feasibility, and incubate full Gitea projects.
---
body:
# ThinkStorm: Autonomous Idea Incubation Platform
**ThinkStorm** is a self-hosted platform built to solve the cold-start problem of idea generation and execution. It eliminates cognitive friction during initial capture, protects user intent by preserving immutable submission logs, and orchestrates local and cloud AI models to research, evaluate, and scaffold ideas into full projects.
## Core Architectural Pillars
### 1. Frictionless Anonymous Intake
- **Zero Required Fields**: Users submit free-form thoughts without mandatory titles, categories, or tags.
- **Embedded URL Safety Boundary**: URLs inside submissions are automatically extracted and scanned via VirusTotal. High-risk links trigger automated quarantine and administrative review before processor execution.
### 2. Multi-Stage Research & Prior Art Pipeline
- **Title & Summary Extraction**: AI extracts semantic essence and tags while keeping the original submission immutable.
- **Prior Art & Alternative Discovery**: Integrates with SearXNG and Perplexica to search the live web and identify existing competitors, similar projects, and differentiators.
- **Deep Research & Technical Feasibility**: Generates comprehensive feasibility scores, risk critiques, and architecture blueprints.
### 3. Canonical Gitea Project Repositories
- Every incubated idea is provisioned as a full Git repository under the **`thinkstorm`** organization on Gitea (e.g. `https://git.labyricorn.com/thinkstorm/ts-0032`).
- Structured repository dossiers include executive `README.md`, `metadata.json`, `research/prior-art.md`, `research/analysis.md`, `research/feasibility.md`, and multi-modal `outputs/`.
- Full Git history, branch management, issue tracking, and SSH/HTTPS cloning capabilities are natively supported.
### 4. Multi-Modal Work Tracks & Versioning
- Ideas can be claimed and activated into specialized work tracks:
- **Article & Whitepaper**: Long-form structured essays and publication-ready Markdown.
- **Coding Project**: Full codebase architectures, implementation requirements, and Gitea repository graduation.
- **Blog Entry**: Conversational announcement posts and technical retrospectives.
- Complete versioning and deletion system preserves iterative revision histories while allowing fine-grained lifecycle management.
### 5. Self-Hosted Infrastructure & Authentication
- **Gitea OAuth2 SSO**: Unified single sign-on with Gitea accounts.
- **Local SQLite Storage**: High-concurrency Write-Ahead Logging (WAL) with automatic sequence generation and schema migration.
- **OmniRoute Integration**: Flexible routing between local LLM backends (Ollama, vLLM) and upstream cloud models with token usage telemetry and prompt catalog versioning.
+200
View File
@@ -0,0 +1,200 @@
# ⚡ ThinkStorm
> **Frictionless Anonymous Idea Intake, Autonomous AI Research, and Project Incubation Engine.**
ThinkStorm captures unformed ideas with zero friction, preserves original submissions immutably, and orchestrates self-hosted AI models to research the landscape, synthesize feasibility, and incubate full Gitea projects.
---
## 🌟 Key Features
### 1. Frictionless Anonymous Intake
- **Zero Required Metadata**: Submit raw, free-form thoughts without mandatory titles, categories, or tags.
- **Embedded URL Isolation & Safety Scanner**: URLs inside submissions are extracted and evaluated via VirusTotal. Unsafe or suspicious links are automatically quarantined for administrator review before any model processing runs.
- **Immutable Prompt Preservation**: The original submission prompt is permanently locked in the database and displayed verbatim across all views.
### 2. Multi-Stage Research & Evaluation Pipeline
- **Title & Summary Extraction**: AI extracts semantic titles, executive summaries, and initial taxonomies.
- **Duplicate & Relationship Mapping**: Automatically identifies conceptual overlaps and clusters related ideas.
- **Prior Art & Competitor Search**: Queries live web indices via SearXNG and Perplexica to discover similar projects, existing tools, and architectural differentiators.
- **Deep Technical Feasibility & Risk Critique**: Generates feasibility scores (010), technical constraints, implementation roadmaps, and risk matrices.
### 3. Canonical Gitea Project Repositories
- Every incubated idea is automatically provisioned as a full Git repository under the **`thinkstorm`** organization on Gitea (e.g. `https://git.labyricorn.com/thinkstorm/ts-0032`).
- Repositories include:
- `README.md`: Executive summary, prompt quotation, lifecycle badge, and project structure guide.
- `metadata.json`: Machine-readable metadata schema.
- `research/`: `prior-art.md`, `analysis.md`, `feasibility.md`.
- `outputs/`: Multi-modal work track deliverables organized by track.
- `provenance/`: Execution run logs and token attribution telemetry.
- Full Git history, branch management, issue tracking, and SSH/HTTPS cloning (`git clone ...`) are natively supported.
### 4. Multi-Modal Work Tracks & Versioning
- Ideas can be claimed by authenticated users to unlock specialized Work Tracks:
- 📄 **Article & Whitepaper**: Structured long-form analysis and publication drafts.
- 💻 **Coding Project**: Software architectures, API specs, and codebase scaffolds.
- ✍️ **Blog Entry**: Conversational announcement posts and technical retrospectives.
- **Iterative Deliverable Versioning**: Subsequent workflow runs archive previous outputs as versions (`v1`, `v2`, `v3`) with model attribution, allowing fine-grained deletion and draft comparison.
### 5. Authentication & Access Control
- **Gitea OAuth2 Single Sign-On**: Log in with your Gitea account with automatic profile provisioning.
- **Local Fallback Authentication**: Secure bcrypt password authentication for administrative and local accounts.
- **Role-Based Access Control (RBAC)**: Supports `ANONYMOUS`, `USER`, and `ADMIN` permission tiers.
### 6. Administration, Queue & Provenance
- **Background Job Queue**: Asynchronous, non-blocking pipeline execution with foreground prioritization and retry controls.
- **Prompt Catalog & Semantic Versioning**: Edit prompt templates dynamically with version tracking and profile activation.
- **Token Accounting & Model Telemetry**: Comprehensive auditing of input/output token usage, provider durations, and execution status.
---
## 🏗️ Architecture & Technology Stack
```mermaid
graph TD
User([User / Browser]) -->|Submit / Browse| Web[FastAPI Web Server]
Web -->|Store / Query| DB[(SQLite WAL Database)]
Web -->|Enqueue Jobs| Queue[Background Queue Worker]
Queue --> P1[1. Safety & URL Evaluator]
Queue --> P2[2. Semantic Title & Tag Extractor]
Queue --> P3[3. Duplicate & Cluster Detector]
Queue --> P4[4. Prior Art Search]
Queue --> P5[5. Feasibility & Risk Synthesis]
P1 -->|Scan URLs| VT[VirusTotal API]
P4 -->|Web Search| SearXNG[SearXNG & Perplexica]
P2 & P3 & P4 & P5 -->|LLM Synthesis| Omni[OmniRoute LLM Router]
Queue -->|Push Dossier Repo| Gitea[Gitea Project Host]
Queue -->|Push Gist Snippets| OpenGist[OpenGist Service]
```
- **Backend**: Python 3.13, FastAPI, Uvicorn, AsyncIO, Pydantic
- **Database**: SQLite 3 with Write-Ahead Logging (`WAL`), Foreign Keys, and automated migrations
- **Frontend**: Jinja2 Templates, Vanilla Modern CSS (Dark Mode, Glassmorphism), Modular JavaScript
- **AI Orchestration**: OmniRoute (Ollama, OpenAI, Anthropic, vLLM endpoints)
- **Web Search**: SearXNG (Meta-search API) & Perplexica (Search Backend)
- **Code & Version Control**: Gitea (OAuth2 SSO & Git Repositories), OpenGist (Gist Dossiers)
- **Security & Safety**: VirusTotal API (URL domain risk analysis)
---
## 📁 Repository Structure
```
.
├── thinkstorm/ # Core application package
│ ├── main.py # FastAPI application entrypoint & middleware
│ ├── config.py # Service configuration & dynamic .env loader
│ ├── database.py # SQLite database schema, migrations & seed defaults
│ ├── models.py # Dataclasses, Enums, and Pydantic models
│ ├── auth.py # Password hashing, JWT sessions & RBAC
│ ├── api/ # REST API routers
│ │ ├── ideas.py # Idea submission, claiming, work tracks & sync
│ │ ├── auth_routes.py # Local login & Gitea OAuth2 SSO callback
│ │ └── admin.py # Service testing, queue retry, quarantine moderation
│ ├── processors/ # Pipeline execution engine
│ │ └── pipeline.py # Bounded processors 1-5 & work track workflows
│ ├── prompts/ # Prompt definitions & profile catalog
│ │ └── catalog.py # System prompts, profile alignment & versioning
│ ├── queue/ # Background job queue
│ │ └── worker.py # Async worker queue with foreground priority
│ ├── services/ # External service adapters
│ │ ├── gitea.py # Gitea repository creation & file sync
│ │ ├── opengist.py # OpenGist REST sync adapter
│ │ ├── omniroute.py # LLM completion & streaming adapter
│ │ ├── searxng.py # Live meta-search adapter
│ │ ├── perplexica.py # Perplexica search backend adapter
│ │ └── virustotal.py # URL reputation scanner adapter
│ ├── templates/ # Jinja2 HTML templates
│ │ ├── base.html # Core layout, navigation & status bar
│ │ ├── index.html # Frictionless hero intake page
│ │ ├── ideas.html # Filterable ideas list with lifecycle states
│ │ ├── idea_detail.html # Comprehensive idea dossier & work tracks
│ │ ├── login.html # Gitea SSO & local login portal
│ │ ├── prompt_catalog.html # Prompt template editor & version manager
│ │ └── admin.html # Service health & queue management
│ └── static/ # Static assets
│ ├── css/style.css # Glassmorphic dark design system
│ └── js/app.js # Interactive tab navigation, toasts & sync
├── tests/ # Test suite
│ ├── test_api.py # API endpoint & permission tests
│ └── test_artifact_versioning.py # Versioning & lifecycle tests
├── .labyricorn/ # Labyricorn exhibition & devlog space
│ ├── project/contents.lr # Project exhibition record
│ └── devlog/ # Public development logs
├── data/ # Persistent local data & SQLite store
└── README.md # This documentation file
```
---
## ⚙️ Configuration (`.env`)
ThinkStorm loads environment settings from `/root/.env`, `.env`, and `thinkstorm/.env`:
| Variable | Description | Default / Example |
| :--- | :--- | :--- |
| `OMNIROUTE_URL` | OmniRoute LLM API base endpoint | `https://llm.example.com` |
| `OMNIROUTE_API_KEY` | Bearer token for LLM routing | `<your-omniroute-api-key>` |
| `SEARXNG_URL` | SearXNG meta-search instance | `https://search.example.com` |
| `PERPLEXICA_URL` | Perplexica backend instance | `https://perplexica.example.com` |
| `GITEA_URL` | Gitea instance public URL | `https://git.example.com` |
| `GITEA_API_TOKEN` | Gitea admin/user access token | `<your-gitea-api-token>` |
| `GITEA_CLIENT_ID` | Gitea OAuth2 Application Client ID | `<your-gitea-client-id>` |
| `GITEA_CLIENT_SECRET` | Gitea OAuth2 Application Secret | `<your-gitea-client-secret>` |
| `OPENGIST_URL` | OpenGist public URL | `https://gist.example.com` |
| `OPENGIST_API_TOKEN` | OpenGist Personal Access Token | `<your-opengist-api-token>` |
| `VIRUSTOTAL_API_KEY` | VirusTotal API Key for URL safety | `<your-virustotal-api-key>` |
| `ADMIN_BOOTSTRAP_KEY` | Default admin account password | `<your-admin-bootstrap-password>` |
---
## 🚀 Getting Started
### 1. Installation
```bash
# Clone the repository
git clone https://git.labyricorn.com/thinkstorm/thinkstorm.git
cd thinkstorm
# Install Python dependencies
pip install fastapi uvicorn pydantic jinja2 requests pytest pytest-asyncio httpx
```
### 2. Running the Server
```bash
# Start ThinkStorm on port 8000
python3 -m uvicorn thinkstorm.main:app --host 0.0.0.0 --port 8000 --proxy-headers --forwarded-allow-ips='*'
```
Access the application in your browser at **`http://localhost:8000`** (or your domain e.g. **`https://ts.labyricorn.com`**).
### 3. Running Automated Tests
```bash
PYTHONPATH=. pytest tests/ -v
```
---
## 📖 API Endpoints Summary
- `POST /api/ideas`: Anonymous free-form idea submission.
- `GET /api/ideas`: Filterable list of ideas by lifecycle state, category, or tag.
- `GET /api/ideas/{idea_id}`: Full idea details, provenance, research, and work tracks.
- `POST /api/ideas/{idea_id}/claim`: Claim an available idea for incubation.
- `POST /api/ideas/{idea_id}/activate`: Activate an idea into active development.
- `POST /api/ideas/{idea_id}/sync-gitea`: Push or re-sync dossier files to Gitea repository.
- `POST /api/ideas/work-tracks/{track_id}/activate`: Run or re-run a work track workflow.
- `DELETE /api/ideas/work-tracks/outputs/{output_id}`: Delete specific deliverable version.
- `POST /api/auth/login`: Local username/password authentication.
- `GET /api/auth/gitea/url`: Get Gitea OAuth2 authorization URL.
- `GET /auth/gitea/callback`: OAuth2 redirect callback & session issuer.
- `POST /api/admin/services/{service_id}/test`: Real-time health check for external services.
---
## 📄 License & Attribution
Developed by **Labyricorn**. Licensed under the [MIT License](LICENSE).