Production RAG Chatbot on AWS: Development Plan (Concise)
**Objective:** Build and deploy a scalable, production-ready RAG chatbot on AWS using proprietary data, supporting multiple LLMs, with VTION branding, developed component-by-component.
**Core Principles:** Modular, User Control, Iterative Development (especially for prompts), Comprehensive (yet concise) Documentation, Detailed Logging, Best Practices, Security, VTION Branding (`#00afaf`, `#0c0e2d`, light theme).
---
Phase 0: Project Setup & Foundational Configuration
**Goal:** Establish environment, AWS services, GitHub, LLM access, and project structure.
* [ ] **Task 0.1: Environment & Tooling** * [ ] Confirm local dev setup (IDE, Git). * [ ] Standardize: **Python 3.11**. * [ ] List/confirm library versions (FastAPI/Flask, Boto3, Frontend framework). * [ ] **AI Action:** Verify with user.
* [ ] **Task 0.2: AWS Setup/Verification** * [ ] Confirm AWS account & IAM permissions for: S3, Bedrock, Lambda, API Gateway, Athena/Redshift Spectrum, IAM, CloudWatch, (Optional: ECS, EKS, Amplify). * [ ] **AI Action:** Prompt user for AWS config/entitlement confirmation. Document services.
* [ ] **Task 0.3: GitHub & CI/CD Foundation** * [ ] Create private GitHub repo. * [ ] Define branching strategy (e.g., `main`, `develop`, `feature/*`). * [ ] Basic GitHub Actions: linting, placeholder for build/test/deploy. * [ ] **AI Action:** Guide repo creation & initial workflow.
* [ ] **Task 0.4: LLM Access & Key Management** * [ ] Confirm Bedrock access: **Claude 3 Sonnet** (default). * [ ] Confirm access/keys for optional LLMs (OpenAI, Gemini). * [ ] Secure API key management (AWS Secrets Manager or env vars). **No hardcoded keys.** * [ ] **AI Action:** Confirm LLM access; discuss key management.
* [ ] **Task 0.5: Define Core Technologies (Revisit)** * [ ] Backend: FastAPI (recommended) or Flask. * [ ] Frontend: React with Vite (recommended) or HTML/JS/Tailwind. * [ ] AWS SDK: Boto3 (latest). * [ ] **AI Action:** Discuss choices; document.
* [ ] **Task 0.6: Project Directory Structure** * [ ] Define structure (backend, frontend, data, scripts, docs, .github). ``` /project-root ├── /backend (app, Dockerfile, requirements.txt) ├── /frontend (public, src, package.json, Dockerfile) ├── /data (sample data, S3 upload scripts, generated_context/) ├── /scripts (helpers, deployment) ├── /docs ├── .github/workflows ├── .gitignore └── README.md ``` * [ ] **AI Action:** Propose; get approval.
---
Phase 1: Data Understanding & Context Generation
**Goal:** Analyze data, build modular data loader, implement "first-time context & dictionary builder."
* [ ] **Task 1.1: Analyze Sample Data (S3)** * [ ] User provides sample data/schema. * [ ] Identify types, patterns, PII. Storage format in S3 (CSV, JSON, Parquet). * [ ] **AI Action:** Request sample/schema. Discuss S3 format.
* [ ] **Task 1.2: Design Modular S3 Data Access (Athena/Spectrum)** * [ ] Python module for S3 data interaction (via Athena/Spectrum). * [ ] Configurable for S3 paths & data formats. * [ ] **AI Action:** Propose design.
* [ ] **Task 1.3: Implement Initial Data Profiling** * [ ] Script: list columns, infer types, basic stats (via Athena/Spectrum). * [ ] **AI Action:** Code utility.
* [ ] **Task 1.4: Develop "First-Time Context & Dictionary Builder"** * [ ] **Concept:** One-time LLM process (Claude 3 Sonnet) on data subset (schema, sample rows) for: * ~500-word **domain context**: Overview of data, purpose, common query types, business rules. * **Data dictionary**: Column names, detailed descriptions, data types (Athena/Presto specific), table relationships, value distributions/enums where applicable, sample valid values. * [ ] **Implementation:** Script to fetch schema/sample, prompt LLM, store context/dictionary (e.g., JSON/YAML in S3 `/generated_context/` or DynamoDB). Ensure efficient loading/caching by backend. * [ ] **AI Action:** Design prompt, write script, discuss storage & efficient access.
---
Phase 2: Backend Skeleton - API & Core Logic
**Goal:** Setup backend app, API endpoints, LLM module, text-to-SQL, data querying.
* [ ] **Task 2.1: Design Core API Endpoints** * [ ] `/query` (POST), `/capabilities` (GET), `/config/llm` (GET/POST, Admin), `/admin/context/regenerate` (POST, Admin). * [ ] **AI Action:** Define OpenAPI/Swagger spec.
* [ ] **Task 2.2: Setup Basic Backend App (FastAPI/Flask)** * [ ] Init framework, "hello world" endpoint, `requirements.txt`, `Dockerfile`. * [ ] **AI Action:** Provide boilerplate.
* [ ] **Task 2.3: Implement LLM Interaction Module (`llm_clients.py`)** * [ ] Unified interface for LLMs. * [ ] **Sub-tasks:** Bedrock (Claude 3 Sonnet), optional OpenAI, optional Gemini. Secure key handling. * [ ] Easy model switching via config. * [ ] **AI Action:** Code module, start with Bedrock.
* [ ] **Task 2.4: Develop Text-to-SQL Component** * [ ] Takes user query, loads comprehensive context/dictionary (Task 1.4). * [ ] **Advanced Prompt Engineering:** * Use detailed context (schema, descriptions, relationships, sample values/queries, business rules). * Provide few-shot examples of good text-to-SQL pairs if necessary. * Clear instructions on SQL dialect (Athena/Presto), functions to use/avoid. * Security constraints (e.g., generate read-only queries, avoid specific clauses if risky). * [ ] Prompts LLM (Claude 3 Sonnet) for SQL. * [ ] **Robust SQL validation/sanitization:** e.g., allowlisting commands/patterns, parsing query structure, checking for DML or harmful operations, regex checks. * [ ] **AI Action:** Design prompt, implement component with validation.
* [ ] **Task 2.5: Implement AWS Data Querying (Athena/Redshift Spectrum)** * [ ] Executes SQL, retrieves results. * [ ] **Crucial:** Limit to 50 rows for LLM. Aggregate if >50 rows. No raw DB info to next LLM. * [ ] Ensure query execution IAM roles have least privilege access to data. * [ ] **AI Action:** Code module.
* [ ] **Task 2.6: Implement Result Processing & Formatting (LLM-based)** * [ ] Takes query data (max 50 rows), original query. * [ ] Prompts LLM (Claude 3 Sonnet) for natural language answer & graphical representation suggestion (e.g., `{type: 'bar', data: {...}}`). * [ ] **Strictly no raw table data in final response.** * [ ] **AI Action:** Design prompt, implement component.
* [ ] **Task 2.7: Implement Robust Logging Framework** * [ ] Python `logging` module, structured JSON logs (for CloudWatch). * [ ] Logs: timestamp, level, component, correlation ID, clear messages (mind PII). * [ ] **Strictly maintain detailed logging.** * [ ] **AI Action:** Setup config, integrate into components.
---
Phase 3: Frontend Skeleton - User Interface
**Goal:** Develop basic UI with VTION branding.
* [ ] **Task 3.1: Choose Frontend Framework (Confirm 0.5)** * [ ] Confirm (React, Vue, or HTML/JS/Tailwind). Setup project. * [ ] **AI Action:** Provide boilerplate.
* [ ] **Task 3.2: Design Basic UI Layout** * [ ] Chat interface: input, conversation history, graphics area. * [ ] VTION branding: colors (`#00afaf`, `#0c0e2d`), light theme (e.g., `#f8f9fa`), logo. Responsive. * [ ] **AI Action:** Create HTML/CSS or React components.
* [ ] **Task 3.3: Implement "Capabilities Screen/Initial Message"** * [ ] On load/help: explain data, example questions, limitations. Static or dynamic. * [ ] **AI Action:** Implement.
* [ ] **Task 3.4: Implement Chat Input & Display Components** * [ ] User query input. Display user/bot messages (text & graphics placeholder). * [ ] **AI Action:** Develop UI components.
* [ ] **Task 3.5: Integrate Frontend with Backend API (`/query`)** * [ ] JS/TS: send query, receive response, display text, pass graphics data to stub. * [ ] Handle API states (loading, success, error). * [ ] **AI Action:** Implement API integration.
* [ ] **Task 3.6: Apply VTION Branding & Styling** * [ ] Ensure UI adheres to VTION colors/theme. Clean font. * [ ] **AI Action:** Refine CSS/styling.
---
Phase 4: Connecting Pipeline & Initial End-to-End Testing
**Goal:** Integrate backend components, connect frontend for basic E2E pipeline.
* [ ] **Task 4.1: Backend: Integrate Context with Text-to-SQL** * [ ] Task 2.4 uses context/dictionary from Task 1.4. Test. * [ ] **AI Action:** Verify, debug.
* [ ] **Task 4.2: Backend: Integrate Text-to-SQL with Data Querying** * [ ] Task 2.4 SQL output to Task 2.5. Verify aggregation/50-row limit. * [ ] **AI Action:** Verify, debug.
* [ ] **Task 4.3: Backend: Integrate Query Results with LLM for Final Response** * [ ] Task 2.5 data to Task 2.6. Verify answer/graphic suggestion. * [ ] **AI Action:** Verify, debug.
* [ ] **Task 4.4: Full End-to-End Test (Manual)** * [ ] UI query -> Backend (Text-to-SQL -> Data Query -> Result Processing) -> UI response. * [ ] **Log extensively.** * [ ] **AI Action:** Guide testing, analyze logs.
* [ ] **Task 4.5: Implement Graphical Representation Stubs (Frontend)** * [ ] Placeholders for charts (log or display raw data). * [ ] **AI Action:** Implement stubs.
---
Phase 5: Enhancements & Production Readiness
**Goal:** Add graphics, error handling, admin console, prepare for production (security, scalability).
* [ ] **Task 5.1: Develop Graphical Representation Component (Frontend)** * [ ] Charting library (Chart.js, D3.js, Recharts). Render chart from backend data. Responsive, VTION colors. * [ ] **AI Action:** Implement.
* [ ] **Task 5.2: Implement Robust Error Handling & User Feedback** * [ ] **Backend:** Global error handlers, meaningful errors/HTTP codes (no sensitive details to user). * [ ] **Frontend:** Graceful API error handling, user-friendly messages. * [ ] **AI Action:** Review, enhance.
* [ ] **Task 5.3: Build Basic Admin Console (Optional)** * [ ] Secured interface: view logs, trigger context regen, config LLM, view usage stats. * [ ] Secure appropriately. * [ ] **AI Action:** Design, implement.
* [ ] **Task 5.4: Security Hardening** * [ ] Input validation, SQLi prevention (validate LLM SQL per Task 2.4), API auth, least privilege IAM for all AWS services (including Bedrock model invocation). * [ ] Secure data handling, dependency scanning. Review Bedrock security best practices. * [ ] **AI Action:** Security review.
* [ ] **Task 5.5: Scalability & Performance** * [ ] **Backend:** Async (FastAPI), Lambda/API Gateway or ECS for scaling. Efficient loading/caching of context (Task 1.4). * [ ] **Data Querying:** Optimize Athena/Spectrum. Partition S3 data. * [ ] **LLM Calls:** Retries, timeouts. * [ ] **Frontend:** CDN for static assets. * [ ] **AI Action:** Review, suggest scaling strategies.
* [ ] **Task 5.6: Comprehensive CI/CD Pipeline (GitHub Actions)** * [ ] **Backend/Frontend:** Lint, test, build, deploy to staging/prod (Docker, ECR, Lambda/ECS, S3/Amplify). * [ ] **AI Action:** Implement CI/CD.
---
Phase 6: Documentation & Logging Refinement
**Goal:** Finalize comprehensive documentation and perfect logging.
* [ ] **Task 6.1: Create User Manual** * [ ] Access, usage, examples, graphics, troubleshooting. * [ ] **AI Action:** Draft.
* [ ] **Task 6.2: Create Developer & Operations Documentation** * [ ] Architecture, setup, component details (including prompt strategies), data schema/context, deployment, troubleshooting, LLM config. * [ ] **Detailed for non-programmers.** * [ ] **AI Action:** Compile/refine throughout project.
* [ ] **Task 6.3: Finalize and Test Logging** * [ ] Review logs (CloudWatch). Ensure clarity, consistency, correlation IDs. No sensitive PII. * [ ] **AI Action:** Review logging.
---
Recurring Tasks (For Each Component/Major Step)
* [ ] **R.1: Code:** Write. **Self-Reflect:** Correct, consistent, aligned? Prompts effective? * [ ] **R.2: Unit Test:** Write, pass. * [ ] **R.3: Push to GitHub:** Feature branch, PR. * [ ] **R.4: CI Checks:** Pass. * [ ] **R.5: Peer Review:** (Optional). * [ ] **R.6: Deploy to Staging (AWS):** CI/CD or manual. * [ ] **R.7: Test on Staging:** Integration, UAT. Verify, check logs, test various prompts. * [ ] **R.8: User Approval:** On staging. * [ ] **R.9: Merge:** To `develop`/`main`. * [ ] **R.10: Deploy to Production (AWS).** * [ ] **R.11: Update Documentation.**
--- This task list is a living document.