From ae15f302537d4e2fcbb34aa7eca1fac81bd49f22 Mon Sep 17 00:00:00 2001 From: chungyan5 Date: Mon, 6 Oct 2025 02:03:04 +0000 Subject: [PATCH] create rvs-prxy architecture doc. --- Design_Doc/DevProc.md | 32 +++ Design_Doc/architecture.md | 276 +++++++++++++++++++++++++ Design_Doc/auto-sys-deploy.sh.md | 61 ++++++ Design_Doc/instructions.md | 0 Design_Doc/update-compose-config.sh.md | 83 ++++++++ 5 files changed, 452 insertions(+) create mode 100644 Design_Doc/DevProc.md create mode 100644 Design_Doc/architecture.md create mode 100644 Design_Doc/auto-sys-deploy.sh.md create mode 100644 Design_Doc/instructions.md create mode 100644 Design_Doc/update-compose-config.sh.md diff --git a/Design_Doc/DevProc.md b/Design_Doc/DevProc.md new file mode 100644 index 0000000..c9682ca --- /dev/null +++ b/Design_Doc/DevProc.md @@ -0,0 +1,32 @@ +# Purpose of this file +- This shows the Development Processes work. +- Current tool(s) with LLM Qwen-coder-plus, deepseek, etc. + +# Development Processes: The following is sorted with the latest items at the top. + +## make a generic doc. for this arch. +- Add more detail into `Design_Doc/architecture.md`: 1) Add block diagram(s) as visualization of the description of `Design_Doc/architecture.md`, 2) attached `auto-sys-deploy-ref.sh` and `update-compose-config-ref.sh` with corresponding `auto-sys-deploy.sh.md` and `update-compose-config.sh.md` as an example, using real example parameters, outputs, etc. during the explanation. + +- Base your understand of `auto-sys-deploy-ref.sh` and `update-compose-config-ref.sh` with your mardown documents at `Design_Doc/`. I aim to have a summary document at `Design_Doc/architecture.md`, it talks about the generic concept of this architecture which can apply to any applications with clearly listing the parameters and corresponding outputs. For example, currently `auto-sys-deploy-ref.sh` and `update-compose-config-ref.sh`: + - Prerequisites + - Using FRP (fast reverse proxy) service with frpc and frps + - using Traefik + - Docker and Docker Compose installed + - SSH key at `~/.ssh/itl-devs1` for proxy server access + - `DOWNLOAD_SOURCE` environment variable set + - User permissions for Docker commands + - SSH keys for authentication (`~/.ssh/authorized_keys` and `~/.ssh/devs1`) + - parameters + - **Instance Name**: Alphanumeric, hyphens, and underscores only + - **Docker Network**: Existing Docker network or custom network name + - **FRP Server Address**: IP address of the FRP server + - **FRP Server Port**: Default is 16999 + - **Proxy Ports**: Configurable remote ports for rad (default 8776), ssh (default 22), and web (default 80) proxies + - application port(s) + - Domain name for Traefik routing rules + - outcomes + - public domain connected to a internal service. + - etc. + +## Understand how previous to do +- Analysis and explain these two scripts to me: `auto-sys-deploy-ref.sh` and `update-compose-config-ref.sh`; Write your explaination into `Design_Doc/auto-sys-deploy.sh.md` and `Design_Doc/update-compose-config.sh.md` corresponding. \ No newline at end of file diff --git a/Design_Doc/architecture.md b/Design_Doc/architecture.md new file mode 100644 index 0000000..da46f51 --- /dev/null +++ b/Design_Doc/architecture.md @@ -0,0 +1,276 @@ +# Generic Reverse Proxy Architecture + +## Overview +This document describes a generic reverse proxy architecture that enables internal services to be accessible via public domains. The architecture uses FRP (Fast Reverse Proxy) for tunneling and Traefik for HTTP/HTTPS routing with automatic certificate management. + +## Architecture Block Diagram + +``` ++-------------------+ SSH Commands +-------------------+ +| Local Host | ------------------> | Proxy Server | +| | | | +| +-------------+ | FRP Tunnel | +-------------+ | +| | Application | | ------------------> | | FRPS | | +| | Service | | | | (Server) | |<--- Public IPs +| | | | | | | | Port: 16999 +| +-------------+ | | +-------------+ | +| | | | +| +-------------+ | FRP Client | +-------------+ | +| | FRPC | | ------------------> | | Traefik | |<--- Domains +| | (Client) | | | | (Router) | | HTTPS/TLS +| | | | | | | | +| +-------------+ | | +-------------+ | +| | | | +| +-------------+ | | +-------------+ | +| | Docker | | | | Docker | | +| | Container | | | | Container | | +| +-------------+ | | +-------------+ | ++-------------------+ +-------------------+ + | | + | | + +------------------ Volumes ------------------+ + | rad-git-sshd-itl-frpc- | + +-------------------------------------------+ +``` + +## Architecture Components + +### Core Components +- **FRP (Fast Reverse Proxy)**: Handles TCP/UDP tunneling between internal and external networks + - FRPS (FRP Server): Runs on the public-facing server + - FRPC (FRP Client): Runs on internal service hosts +- **Traefik**: HTTP/HTTPS router with automatic Let's Encrypt certificate management +- **Docker & Docker Compose**: Container orchestration and deployment +- **SSH**: Secure communication for remote configuration management + +### Supporting Elements +- **Configuration Storage**: Docker volumes for persistent configuration +- **DNS Management**: Domain-to-IP mapping for public access +- **Certificate Management**: Automatic SSL/TLS certificate provisioning and renewal + +## Prerequisites +- FRP (Fast Reverse Proxy) service with frpc and frps +- Traefik reverse proxy +- Docker and Docker Compose installed +- SSH key for remote server access +- Environment variable for configuration source (DOWNLOAD_SOURCE) +- User permissions for Docker commands +- Authentication keys for secure access + +## Parameters + +### Required Parameters +- **Instance Name**: Unique identifier for the service instance (alphanumeric, hyphens, underscores only) +- **Application Port(s)**: Internal port(s) where the service is listening +- **Domain Name**: Public domain for Traefik routing rules +- **FRP Server Address**: Public IP address of the FRP server +- **FRP Server Port**: Port for FRP communication (default 16999) + +### Optional Parameters +- **Docker Network**: Network for container isolation (default uses Docker bridge) +- **Proxy Ports**: Custom remote ports for external access (defaults: TCP ports matching internal ports) +- **Service Type**: Protocol specification (TCP/UDP/HTTP/HTTPS) + +## Outcomes +- Public domain connectivity to internal service +- Secure encrypted tunnel between internal and external networks +- Automatic SSL/TLS certificates via Let's Encrypt +- Persistent configuration through Docker volumes +- Automated service discovery and routing + +## Architecture Flow + +### 1. Configuration Phase +1. User provides service parameters via interactive prompts or command-line +2. System validates parameters and checks for existing configurations +3. Generates appropriate FRP client configuration (frpc.toml) +4. Persists configuration to Docker volumes for continuity + +### 2. Deployment Phase +1. Creates Docker container with FRP client +2. Establishes connection to FRP server +3. Sets up port forwarding rules +4. Updates proxy server to recognize new instance + +### 3. Routing Phase +1. Traefik configuration updated with new routing rules +2. Domain name routes to appropriate internal service +3. Automatic SSL/TLS certificate provisioned +4. Health checks and monitoring established + +### 4. Management Phase +1. Persistent configuration for service continuity +2. Backup and recovery mechanisms +3. Restart procedures for configuration changes +4. Service status monitoring + +## Real-World Example: Using auto-sys-deploy.sh and update-compose-config.sh + +### Example: Deploying a New FRP Client Instance + +Based on `auto-sys-deploy.sh.md` and `update-compose-config.sh.md`, here's a real-world example of the architecture in action: + +#### Initial Setup +- **Prerequisites**: Docker, SSH key at `~/.ssh/itl-devs1`, `DOWNLOAD_SOURCE` environment variable +- **Command**: `./auto-sys-deploy.sh setup-frpc` + +#### Interactive Configuration (from auto-sys-deploy.sh) +1. **Instance Name Selection**: + - Default: `setup-frpc-$(date +%Y%m%d-%H%M%S)` + - User input: `my-app-instance-20231005-143000` + - Validation: alphanumeric, hyphens, underscores only + +2. **Docker Network Selection**: + - Options: Select from existing networks, custom network, or default + - User input: Select network `bridge` or create custom + - Outcome: Container runs on specific Docker network + +3. **FRP Server Configuration**: + - Server Address: `192.168.1.100` (example IP) + - Server Port: `16999` (default) + - Validation: Ensure server is accessible + +4. **Proxy Configuration**: + - **Rad Proxy** (from auto-sys-deploy.sh): + - Local Port: `8776` (default) + - Remote Port: `8779` (user input) + - Name: `rad-my-app-instance-20231005-143000` + - **SSH Proxy** (from auto-sys-deploy.sh): + - Local Port: `22` (default) + - Remote Port: `13076` (user input) + - Name: `ssh-my-app-instance-20231005-143000` + - **Web Proxy** (from auto-sys-deploy.sh): + - Local Port: `80` (default) + - Remote Port: `13077` (user input) + - Name: `web-my-app-instance-20231005-143000` + +5. **Domain Configuration**: + - Input: `myapp.example.com` + - Used for Traefik routing rules + +#### Configuration Generation (from auto-sys-deploy.sh) +The script generates `frpc.toml` with: + +```toml +serverAddr = "192.168.1.100" +serverPort = 16999 + +[[proxies]] +name = "rad-my-app-instance-20231005-143000" +type = "tcp" +localPort = 8776 +localIP = "rad-git-sshd-my-app-instance-20231005-143000" +remotePort = 8779 + +[[proxies]] +name = "ssh-my-app-instance-20231005-143000" +type = "tcp" +localPort = 22 +localIP = "rad-git-sshd-my-app-instance-20231005-143000" +remotePort = 13076 + +[[proxies]] +name = "web-my-app-instance-20231005-143000" +type = "tcp" +localPort = 80 +localIP = "rad-git-sshd-my-app-instance-20231005-143000" +remotePort = 13077 +``` + +#### Container Deployment (from auto-sys-deploy.sh) +- Creates Docker volume: `rad-git-sshd-itl-frpc-my-app-instance-20231005-143000` +- Starts container: `rad-git-sshd-frpc-my-app-instance-20231005-143000` +- Uses image: `snowdreamtech/frpc:0.62.0-alpine3.21` +- Mounts config volume: `/etc/frp` + +#### Proxy Server Configuration (from update-compose-config.sh) +The script updates `docker-compose.app.yml` with: + +**Ports Section:** +```yaml +ports: + # ... other ports ... + + # my-app-instance-20231005-143000 Self-hosting ports ==== + - "8779:8779/tcp" + - "13076:13076/tcp" + - "13077:13077/tcp" + # ========================= + + # ... other ports ... +``` + +**Labels Section:** +```yaml +labels: + # ... other labels ... + + # my-app-instance-20231005-143000 Self-hosting labels ==== + - "traefik.http.routers.frps-my-app-instance-20231005-143000-http.entrypoints=web" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-http.rule=Host(`myapp.example.com`)" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-http.service=frps-my-app-instance-20231005-143000" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-http.middlewares=http2https@file" + + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.entrypoints=websecure" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.rule=Host(`myapp.example.com`)" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.service=frps-my-app-instance-20231005-143000" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.tls=true" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.tls.certresolver=letsencrypt" + - "traefik.http.routers.frps-my-app-instance-20231005-143000-https.tls.domains[0].main=myapp.example.com" + + - "traefik.http.services.frps-my-app-instance-20231005-143000.loadbalancer.server.port=13077" + # ========================= + + # ... other labels ... +``` + +#### Backup and Service Restart (from update-compose-config.sh) +- Creates backup: `docker-compose.app.yml.backup_YYYYMMDD_HHMMSS` +- Restarts service with: `./stop.sh && ./start.sh` +- Verifies deployment with: `docker logs --tail 50 frps_frps_1` + +## Extensibility Points + +### Protocol Support +- **TCP/UDP**: Native FRP support for any TCP/UDP based service +- **HTTP/HTTPS**: Enhanced routing via Traefik with automatic certificates +- **Custom Protocols**: Extensible architecture for proprietary protocols + +### Service Types +- **Web Applications**: HTTP/HTTPS routing with domain-based access +- **Database Services**: Secure TCP tunneling for database connections +- **SSH Services**: Encrypted access to internal SSH servers +- **API Services**: Secure API endpoint exposure + +### Scaling Options +- **Load Balancing**: Multiple instances behind single domain +- **Geographic Distribution**: Multiple proxy servers in different regions +- **Protocol Multiplexing**: Multiple services through single tunnel + +## Security Considerations +- Encrypted tunnels between internal and external networks +- Access control through SSH keys and authentication +- Firewall isolation of internal services +- Automatic certificate management for HTTPS +- Docker container isolation + +## Maintenance Requirements +- Regular backup of configuration files +- Certificate renewal monitoring +- Connection health monitoring +- Log aggregation and analysis +- Docker image updates + +## Generic Application Examples + +### Web Application Deployment +- **Parameters**: Instance name, internal web port (80/443), domain name +- **Outcome**: Public website accessible via configured domain + +### Database Access +- **Parameters**: Instance name, internal database port, restricted domain +- **Outcome**: Secure external access to internal database + +### Custom Service Exposure +- **Parameters**: Instance name, custom application port, specific domain +- **Outcome**: External access to internal custom service \ No newline at end of file diff --git a/Design_Doc/auto-sys-deploy.sh.md b/Design_Doc/auto-sys-deploy.sh.md new file mode 100644 index 0000000..893f680 --- /dev/null +++ b/Design_Doc/auto-sys-deploy.sh.md @@ -0,0 +1,61 @@ +# auto-sys-deploy.sh Documentation + +## Overview +The `auto-sys-deploy.sh` script is an automated system deployment script for FRPC (Fast Reverse Proxy Client). It's designed to set up and configure FRP client instances with interactive prompts for user input. The script handles instance configuration, Docker container management, and updates the proxy server configuration automatically. + +## Features +- Interactive instance name selection with validation +- Docker network selection with support for existing networks +- FRP client configuration generation with presets +- Docker container lifecycle management +- Automatic proxy server configuration updates +- Support for multiple proxy configurations (rad, ssh, web) +- Backup and restore of existing configurations +- Comprehensive logging with color-coded output + +## Functions + +### Core Functions +- `prompt_instance_name()`: Handles user prompts for instance name, with validation and existing instance detection +- `select_docker_network()`: Allows user to select Docker network with options for custom networks +- `generate_frpc_config()`: Creates FRP client configuration (frpc.toml) with three proxy types +- `start_docker_container()`: Manages the Docker container lifecycle (create/start) +- `update_proxy_server_config()`: Updates the proxy server configuration remotely via SSH + +### Utility Functions +- `log_info()`, `log_error()`, `log_warn()`: Color-coded logging functions +- `save_parameters()`, `load_parameters()`: Parameter persistence to Docker volumes +- `extract_frp_server_address()`, `extract_remote_ports()`: Configuration retrieval functions +- `display_container_commands()`: Shows useful Docker commands for the deployed instance + +## Usage +``` +./auto-sys-deploy.sh [COMMAND] [OPTIONS] + +Available commands: + setup-frpc + +Available options: + -l, --list List all available systems + -h, --help Show this help message +``` + +## Parameters +- **Instance Name**: Alphanumeric, hyphens, and underscores only +- **Docker Network**: Existing Docker network or custom network name +- **FRP Server Address**: IP address of the FRP server +- **FRP Server Port**: Default is 16999 +- **Proxy Ports**: Configurable remote ports for rad (default 8776), ssh (default 22), and web (default 80) proxies + +## Architecture +The script uses Docker containers with the `snowdreamtech/frpc:0.62.0-alpine3.21` image and creates persistent volumes with the naming convention `rad-git-sshd-itl-frpc-` to store configuration files. + +## Prerequisites +- Docker and Docker Compose installed +- SSH key at `~/.ssh/itl-devs1` for proxy server access +- `DOWNLOAD_SOURCE` environment variable set +- User permissions for Docker commands +- SSH keys for authentication (`~/.ssh/authorized_keys` and `~/.ssh/devs1`) + +## Configuration Persistence +The script stores network selection preferences in a `.para` file within the Docker volume, allowing for configuration continuity between runs. \ No newline at end of file diff --git a/Design_Doc/instructions.md b/Design_Doc/instructions.md new file mode 100644 index 0000000..e69de29 diff --git a/Design_Doc/update-compose-config.sh.md b/Design_Doc/update-compose-config.sh.md new file mode 100644 index 0000000..a817bf6 --- /dev/null +++ b/Design_Doc/update-compose-config.sh.md @@ -0,0 +1,83 @@ +# update-compose-config.sh Documentation + +## Overview +The `update-compose-config.sh` script is designed to update the `docker-compose.app.yml` file on a proxy server, specifically for managing FRP (Fast Reverse Proxy) server configurations. It adds or updates port mappings and Traefik labels for new or existing instances, and can optionally restart the Docker Compose service. + +## Features +- Command-line argument parsing for configuration parameters +- Port configuration management in docker-compose files +- Traefik label configuration for HTTP/HTTPS routing +- Backup creation before making changes +- Status checking (not_found, up_to_date, outdated, incomplete) +- Automatic service restart capability +- Safe configuration updates with validation + +## Functions + +### Core Functions +- `parse_args()`: Parses command-line arguments to configure instance parameters +- `check_ports_config_status()`: Checks if ports configuration is current +- `check_labels_config_status()`: Checks if labels configuration is current +- `update_ports_config()`: Updates or adds port mappings in the compose file +- `update_labels_config()`: Updates or adds Traefik labels in the compose file +- `restart_docker_compose()`: Restarts the Docker Compose service + +### Utility Functions +- `usage()`: Displays help information and usage examples +- `find_section_insertion_point()`: Determines where to insert new configurations +- `update_compose_config()`: Main function that coordinates all updates + +## Usage +``` +./update-compose-config.sh [OPTIONS] + +Options: + -f, --file FILE Path to docker-compose.app.yml (default: /data/ssd/rvs-prxy/frps/apps/zzthis/docker-compose.app.yml) + -i, --instance NAME Instance name for the configuration + -r, --rad-port PORT Remote port for rad proxy + -s, --ssh-port PORT Remote port for ssh proxy + -w, --web-port PORT Remote port for web proxy + -d, --domain DOMAIN Domain name for Traefik routing + -h, --help Show this help message + +Example: + ./update-compose-config.sh -i myinstance -r 8779 -s 13076 -w 13077 -d example.com +``` + +## Parameters +- **COMPOSE_FILE**: Path to the docker-compose.app.yml file (default: `/data/ssd/rvs-prxy/frps/apps/zzthis/docker-compose.app.yml`) +- **INSTANCE_NAME**: Name of the instance being configured +- **RAD_PORT**: Remote port for rad proxy +- **SSH_PORT**: Remote port for SSH proxy +- **WEB_PORT**: Remote port for web proxy +- **DOMAIN_NAME**: Domain name for Traefik routing rules + +## Configuration Sections +The script manages two main configuration sections in the docker-compose file: + +### Ports Section +- Adds port mappings in the format: `:/tcp` +- Marked with comment markers for easy identification and updates + +### Labels Section +- Configures Traefik routing rules for HTTP and HTTPS +- Sets up TLS with Let's Encrypt certificate resolver +- Configures load balancer to route to the appropriate web port + +## Architecture +The script uses strategic comment markers to identify and update specific sections: +- `# Self-hosting ports ====` +- `# Self-hosting labels ====` +- `# =========================` + +This allows for precise updates without affecting other configurations in the file. + +## Safety Features +- Creates timestamped backups before making changes +- Validates required arguments before proceeding +- Checks configuration status before updating +- Provides option to restart services after configuration +- Safe insertion and removal of configuration blocks + +## Service Restart +The script can optionally restart the Docker Compose service after making changes, with support for custom start/stop scripts or standard docker-compose commands. \ No newline at end of file