rvs-prxy/Design_Doc/Tech_Desc.md

559 lines
28 KiB
Markdown

# Technical Architecture and Implementation Details
## Script Architecture
The `deploy-rvs-prxy.sh` script is an automated deployment tool supporting both FRP and Traefik modes.
**Key Architecture:** The script runs directly on the same server where the application is hosted (local deployment), eliminating the need for SSH-based remote operations in Traefik mode.
**Deployment Modes:**
1. **Setup Reverse Proxy**: Deploys new Traefik or FRP Tunnel with Traefik
2. **Setup New Domain**: Adds new domain configuration to existing reverse proxy
**Configuration File Structure:**
- `.env` - Reverse proxy instance configuration (FRP mode, Traefik version, etc.)
- `.env-<domain>` - Domain-specific configuration (DOMAIN_NAME, LOCAL_SERVICE_NAME, LOCAL_SERVICE_PORT)
- Domain format: Any domain format supported (`.com`, `.net`, `.org`, `.org.hk`, `.com.hk`, subdomains, etc.)
- Filename pattern: Dots replaced with dashes (e.g., `app.example.com` → `.env-app-example-com`)
- Multiple domain files can coexist for multiple domains
**Automatic Mode Detection:**
- No config files → Interactive wizard for proxy setup
- `.env` exists, proxy not running → Setup Reverse Proxy
- `.env` exists, proxy running, no domain file → Interactive wizard for domain setup
- `.env` + `.env-<domain>` exists → Setup New Domain
**Validation:**
- **Proxy Setup Mode**: Validates proxy-specific settings (TRAEFIK_VER, LETSENCRYPT_EMAIL for Traefik; FRP_SERVER_ADDR for FRP)
- **Domain Setup Mode**: Validates domain-specific settings (DOMAIN_NAME, LOCAL_SERVICE_NAME, LOCAL_SERVICE_PORT)
**Workflow:**
First Run (No .env):
1. Interactive wizard asks for proxy instance settings
2. Creates .env with Traefik/FRP settings
3. Validation checks proxy settings (TRAEFIK_VER, LETSENCRYPT_EMAIL)
4. Deploys reverse proxy
5. Shows message: "Next: Run the script again to setup your first domain"
Second Run (.env exists, proxy running):
1. Detects running proxy
2. Interactive wizard asks for domain settings
3. Creates .env-<domain> with domain settings
4. Validation checks domain settings (DOMAIN_NAME, LOCAL_SERVICE_NAME, LOCAL_SERVICE_PORT)
5. Checks if application service is running
6. Configures domain in existing reverse proxy
7. Shows: "Service will be accessible at: https://domain.name"
### Core Components
**1. Environment Loading (`load_env`)**
- Loads configuration from `.env.development`, `.env.production`, or `.env`
- Supports files in current directory or script directory
- **Interactive Setup Wizard**: If no `.env` file is found, automatically launches a guided setup wizard
- Normalizes boolean values (FRP variable)
- Validates required parameters for each mode
**Interactive Setup Wizard Features:**
- Step-by-step guided configuration
- Mode selection (Traefik or FRP)
- Mode-specific configuration prompts
- Sensible defaults for all settings
- Automatic saving to `.env` file for future runs
**2. Instance Management (`prompt_instance_name`)**
- Auto-detects existing instances from Docker volumes (FRP mode)
- Supports instance name override via `INSTANCE_NAME` environment variable
- Generates timestamped default names
- Saves selections to `.env` for future runs
**3. Docker Network Selection (`select_docker_network`)**
- Lists available Docker networks
- Supports custom network names
- Remembers previous selections via `.para` file in volume (FRP mode)
**4. Configuration Generation**
- **FRP Mode**: Generates `frpc.toml` with dynamic proxy configurations
- **Traefik Mode**: Generates YAML configuration files for routing rules (local deployment)
**5. Container Management**
- Creates and manages Docker containers with persistent volumes
- Supports container restart and lifecycle operations
- Displays operation commands for user reference
## Traefik Mode Implementation (Local Deployment)
### Setup Flow (`setup_traefik_only`)
1. **Pre-request Checks**
- Verify Docker command permissions
2. **Application Service Verification** (`check_app_service_local`)
- Confirms target service is running locally
- Verifies connection to `traefik-public` network
3. **Traefik Container Creation** (`create_traefik_container_local`)
- Creates `traefik_data` volume for persistence
- Generates dynamic configuration with:
- Dashboard routing
- HTTP to HTTPS redirect
- Security headers
- Creates `traefik-public` Docker network
- Uses Docker Compose for container orchestration
- Detects Docker version for command compatibility (`docker compose` vs `docker-compose`)
4. **Service Configuration** (`configure_traefik_local`)
- Generates service-specific routing rules
- Supports sub-path routing (e.g., `example.com/api`)
- Creates unique configuration filenames to avoid conflicts
- Supports dynamic configuration loading (no restart required by default)
### Local Docker Operations
- `local_docker_exec()`: Executes Docker commands directly on the local server
- `local_volume_copy()`: Copies files to Docker volumes using temporary containers
### Configuration File Structure
**Traefik Main Config** (`traefik.yml`):
```yaml
http:
routers:
api:
rule: "PathPrefix(`/api`)"
service: api@internal
dashboard:
rule: "PathPrefix(`/dashboard`)"
service: api@internal
http-catchall:
rule: "PathRegexp(`{any:.*}`)"
entryPoints:
- "web"
service: "noop@internal"
middlewares:
- "redirect-to-https"
middlewares:
api-headers:
headers:
customFrameOptionsValue: SAMEORIGIN
contentTypeNosniff: true
browserXssFilter: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 15552000
redirect-to-https:
redirectScheme:
scheme: "https"
permanent: true
```
**Service Config** (dynamic, per-service):
```yaml
http:
routers:
<service-name>-https:
rule: 'Host(`<domain>`)'
service: <service-name>-service
tls:
certResolver: letsencrypt
entryPoints:
- websecure
<service-name>-http:
rule: 'Host(`<domain>`)'
service: <service-name>-service
entryPoints:
- web
middlewares:
- redirect-to-https
services:
<service-name>-service:
loadBalancer:
servers:
- url: 'http://<service-name>:<port>'
passHostHeader: true
```
## FRP Mode Implementation
### Setup Flow
1. **Instance Setup**: Prompt for instance name
2. **Port Configuration**: Prompt for internal service port
3. **FRP Config Generation** (`generate_frpc_config`)
- Extracts existing configuration from volume
- Supports dynamic proxy configurations via `PROXY_*_PORT` variables
- Generates `frpc.toml` with TCP proxy rules
4. **Container Start** (`start_docker_container`)
- Creates volume if not exists
- Starts frpc container with configuration mounted
5. **Proxy Server Update** (`update_proxy_server_config`)
- Uploads dynamic configuration script to proxy server
- Updates `docker-compose.app.yml` with port mappings and labels
- Supports domain-based routing configuration
### Dynamic Proxy Configuration
The script supports dynamic proxy configurations through environment variables:
- `PROXY_<NAME>_PORT`: Defines remote port for proxy
- `PROXY_<NAME>_LOCAL_PORT`: Defines local port (defaults to `INTERNAL_SERVICE_PORT`)
Example:
```bash
PROXY_WEB_PORT=12077
PROXY_WEB_LOCAL_PORT=3000
PROXY_SSH_PORT=2222
PROXY_SSH_LOCAL_PORT=22
```
### Proxy Server Configuration Update
The script generates and executes a remote script on the proxy server that:
- Checks existing configuration status (not_found, up_to_date, outdated)
- Updates port mappings section in `docker-compose.app.yml`
- Updates labels section for Traefik routing
- Optionally restarts Docker Compose service
## File Deployment (Traefik Mode)
The `deploy` command syncs files from local folder to target server:
- Source: `LOCAL_FOLDER` (default: `../out/`)
- Destination: `DEST_PATH` on target server
- Uses rsync for efficient synchronization with `--delete` option
## Logging and Debugging
**Logging Functions:**
- `log_info()`: Green [INFO] messages
- `log_error()`: Red [ERROR] messages
- `log_warn()`: Yellow [WARN] messages
- `log_debug()`: Blue [DEBUG] messages (only when `DEBUG=true`)
**Status Commands:**
- `check_status_on_target()`: Shows Traefik and Application service status
- `view_traefik_logs()`: Displays last 50 lines of Traefik logs
## Error Handling
- Uses `set -e` for immediate exit on errors
- Validates required parameters before execution
- Provides detailed error messages with suggested fixes
- Graceful handling of missing configurations
## Version Compatibility
**Docker Version Detection:**
- Detects Docker version to determine command syntax
- Uses `docker compose` for Docker 20.10.0+
- Falls back to `docker-compose` for older versions
- Handles Compose file version field (deprecated in v2.0+)
**Traefik Version:**
- Default: `v3.6.9`
- Configurable via `TRAEFIK_VER` environment variable
# AI Prompt
- The content is organised into distinct sections according to the topics below. (Grouped by section)
- Messages are sorted chronologically, with the most recent at the top and the oldest at the bottom. (Sorted by date: newest first, oldest last.)
## AI Prompt habit as a reminder
- Refer to `Design_Doc/instructions.md`
## Workflow
- After "First Run" to Setup Reverse Proxy Instance with Interactive wizard, it show the following issue:
```
--- Step 3: Setup Complete ---
Reverse proxy instance configuration saved to .env
Next: Run the script again to setup your first domain.
=============================================
Setup Complete!
Configuration saved to .env
=============================================
You can edit these settings later in the configuration file.
[INFO] Deployment mode: Setup Reverse Proxy (interactive setup)
[INFO] Setting up reverse proxy...
[INFO] Setting up reverse proxy...
[INFO] Setting up Traefik reverse proxy...
[INFO] Setting up Traefik-only proxy for instance:
[PRE-REQUEST] Make sure your current user has permission to execute docker command.
Auto-proceeding with Traefik setup...
[INFO] Checking if Application Service is running locally...
[ERROR] Application Service is not running locally:
[ERROR] Application Service not found locally. Please ensure the service '' is running.
```
- For the "First Run", it does not need to handle the Application Service issue, but It should be task of "setup domain". So adjust the code to right workflow following `Design_Doc/Spec.md` and `Design_Doc/Operation.md`.
- Result -- Code Fix (deploy-rvs-prxy.sh)
`setup_traefik_only()` function:
- Removed application service check (check_app_service_local)
- Removed call to configure_traefik_local (domain configuration)
- Now only deploys the Traefik reverse proxy instance
- Shows message: "Next: Run the script again to setup your first domain"
`setup_new_domain()` function:
- Added application service check (check_app_service_local)
- Added call to configure_traefik_local (domain configuration)
- Properly validates that application is running before configuring domain
- After "First Run" to Setup Reverse Proxy Instance with Interactive wizard, it show the following issue:
```
--- Step 3: Setup Complete ---
Reverse proxy instance configuration saved to .env
Next: Run the script again to setup your first domain.
=============================================
Setup Complete!
Configuration saved to .env
=============================================
You can edit these settings later in the configuration file.
[INFO] Deployment mode: Setup Reverse Proxy (interactive setup)
[ERROR] LOCAL_SERVICE_NAME and LOCAL_SERVICE_PORT are required for proxy setup
[ERROR] Please set these in your .env file
```
- The script code is running into setup domain which is not right at current workflow, the "setup domain" should be at "Second Run" to Setup New Domain, so adjust the code to right workflow following `Design_Doc/Spec.md` and `Design_Doc/Operation.md`.
- Result -- Code Fix (deploy-rvs-prxy.sh)
Fixed Validation Logic in `load_env()` function:
Before (Incorrect):
- Proxy setup mode was validating LOCAL_SERVICE_NAME and LOCAL_SERVICE_PORT (domain-specific settings)
- This caused the error after first-time setup wizard completed
After (Correct):
- Proxy Setup Mode validates proxy-specific settings:
- Traefik Mode: TRAEFIK_VER, LETSENCRYPT_EMAIL
- FRP Mode: FRP_SERVER_ADDR
- Domain Setup Mode validates domain-specific settings:
- DOMAIN_NAME, LOCAL_SERVICE_NAME, LOCAL_SERVICE_PORT
- Modify the workflow of the script as following the `Design_Doc/Spec.md` -> "Key Features" Section and its sub-section "Workflow".
Result -- 3. Code Changes
`load_env()` function:
- Loads .env for reverse proxy configuration
- Scans for .env-*-com files for domain configuration
- Automatically determines deployment mode based on files found
`save_domain_env()` function (new):
- Creates domain-specific configuration files
- Converts domain names to filename format (your-domain-com)
`interactive_setup_wizard()` function:
- Detects existing reverse proxy
- If proxy exists: guides through domain setup, creates .env-*-com
- If no proxy: guides through proxy setup, creates .env
`setup_new_domain()` function:
- Reports which domain configuration file was loaded
## Cmd structure
- I aim this auto. deployment script with available OPTIONS to form different functions
- Rename `deploy-traefik.sh` to `deploy-rvs-prxy.sh`.
- Remove the following: 1) `./deploy-traefik.sh setup-rvs-prxy`, 2) `FRP=false ./deploy-traefik.sh setup-rvs-prxy` 3) `FRP=true ./deploy-traefik.sh setup-rvs-prxy`. Just simple cmd as `curl -s "https://your.domain.com/your/path/deploy-rvs-prxy.sh" | bash`, then load the `.env`, if missing, triggering interactive wizard or if existing, following the `.env`. These both methods include ask the mode of a) traefik as `FRP=false ...` or b) FRP tunnel as `FRP=true ...`. So using interactive wizard approache or `.env` to replace the CLI with parameters to select mode.
## .env
- I am in the situation as 1) No reverse proxy exists, 2) no `.env`, 3) no `.env-<domain-with-dashes>`, it is right that go into Interactive wizard. In this situation, it should be "Setup Reverse Proxy", so the Interactive wizard is only handle the Reverse Proxy instance Configuration, and finally save those configuration into `.env`. No any Domain related Configuration. However, currently the script handle the Interactive wizard to show the following which is Domain configuation related:
```
--- Step 2: Traefik Configuration ---
Enter your Docker application container name:
Service name (e.g., my-app): traefik
Enter your application's internal port:
Service port (default: 3000):
Enter the domain name for your service:
Domain (e.g., app.example.com):
```
- Adjust the script code to follow the workflow from `Design_Doc/Spec.md` and `Design_Doc/Operation.md`.
- Adjust the content of `Design_Doc/Spec.md` -> "Configuration" Section -> "Key Environment Variables" Section to match the current new `.env` and `.env-any-domain-com`.
- Reminder: for `.env-any-domain-com`, actually it can accept any domain format, not only "any.domain.com", it can be, eg.: "any.domain.net", "any.domain.org.hk", "sub.sub.domain.com.hk", "any.domain.hk", etc.
- Result -- 4. Code Changes (deploy-rvs-prxy.sh)
- Updated load_env() to scan for .env-* pattern (any domain format) instead of just .env-*-com
- Skips .env.development and .env.production in domain file scanning
- save_domain_env() already handles any domain format by replacing all dots with dashes
Domain File Naming Examples
┌───────────────────────┬────────────────────────────┐
│ Domain │ Filename │
├───────────────────────┼────────────────────────────┤
│ example.com │ .env-example-com │
│ sub.domain.net │ .env-sub-domain-net │
│ app.org.hk │ .env-app-org-hk │
│ sub.sub.domain.com.hk │ .env-sub-sub-domain-com-hk │
└───────────────────────┴────────────────────────────┘
- For this auto. deployment script `deploy-rvs-prxy.sh`, it has two modes to execute: 1) setup reverse proxy, such as: traefik, FRP Tunnel with traefik, 2) setup new domain into reverse proxy.
- No matter which mode, when this auto. deployment script `deploy-rvs-prxy.sh` first running, it loads `.env`, a) if missing, trigger interactive wizard, b) if `.env` exists, load its content.
- By interactive wizard to ask question or one of parameters inside `.env` to decide running mode as a) setup reverse proxy or b) setup new domain.
- For setup reverse proxy, When this auto. deployment script `deploy-rvs-prxy.sh` loads the configuration no matter `.env` or interactive wizard, if it detects the existing one is running, skip all setup processes, and let end user to manually remove the existing one in case ready need to setup one new.
- For setup new domain mode, the reverse proxy must be running, just apply the new domain into the reverse proxy.
- `.env` will be kept all reverse proxy parameters.
- `.env-my-domain-com` will be kept all new domain parameters.
- Result
┌────────────────────────────────────────┬─────────────────────┬─────────────────────────────────┐
│ Condition │ Mode │ Action │
├────────────────────────────────────────┼─────────────────────┼─────────────────────────────────┤
│ No reverse proxy exists │ Setup Reverse Proxy │ Deploys new Traefik or FRP │
│ Reverse proxy exists + DOMAIN_NAME set │ Setup New Domain │ Adds domain to existing proxy │
│ Reverse proxy exists + no DOMAIN_NAME │ Setup Reverse Proxy │ Shows warning (already running) │
└────────────────────────────────────────┴─────────────────────┴─────────────────────────────────┘
- Inside `Design_Doc/Operation.md`, it has any examples, like `./deploy-traefik.sh --help`, how do i apply into `curl -s "https://www.ivantito.com/app-pub/demos/test/deploy-traefik.sh" | bash` (which is without any parameters) by adding `--help` parameter?
- Result
1. Code Changes (deploy-traefik.sh)
Main Function Restructure:
- Moved help/list argument checking before load_env function call
- This allows --help and --list to work without requiring .env file or triggering interactive wizard
- When no arguments provided, shows usage instead of triggering interactive setup
Usage Function Update:
- Added --help, -h and --list, -l to available commands
- Added remote execution (curl) examples:
1 curl -s "<url>/deploy-traefik.sh" | bash -s -- --help
2 curl -s "<url>/deploy-traefik.sh" | bash -s -- setup-rvs-prxy
3 curl -s "<url>/deploy-traefik.sh" | bash -s -- status
2. Documentation Updates
Design_Doc/Operations.md:
- Added new section "Using with curl (Remote Execution)" at the top of "Run this System"
- Shows all curl command examples with bash -s -- syntax
- Explains that bash -s -- allows passing arguments to a script read from stdin
- Includes examples for:
- --help - Show usage
- --list - List available systems
- setup-rvs-prxy - Run setup
- status - Check status
- logs - View logs
- Environment variable override with FRP=false
- Current situation as: i run this cmd with following issue, resolve it: `curl -s "https://www.ivantito.com/app-pub/demos/test/deploy-traefik.sh" | bash`, then return following:
```
[ERROR] No .env file found in current directory or script directory.
[WARN] Please create an .env.development file (or .env) with your configuration.
Example .env file:
```
- I aim to the loading `.env` workflow
- guided interactive setup in case missing `.env`
- If no `.env` configuration file exists, the script will guide you through an interactive setup process.
- Simple, step-by-step configuration wizard guides users -- Answer the prompted questions about your deployment requirements. The system will ask for essential variables such as instance name and DEBUG flag.
- Provides sensible default settings
- All responses are automatically saved to a `.env` file -- All settings are saved, will be reused in subsequent runs, eliminating the need for repeated configuration.
- OR if `.env` exists, load existing single configuration file -- `.env`
- uniform configurations
- Result
1. Code Changes (deploy-traefik.sh)
Added Interactive Setup Wizard (`interactive_setup_wizard` function):
- Detects missing .env file automatically
- Guides users through step-by-step configuration:
- Step 1: Deployment mode selection (Traefik or FRP)
- Step 2: Mode-specific configuration
- Traefik Mode: Service name, port, domain, Traefik version, Let's Encrypt email
- FRP Mode: Server address, port, internal service port, instance name
- Step 3: Optional settings (debug mode)
- All responses automatically saved to .env file
- Provides sensible defaults for all settings
Modified `load_env` function:
- Instead of exiting with an error when no .env is found, it now calls the interactive wizard
- After wizard completes, sources the newly created .env file
- Continues with normal validation and initialization
2. Documentation Updates
README.md:
- Added "Interactive Setup Wizard" to Key Features
- Updated Quick Start section with two options:
- Option 1: Interactive Setup (Recommended for First Time)
- Option 2: Using Configuration File
Design_Doc/Spec.md:
- Added "Interactive Setup Wizard" to Key Capabilities
Design_Doc/Operations.md:
- Expanded Quick Start section with detailed wizard description
- Shows what questions the wizard asks
- Explains that settings are saved for future use
Design_Doc/Tech_Desc.md:
- Updated Environment Loading component description
- Added Interactive Setup Wizard Features subsection
The script now provides a user-friendly experience for first-time users while maintaining backward compatibility with existing .env configurations.
- For the content of "Key Capabilities" / "Key Features" and "Quick Start" sections in `Readme.md`
- move and merge the `Readme.md` -> "Key Capabilities" / "Key Features" content to `Design_Doc/Spec.md`.
- move and merge the `Readme.md` -> "Quick Start" content to `Design_Doc/Operations.md`.
- Keep the `Readme.md` -> "Key Capabilities" / "Key Features" section and add a reference as "refer to `Design_Doc/Spec.md` -> Key Features"
- Keep the `Readme.md` -> "Quick Start" section and add a reference as "refer to `Design_Doc/Operations.md` -> Quick Start"
## Re-structure the whole project to run at local Target computer only
- Code
- For the code `deploy-traefik.sh`, original be `auto-sys-deploy.sh`, it should be this old block diagram i mentioned before:
```
+----------------------------+
| Target Server |
| |
| +---------+ |
| | | |
| | Traefik | |
+---------------------+ | | | |
| Any Linux Computer | | +---------+ |
| | | ^ |
| +-------------+ | | | |
| | | | Setup Reverse Proxy | +----+------------------+ |
| | This +-----+------------------------+->| | |
| | Automatical | | for your existing | | Reverse Proxy Setting | |
| | System |<-+ | application | | eg. www.domain.com | |
| | | | | | | | |
| +-------------+ | | | +------+----------------+ |
| | | | | |
| .env | | v |
| | | +-------------+ |
+---------------------+ | | | |
| | Existing | |
| | Application | |
| | | |
| +-------------+ |
| |
+----------------------------+
```
i aim to change this code to follow `Design_Doc/Spec.md` -> Key Features -> Approaches -> Traefik Only -> Architecture. I do not find you have this code modification at previous actions. So, modify the code `deploy-traefik.sh` too.
- Result -- Code Changes (deploy-traefik.sh)
Architecture Change: The script now runs directly on the same server where the application is hosted, eliminating SSH-based remote operations for Traefik mode.
Key modifications:
- Removed SSH-related variables (TARGET_SERVER, TARGET_USER, SSH_PORT, SSH_KEY_FILE_NAME, etc.)
- Added local deployment variables (LOCAL_SERVICE_NAME, LOCAL_SERVICE_PORT)
- Replaced ssh_exec() and scp_upload() with local_docker_exec() and local_volume_copy()
- Renamed functions to reflect local operations:
- check_app_service_on_target → check_app_service_local
- create_traefik_container_on_target → create_traefik_container_local
- configure_traefik_on_target → configure_traefik_local
- check_status_on_target → check_status_local
- view_traefik_logs → view_traefik_logs_local
- Removed the deploy command (file deployment via SSH)
- Updated usage help to reflect new environment variables
- Doc.
- I would like to move the `Readme.md` -> "Key Capabilities" section to `Design_Doc/Spec.md`.
- I would like to move the `Readme.md` -> "Quick Start" section to `Design_Doc/Operations.md`.
- In `Design_Doc/Tech_Desc.md`, i added "Technical Architecture and Implementation Details" Section and move all your previous writing under this section.
- I need to keep the `Design_Doc/Tech_Desc.md` -> "AI Prompt" section and let you un-touch, but you removed this section, You find a way to recover this section to me, and put back at the end of `Design_Doc/Tech_Desc.md`.