rvs-prxy/Design_Doc/procedure.md

# Procedure Guide: Connecting Internal Services to Public Domains

## Overview
This document provides a standardized procedure for connecting internal services to public domains using the reverse proxy architecture. This checklist can be used for any future application deployments following the same pattern. The process includes environment file support for default values.

The document includes:
1. Environment Configuration: How to use .env files for default values
2. Step-by-step Demo: A practical example of connecting an internal service
3. Reusable Procedure: Checklist for future applications

## Environment Configuration

The deployment process supports environment-based configuration using `.env` files to provide default values:

- **Environment Files Supported**: `.env`, `.env.development`, `.env.production`
- **Default Priority**: `.env.production` → `.env.development` → `.env`
- **Auto-Persistence**: Answered parameters are automatically saved to the appropriate `.env` file for future use
- **Interactive Fallback**: If parameters are missing from environment files, scripts will prompt for input

### Supported Environment Variables
- `INSTANCE_NAME`: Default instance name
- `INTERNAL_SERVICE_PORT`: Port exposed by the target Docker container (e.g., 3000)
- `DOCKER_NETWORK`: Docker network for connecting to internal service container
- `FRP_SERVER_ADDR`: FRP server address
- `FRP_SERVER_PORT`: FRP server port (default 16999)
- `DOMAIN_NAME`: Default domain name
- `RAD_REMOTE_PORT`: Remote port for rad proxy (default 8779)
- `SSH_REMOTE_PORT`: Remote port for SSH proxy (default 13076) 
- `WEB_REMOTE_PORT`: Remote port for web proxy (default 13077)


## Internal Service Connection

### Prerequisites for Internal Service
- Target service running on the same computer as the script execution
- Target service accessible on the specified port (e.g., port 3000) via 127.0.0.1
- Script runs on the same physical/virtual machine as the target service

## Demo Scenario
For this demo, let's assume:
- Internal service (e.g., a web application) running on the same computer, accessible at `127.0.0.1:3000`
- Domain name: `myapp.example.com`
- FRP server IP: `192.168.1.100`
- Script executes on the same computer as the target service

## Step-by-Step Demo

### Step 1: Prepare Your Environment
1. Ensure Docker is running:
   ```bash
   docker ps
   ```

2. Verify the target service is accessible locally:
   ```bash
   curl http://127.0.0.1:3000
   # Or telnet 127.0.0.1 3000
   ```

3. Verify SSH key exists:
   ```bash
   ls -la ~/.ssh/itl-devs1
   ```

4. Prepare environment files with default values:
   Create a `.env.development` file in the script directory:
   ```bash
   INSTANCE_NAME=myapp-3000
   INTERNAL_SERVICE_PORT=3000
   DOCKER_NETWORK=app-network
   FRP_SERVER_ADDR=192.168.1.100
   DOMAIN_NAME=myapp.example.com
   RAD_REMOTE_PORT=8776
   SSH_REMOTE_PORT=13076
   WEB_REMOTE_PORT=13077
   ```

5. Verify environment file is in the correct location (same directory as script or current working directory)

### Step 2: Run the Deployment Script
Execute the automated deployment script via SSH:
```bash
ssh -i ~/.ssh/itl-devs1 -o StrictHostKeyChecking=accept-new -p 13076 ubuntu@cmty-repos2.ivantito.com "cat /repos/repos/cmptr-infra/rvs-prxy/auto-sys-deploy.sh" | bash -s -- setup-rvs-prxy
```

### Step 3: Configure Instance Name
- System will display: `=== Instance Setup ===`
- If `INSTANCE_NAME` exists in environment files, it will be used as default
- If not found in environment files, enter instance name: `myapp-3000` (or use the suggested default)
- If prompted, value will be saved to the appropriate `.env` file
- Validation ensures only alphanumeric characters, hyphens, and underscores

### Step 4: Select Docker Network
- System will display available Docker networks
- If `DOCKER_NETWORK` exists in environment files, it will be used as default
- If not found in environment files, select option matching your internal service's network or choose default
- If prompted, value will be saved to the appropriate `.env` file
- Example selection: `app-network` (if your service is on this network)

### Step 5: Configure Internal Service Port
- If `INTERNAL_SERVICE_PORT` exists in environment files, it will be used as default
- If not found in environment files, enter the port where your service is accessible locally: `3000`
- If prompted, value will be saved to the appropriate `.env` file

### Step 6: Configure FRP Server Details
- If `FRP_SERVER_ADDR` exists in environment files, it will be used as default
- If not found in environment files, enter FRP server address: `192.168.1.100`
- If prompted, value will be saved to the appropriate `.env` file
- If `FRP_SERVER_PORT` exists in environment files (default 16999), it will be used as default
- If not found in environment files, the default server port `16999` will be used (press Enter to confirm)
- If prompted, value will be saved to the appropriate `.env` file

### Step 7: Configure Proxy Settings
The script dynamically configures proxies based on environment variables following the format `PROXY_*_PORT`:

**Dynamic Proxy Configuration:**
- Proxies are configured using `PROXY_*_PORT` environment variables
- Format: `PROXY_API_PORT=8080`, `PROXY_DATABASE_PORT=5432`, `PROXY_CUSTOM_PORT=9000`, etc.
- Each `PROXY_*_PORT` variable creates a corresponding proxy with the name derived from the variable (e.g., `PROXY_API_PORT` creates an `api` proxy)
- Optional local port override: `PROXY_API_LOCAL_PORT=8000`, `PROXY_DATABASE_LOCAL_PORT=5433`, etc.
- If no `PROXY_*_PORT` variables are defined, a default proxy is created using the internal service port
- The script will create as many proxies as defined by the environment variables
- Example `.env` configuration:
  ```
  PROXY_WEB_PORT=8080
  PROXY_API_PORT=8081
  PROXY_DB_PORT=5432
  PROXY_WEB_LOCAL_PORT=3000
  PROXY_API_LOCAL_PORT=4000
  ```

### Step 8: Configure Domain Name
- If `DOMAIN_NAME` exists in environment files, it will be used as default
- If not found in environment files, enter domain name: `myapp.example.com`
- If prompted, value will be saved to the appropriate `.env` file

### Step 9: Confirm Deployment
- Confirm with "OK" to proceed with deployment

### Step 10: Monitor the Process
The script will:
1. Create Docker volume: `rad-git-sshd-itl-frpc-myapp-3000`
2. Generate FRP client configuration (`frpc.toml`) with the proxy settings
3. Configure the local IP to point to `127.0.0.1` on the specified port (3000)
4. Start the FRP client container: `rad-git-sshd-frpc-myapp-3000` on the selected Docker network
5. Update proxy server configuration automatically
6. Display container operation commands
7. Any prompted values will be saved to the appropriate `.env` file

### Step 11: Verify Deployment
The output will show:
- Container operation commands
- Configuration details
- Log access instructions
- Confirmation of successful deployment

### Step 12: Access Your Service
1. Ensure your domain (`myapp.example.com`) points to your FRP server IP (`192.168.1.100`)
2. Access your service via the configured domain
3. The service running locally on port 3000 will now be accessible via `https://myapp.example.com`
4. FRP client connects to your local service via 127.0.0.1

## Architecture Flow in Action

### 1. Configuration Phase
- The `auto-sys-deploy.sh` script created a `frpc.toml` configuration file
- This configuration defines tunnels from the public FRP server ports to your local service
- The local IP setting in FRP client points to `127.0.0.1`
- Environment values and prompted values are saved to appropriate `.env` files for future use

### 2. Deployment Phase  
- Docker container `rad-git-sshd-frpc-myapp-3000` was created with the FRP client
- The container connects to your FRP server at `192.168.1.100:16999`

### 3. Connection Phase
- FRP client connects to your local service using 127.0.0.1 as the hostname
- Traffic flows: External → FRP Server → FRP Client Container → Local Service (127.0.0.1:3000)

### 4. Routing Phase
- The `update-compose-config.sh` script updated the proxy server's `docker-compose.app.yml`
- Traefik configuration was added to route `myapp.example.com` to your service
- Automatic SSL/TLS certificate was provisioned via Let's Encrypt

### 5. Access Phase
- Public requests to `myapp.example.com` are handled by Traefik
- Traffic is forwarded through the FRP tunnel to your local service
- End users access your internal service via the public domain

## Pre-Deployment Checklist

### 1. Environment Verification
- [ ] Docker and Docker Compose are installed and running
- [ ] SSH key exists at `~/.ssh/itl-devs1`
- [ ] SSH key exists at `~/.ssh/itl-devs1` for proxy server access
- [ ] User has appropriate permissions for Docker commands
- [ ] Target service is running and accessible locally
- [ ] Target service is accessible on the specified port via 127.0.0.1 (e.g., 3000)
- [ ] FRP server IP and port are accessible from deployment location
- [ ] Domain name is available and will point to FRP server
- [ ] Environment file (.env.development) is prepared with required configuration
- [ ] INSTANCE_NAME will be used to create container and volume named {INSTANCE_NAME}-frpc

### 2. Environment File Preparation (Required)
- [ ] Create `.env.development` file in the script directory with default values
- [ ] Define environment variables such as:
  - `INSTANCE_NAME=myapp-3000`
  - `INTERNAL_SERVICE_PORT=3000`
  - `DOCKER_NETWORK=app-network`
  - `FRP_SERVER_ADDR=192.168.1.100`
  - `DOMAIN_NAME=myapp.example.com`

### 3. Application Information Preparation
- [ ] Service accessible locally at 127.0.0.1 on specified port
- [ ] Port number identified (e.g., 3000)
- [ ] Desired public domain name selected
- [ ] Desired instance name decided (alphanumeric, hyphens, underscores only)
- [ ] FRP server IP address confirmed

## Deployment Procedure

### Step 1: Execute Deployment Script via SSH
```bash
ssh -i ~/.ssh/itl-devs1 -o StrictHostKeyChecking=accept-new -p 13076 ubuntu@cmty-repos2.ivantito.com "cat /repos/repos/cmptr-infra/rvs-prxy/auto-sys-deploy.sh" | bash -s -- setup-rvs-prxy
```

### Step 2: Instance Configuration
- [ ] Script checks for `INSTANCE_NAME` in environment files
- [ ] If found, uses as default; if not found, prompts for input
- [ ] If prompted, value will be saved to appropriate `.env` file
- [ ] Validate instance name format (alphanumeric, hyphens, underscores only)

### Step 3: Network Configuration
- [ ] Script checks for `DOCKER_NETWORK` in environment files
- [ ] If found, uses as default; if not found, prompts for input
- [ ] If prompted, value will be saved to appropriate `.env` file
- [ ] Select Docker network where needed
- [ ] Confirm network selection

### Step 4: Internal Service Configuration
- [ ] Script checks for `INTERNAL_SERVICE_PORT` in environment files
- [ ] If found, uses as default; if not found, prompts for input
- [ ] If prompted, value will be saved to appropriate `.env` file
- [ ] Enter the port where service is accessible locally (e.g., 3000)

### Step 5: FRP Server Configuration
- [ ] Script checks for `FRP_SERVER_ADDR` in environment files
- [ ] If found, uses as default; if not found, prompts for input
- [ ] If prompted, value will be saved to appropriate `.env` file
- [ ] Script checks for `FRP_SERVER_PORT` in environment files (default 16999)
- [ ] If found, uses as default; if not found, uses default value
- [ ] If prompted, value will be saved to appropriate `.env` file

### Step 6: Proxy Configuration
For each proxy type, the script checks for environment variables:

- [ ] **Rad Proxy**: Checks for `RAD_REMOTE_PORT` in environment files (default 8779)
  - If found, uses as default; if not found, prompts for input
  - If prompted, value will be saved to appropriate `.env` file
  - Confirm default port 8776 or enter custom port

- [ ] **SSH Proxy**: Checks for `SSH_REMOTE_PORT` in environment files (default 13076)
  - If found, uses as default; if not found, prompts for input
  - If prompted, value will be saved to appropriate `.env` file
  - Enter custom port (e.g., 13076) or confirm default 22

- [ ] **Web Proxy**: Checks for `WEB_REMOTE_PORT` in environment files (default 13077)
  - If found, uses as default; if not found, prompts for input
  - If prompted, value will be saved to appropriate `.env` file
  - Enter port that will be accessible via domain (e.g., 13077)
  - **This will connect to the local service via 127.0.0.1**

### Step 7: Domain Configuration
- [ ] Script checks for `DOMAIN_NAME` in environment files
- [ ] If found, uses as default; if not found, prompts for input
- [ ] If prompted, value will be saved to appropriate `.env` file
- [ ] Enter the domain name for public access (e.g., `myapp.example.com`)

### Step 8: Confirmation and Deployment
- [ ] Review deployment summary
- [ ] Confirm deployment with "OK"
- [ ] Wait for completion of all steps
- [ ] Verify "FRP client instance deployment completed" message

### Step 9: Post-Deployment Verification
- [ ] Verify FRP client container is running: `docker ps | grep rad-git-sshd-frpc-<instance-name>`
- [ ] Verify container logs: `docker logs --tail 10 rad-git-sshd-frpc-<instance-name>`
- [ ] Confirm container is connected to FRP server
- [ ] Verify proxy server configuration was updated
- [ ] Verify that any prompted values have been added to the appropriate `.env` file

## Post-Deployment Tasks

### 1. DNS Configuration
- [ ] Update DNS records to point domain to FRP server IP
- [ ] Allow time for DNS propagation (typically 1-4 hours)

### 2. Testing
- [ ] Test internal service accessibility
- [ ] Test public domain access
- [ ] Verify SSL certificate is properly provisioned
- [ ] Test all configured proxy ports

### 3. Documentation
- [ ] Record instance name for future reference
- [ ] Document container name and volume name
- [ ] Note the deployment-specific ports and domain
- [ ] Save the displayed container operation commands

## Troubleshooting Common Issues

### Issue: Connection Refused
**Solution:** 
- Verify the FRP server IP and port are accessible
- Check that the internal service is running on the expected port
- Ensure the Docker container is running: `docker ps | grep myapp`

### Issue: Domain Not Resolving
**Solution:**
- Verify DNS settings point to the FRP server IP
- Check Traefik configuration in `docker-compose.app.yml`
- Review the domain name configuration in the labels section

### Issue: SSL Certificate Issues
**Solution:**
- Check that ports 80 and 443 are accessible on the FRP server
- Verify Let's Encrypt can reach your domain for certificate validation
- Check Traefik logs for certificate-related errors

## Container Management Commands

After deployment, use these commands for container management:

### Start/Stop Commands
```bash
# Start container
docker start rad-git-sshd-frpc-<instance-name>

# Stop container
docker stop rad-git-sshd-frpc-<instance-name>

# Restart container
docker restart rad-git-sshd-frpc-<instance-name>
```

### Access and Monitoring Commands
```bash
# Access container shell
docker exec -it rad-git-sshd-frpc-<instance-name> sh

# View logs
docker logs --tail 50 rad-git-sshd-frpc-<instance-name>

# Inspect container
docker inspect rad-git-sshd-frpc-<instance-name>
```

### Removal Commands
```bash
# Remove container (stops if running)
docker rm -f rad-git-sshd-frpc-<instance-name>

# Remove associated volume (CAUTION: This will lose configuration)
docker volume rm rad-git-sshd-itl-frpc-<instance-name>
```

## Troubleshooting Checklist

### Connection Issues
- [ ] Verify FRP server IP and port accessibility
- [ ] Check container status: `docker ps | grep <instance-name>`
- [ ] Review container logs for errors
- [ ] Confirm FRP client configuration is valid

### Domain/SSL Issues
- [ ] Verify DNS records point to correct IP
- [ ] Check Traefik logs on proxy server
- [ ] Confirm ports 80/443 are accessible
- [ ] Verify domain name in configuration files

### Configuration Issues
- [ ] Review `frpc.toml` configuration in volume
- [ ] Confirm all ports are correctly mapped
- [ ] Check for conflicts with existing instances
- [ ] Verify proxy server configuration updates

## Rollback Procedure

If deployment fails or needs to be reverted:

1. Stop the container:
   ```bash
   docker stop rad-git-sshd-frpc-<instance-name>
   ```

2. Remove the container:
   ```bash
   docker rm rad-git-sshd-frpc-<instance-name>
   ```

3. If needed, remove the associated volume:
   ```bash
   docker volume rm rad-git-sshd-itl-frpc-<instance-name>
   ```

4. Revert proxy server configuration (if automatic update occurred):
   - Restore from backup: `docker-compose.app.yml.backup_<timestamp>`
   - Restart Docker Compose service

## Container Management Commands

After deployment, use these commands for container management:

### Start/Stop Commands
```bash
# Start container
docker start rad-git-sshd-frpc-<instance-name>

# Stop container
docker stop rad-git-sshd-frpc-<instance-name>

# Restart container
docker restart rad-git-sshd-frpc-<instance-name>
```

### Access and Monitoring Commands
```bash
# Access container shell
docker exec -it rad-git-sshd-frpc-<instance-name> sh

# View logs
docker logs --tail 50 rad-git-sshd-frpc-<instance-name>

# Inspect container
docker inspect rad-git-sshd-frpc-<instance-name>
```

### Removal Commands
```bash
# Remove container (stops if running)
docker rm -f rad-git-sshd-frpc-<instance-name>

# Remove associated volume (CAUTION: This will lose configuration)
docker volume rm rad-git-sshd-itl-frpc-<instance-name>
```

## Troubleshooting Checklist

### Connection Issues
- [ ] Verify FRP server IP and port accessibility
- [ ] Check container status: `docker ps | grep <instance-name>`
- [ ] Review container logs for errors
- [ ] Confirm FRP client configuration is valid

### Domain/SSL Issues
- [ ] Verify DNS records point to correct IP
- [ ] Check Traefik logs on proxy server
- [ ] Confirm ports 80/443 are accessible
- [ ] Verify domain name in configuration files

### Configuration Issues
- [ ] Review `frpc.toml` configuration in volume
- [ ] Confirm all ports are correctly mapped
- [ ] Check for conflicts with existing instances
- [ ] Verify proxy server configuration updates

## Rollback Procedure

If deployment fails or needs to be reverted:

1. Stop the container:
   ```bash
   docker stop rad-git-sshd-frpc-<instance-name>
   ```

2. Remove the container:
   ```bash
   docker rm rad-git-sshd-frpc-<instance-name>
   ```

3. If needed, remove the associated volume:
   ```bash
   docker volume rm rad-git-sshd-itl-frpc-<instance-name>
   ```

4. Revert proxy server configuration (if automatic update occurred):
   - Restore from backup: `docker-compose.app.yml.backup_<timestamp>`
   - Restart Docker Compose service