CTFd 多服务 WEB dynamic docker compose task实现

前言

一年前在出多服务web题时候的想法是，把一个服务当做基础的docker0，在这个docker里再塞一个docker1，在docker1中实现第二个服务，但是考虑到嵌套的性能和docker network的各种原因最后放弃了。

前阵子我们在主站上新加了CTFd-owl，使得docker-compose成为了可能。直接把两个服务塞进docker-compose里，但等到部署的时候发现了一个问题，包括CTFd-whale和CTFd-owl在内的动态容器插件，大多都只能映射出来一个随机端口，但如果多个服务都需要对外暴露怎么办？在各种尝试之下完成了下面这个不用再次魔改插件的dynamic-docker-compose-task板子

ENV

• CTFd 3.5.3
• CTFd-owl

Implementation

首先我们需要明确需要扔进docker-compose的服务有哪些，在这里我用的是python+mysql+ChatGPT-Next-Web的组合，也是一个简单的ssrf的题目

scr1wgpt使用的是ChatGPT-Next-Web的2.11.2版本，但是进行了细微的魔改，这个之后再说，在这里需要一些trick来部署

scr1wgpt-web使用的是基础python+flask，没什么多余的东西。在这里作为基础服务

scr1wgpt-mysql使用的也是基础mysql，但是同样需要进行一些调整

scr1wgpt-nginx需要提前将nginx.conf放进docker中，来做反代

为什么要使用nginx

一方面是，web和gpt两个服务都需要暴露端口，但是在不修改插件的前提下很难做到，所以需要用nginx做反代，这里是将scr1wgpt-web作为基础的服务，不用做什么修改，而将scr1wgpt这个服务扔到/chat路由下

scr1wgpt

由于我们把整个服务的都扔到了/chat的路由，所以需要对/#/auth，/#/setting等路径进行转换，全部转换成/chat/#/...编译后重新打包成一个镜像即可，这里是shiraikuroko/gpt

scr1wgpt-web

app.py

from flask import Flask, request, render_template, redirect, url_for, session
from sqlalchemy import create_engine
from sqlalchemy.engine.url import make_url
import pymysql.cursors
import time
import os
import socket
from flask_limiter import Limiter
from flask_limiter.util import get_remote_address

app = Flask(__name__)
app.secret_key = os.urandom(64)
# limiter = Limiter(
#     app=app,
#     key_func=get_remote_address,  # 使用客户端IP作为限制的键
#     default_limits=["5 per minute"]  # 默认限制，每分钟最多5次请求
# )

# 等待数据库初始化上线
url = make_url("mysql+pymysql://root:sssctf2024_rootpassword@scr1wgpt-mysql/sssctf2024_db")
url.database = None
engine = create_engine(url)
print(f"等待数据库就绪。。。", end="", flush=True)
while True:
    try:
        engine.raw_connection()
        break
    except Exception:
        print("。", end="", flush=True)
        time.sleep(3)
print("", flush=True)

# MySQL 连接配置
conn = pymysql.connect(
    host='scr1wgpt-mysql',
    user='root',
    password='sssctf2024_rootpassword',
    database='sssctf2024_db',
    cursorclass=pymysql.cursors.DictCursor
)

@app.route('/')
def index():
    return redirect(url_for('login'))

@app.route('/login', methods=['GET', 'POST'])
# @limiter.limit("10 per minute")  # Limits to 10 login attempts per minute
def login():
    msg = ''
    if request.method == 'POST':
        username = request.form['username']
        password = request.form['password']

        waf = ['or', 'and', '/', '0x', '0b', '0o', ';', 'outfile', 'load_file', 'terminated', 'field']
        ulow = username.lower()
        plow = password.lower()
        for waff in waf:
            if waff in ulow or waff in plow:
                msg = "Login failed!"
                return render_template('login.html', msg=msg)

        cur = conn.cursor()
        query = f"SELECT * FROM sssctf2024_users WHERE username='{username}' AND password='{password}' LIMIT 0,1"

        try:
            cur.execute(query)     # 执行SQL查询
            result = cur.fetchall()         # 获取所有结果
            if result:
                if not (username == 'Scr1w_admin' and password == 'sssctf2024_P@ssvv0rd'):
                    msg = 'Login failed!'
                    return render_template('login.html', msg=msg)

                session['username'] = username  # 将用户名存储在会话中
                return redirect(url_for('options'))
            else:
                msg = 'Login failed!'

        except Exception as error:
            print(f"An error occurred: {error}", flush=True)
            msg = 'Login failed!'

    return render_template('login.html', msg=msg)

@app.route('/options')
def options():
    if 'username' not in session:
        return redirect(url_for('login'))
    return render_template('options.html')

def get_scr1wgpt_ip():
    try:
        return socket.gethostbyname('scr1wgpt')
    except socket.gaierror:
        return None

@app.route('/generator')
def flag():
    scr1wgpt_ip = get_scr1wgpt_ip()
    if scr1wgpt_ip is None:
        return 'Unable to resolve scr1wgpt domain'
    if request.remote_addr != scr1wgpt_ip:
        return render_template('generator.html', message='Remote test flag has been generated:\nSsS(tF{f@k3_f|4g_l-lAl-lA]')
    flag_value = os.getenv('FLAG', 'Flag not set')
    return render_template('generator.html', message="Local flag has been generated:\n" + flag_value)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port='80')

Dockerfile

FROM python:3.9-slim-buster

RUN mkdir -p /app
COPY ./templates /app/templates
COPY ./app.py /app/app.py
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app

RUN sed -i "s|http://deb.debian.org/debian|https://mirror.sjtu.edu.cn/debian|g" /etc/apt/sources.list \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
        build-essential \
        libffi-dev \
        libssl-dev \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* 

RUN pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

不用做其他的适配，wp可以看后面的链接

scr1wgpt-mysql

在这里遇到过很多坑，也是这个地方修的最久，数据库的初始化一直有问题，猜测是由于服务器性能不行，mysql的容器启动太久，导致初始化有问题，所以这里选择了另一种方式

init.sql

CREATE DATABASE IF NOT EXISTS sssctf2024_db;

USE sssctf2024_db;

CREATE TABLE IF NOT EXISTS sssctf2024_users (
    id INT AUTO_INCREMENT PRIMARY KEY,
    username VARCHAR(255) NOT NULL,
    password VARCHAR(255) NOT NULL
);

INSERT INTO sssctf2024_users (username, password) VALUES ('Scr1w_admin', 'sssctf2024_P@ssvv0rd');

SQLDocketfile

FROM mysql:latest

COPY ./init.sql /docker-entrypoint-initdb.d/

scr1wgpt-nginx

这里的nginx是最关键的一部分，需要把整个gpt的服务都放到/chat这个路由下

nginx.conf

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    sendfile on;

    upstream web_backend {
        server scr1wgpt-web;
    }

    upstream chat_backend {
        server scr1wgpt:3000;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://web_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /chat {
            proxy_pass http://chat_backend/#/chat;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /_next {
            proxy_pass http://chat_backend$request_uri;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /serviceWorkerRegister.js {
            proxy_pass http://chat_backend/serviceWorkerRegister.js;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /site.webmanifest {
            proxy_pass http://chat_backend/site.webmanifest;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /serviceWorker.js {
            proxy_pass http://chat_backend/serviceWorker.js;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /favicon.ico {
            proxy_pass http://chat_backend/favicon.ico;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /api {
            proxy_pass http://chat_backend$request_uri;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /prompts.json {
            proxy_pass http://chat_backend/prompts.json;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        location /google-fonts {
            proxy_pass http://chat_backend$request_uri;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

但是这里又出了同样的问题，初始化复制进去大概率访问不到，所以还是需要提前打包

Dockerfile

FROM nginx:latest

COPY ./nginx.conf /etc/nginx/nginx.conf

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]

这样基本上就完成了

docker-compose

version: '3'
services:
  scr1wgpt-web:
    build: .
    restart: always
    command: python /app/app.py
    env_file:
      - .env
    environment:
      - FLAG=${FLAG}
    depends_on:
      - scr1wgpt-mysql

  scr1wgpt-mysql:
    build:
      context: .
      dockerfile: SQLDockerfile
    restart: always
    command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --default-authentication-plugin=mysql_native_password
    environment:
      - MYSQL_ROOT_PASSWORD=sssctf2024_rootpassword

  scr1wgpt:
    image: shiraikuroko/gpt
    dns:
      - 202.118.66.6
    environment:
      - OPENAI_API_KEY=sk-xxx
      - BASE_URL=https://api.chatanywhere.tech
      - CODE=sssctf2024

  scr1wgpt-nginx:
    image: zedsich/scr1wgpt-nginx:latest
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/nginx.conf
    depends_on:
      - scr1wgpt-web
      - scr1wgpt
    ports:
        - "9999:80"

Link

github

wp