1734 lines
46 KiB
Markdown
1734 lines
46 KiB
Markdown
# GasFlux Web API 文档
|
||
|
||
## 概述
|
||
|
||
GasFlux Web API 是一个基于 Flask 的 RESTful API，用于上传数据文件、执行气体通量分析处理，并下载处理结果。该 API 支持异步处理，能够处理大量数据并提供实时状态监控。
|
||
|
||
## 快速开始
|
||
|
||
### 基础信息
|
||
|
||
- **基础 URL**: `http://localhost:5000`
|
||
- **认证**: 无需认证
|
||
- **数据格式**: JSON
|
||
- **文件大小限制**: 100MB
|
||
- **支持的文件类型**:
|
||
- 数据文件: `.xlsx`, `.xls`
|
||
- 配置文件: `.yaml`, `.yml`
|
||
|
||
### 完整工作流程示例
|
||
|
||
```python
|
||
import requests
|
||
import time
|
||
|
||
def process_gasflux_data():
|
||
# 1. 检查 API 健康状态
|
||
health_response = requests.get('http://localhost:5000/health')
|
||
print(f"API Status: {health_response.json()['status']}")
|
||
|
||
# 2. 上传数据文件
|
||
with open('data.xlsx', 'rb') as f:
|
||
files = {'file': f}
|
||
upload_response = requests.post('http://localhost:5000/upload', files=files)
|
||
|
||
result = upload_response.json()
|
||
task_id = result['job_id']
|
||
print(f"任务已创建: {task_id}")
|
||
|
||
# 3. 监控处理状态
|
||
while True:
|
||
status_response = requests.get(f'http://localhost:5000/task/{task_id}')
|
||
status = status_response.json()
|
||
|
||
print(f"Status: {status['status']} - {status['message']}")
|
||
|
||
if status['status'] == 'completed':
|
||
# 4. 下载结果文件
|
||
for result_file in status['results']:
|
||
download_url = f"http://localhost:5000{result_file['download_url']}"
|
||
download_response = requests.get(download_url)
|
||
|
||
with open(result_file['name'], 'wb') as f:
|
||
f.write(download_response.content)
|
||
print(f"Downloaded: {result_file['name']}")
|
||
break
|
||
elif status['status'] == 'failed':
|
||
print(f"Task failed: {status.get('error', '未知错误')}")
|
||
break
|
||
|
||
time.sleep(3) # 每3秒检查一次状态
|
||
```
|
||
|
||
## API 端点
|
||
|
||
### 🔍 监控和健康检查
|
||
|
||
#### 1. 获取健康状态
|
||
**端点**: `GET /health`
|
||
|
||
**描述**: 获取 API 的健康状态、系统信息和性能指标。健康检查会评估多个关键指标，当任何指标超出正常范围时，服务状态会标记为 `degraded`。
|
||
|
||
**健康检查逻辑**:
|
||
- **存储检查**: 验证上传和输出文件夹是否可写
|
||
- **负载检查**: 活跃任务数量超过20个时发出警告
|
||
- **错误率检查**: HTTP错误率超过10%时标记为不健康
|
||
- **综合评估**: 任何一项检查失败都会影响整体健康状态
|
||
|
||
**响应示例** - 健康状态 (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "健康检查完成",
|
||
"data": {
|
||
"status": "healthy",
|
||
"version": "1.0.0",
|
||
"timestamp": 1705257600.123,
|
||
"uptime": "2h 30m 15s",
|
||
"storage": {
|
||
"uploads_writable": true,
|
||
"outputs_writable": true
|
||
},
|
||
"tasks": {
|
||
"active_count": 2,
|
||
"total_tracked": 15,
|
||
"total_processed": 13,
|
||
"success_rate_percent": 92.31
|
||
},
|
||
"performance": {
|
||
"requests_per_second": 0.08,
|
||
"avg_response_time_ms": 234.56,
|
||
"error_rate_percent": 1.5
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**响应示例** - 不健康状态 (503):
|
||
```json
|
||
{
|
||
"code": 503,
|
||
"message": "服务不可用",
|
||
"data": {
|
||
"status": "degraded",
|
||
"version": "1.0.0",
|
||
"timestamp": 1705257600.123,
|
||
"uptime": "1h 30m 45s",
|
||
"storage": {
|
||
"uploads_writable": true,
|
||
"outputs_writable": true
|
||
},
|
||
"tasks": {
|
||
"active_count": 0,
|
||
"total_tracked": 5,
|
||
"total_processed": 3,
|
||
"success_rate_percent": 60.0
|
||
},
|
||
"performance": {
|
||
"requests_per_second": 0.12,
|
||
"avg_response_time_ms": 145.67,
|
||
"error_rate_percent": 50.0
|
||
},
|
||
"issues": [
|
||
"错误率过高 (50.0%)",
|
||
"活跃任务数量过多 (25)"
|
||
]
|
||
}
|
||
}
|
||
```
|
||
|
||
**状态码**:
|
||
- `200`: API 健康 (`status: "healthy"`)
|
||
- `503`: API 不健康 (`status: "degraded"`) - 服务不可用
|
||
|
||
**健康状态说明**:
|
||
- **healthy**: 所有检查通过，服务正常运行
|
||
- **degraded**: 部分检查失败，服务仍可运行但需要关注
|
||
- 错误率 > 10%: HTTP请求错误率过高
|
||
- 活跃任务 > 20: 系统负载过高
|
||
- 存储不可写: 文件系统权限问题
|
||
|
||
**字段说明**:
|
||
- `storage.uploads_writable`: 上传文件夹是否可写
|
||
- `storage.outputs_writable`: 输出文件夹是否可写
|
||
- `tasks.active_count`: 当前活跃的任务数量
|
||
- `performance.error_rate_percent`: HTTP请求错误率百分比
|
||
- `issues`: 当状态为degraded时的具体问题列表
|
||
|
||
---
|
||
|
||
#### 2. 获取系统统计信息
|
||
**端点**: `GET /stats`
|
||
|
||
**描述**: 获取详细的 API 统计信息、性能指标和系统监控数据，包括请求统计、任务状态、性能指标和系统资源使用情况。
|
||
|
||
**响应示例** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "统计信息获取成功",
|
||
"data": {
|
||
"summary": {
|
||
"uptime_seconds": 3600.5,
|
||
"uptime_formatted": "1h 0m 0s",
|
||
"requests_total": 150,
|
||
"requests_per_second": 0.04,
|
||
"error_rate_percent": 2.0,
|
||
"active_tasks": 1
|
||
},
|
||
"requests": {
|
||
"by_method": {
|
||
"GET": 120,
|
||
"POST": 30
|
||
},
|
||
"by_status": {
|
||
"200": 145,
|
||
"400": 3,
|
||
"500": 2
|
||
},
|
||
"top_endpoints": {
|
||
"/task/abc123": 45,
|
||
"/health": 30,
|
||
"/": 25
|
||
}
|
||
},
|
||
"tasks": {
|
||
"total_created": 25,
|
||
"total_completed": 20,
|
||
"total_failed": 2,
|
||
"success_rate_percent": 90.91,
|
||
"by_status": {
|
||
"pending": 1,
|
||
"processing": 1,
|
||
"completed": 20,
|
||
"failed": 2
|
||
}
|
||
},
|
||
"performance": {
|
||
"avg_response_time_ms": 245.67,
|
||
"max_response_time_ms": 1250.34,
|
||
"min_response_time_ms": 12.45
|
||
},
|
||
"system": {
|
||
"memory_usage_percent": 45.2,
|
||
"memory_used_gb": 7.3,
|
||
"memory_total_gb": 16.0,
|
||
"disk_usage_percent": 23.1,
|
||
"disk_used_gb": 46.8,
|
||
"disk_total_gb": 203.2
|
||
},
|
||
"recent_tasks": [
|
||
{
|
||
"task_id": "abc123-def456",
|
||
"status": "completed",
|
||
"age_seconds": 45.2,
|
||
"message": "处理完成成功"
|
||
}
|
||
]
|
||
}
|
||
}
|
||
```
|
||
|
||
**字段说明**:
|
||
- `summary.uptime_seconds`: API运行时间（秒）
|
||
- `summary.uptime_formatted`: 格式化的运行时间（如 "1h 0m 0s"）
|
||
- `summary.requests_total`: 总请求数
|
||
- `summary.requests_per_second`: 平均每秒请求数
|
||
- `summary.error_rate_percent`: 请求错误率百分比
|
||
- `summary.active_tasks`: 当前活跃任务数（pending或processing状态）
|
||
- `requests.by_method`: 按HTTP方法分组的请求统计
|
||
- `requests.by_status`: 按HTTP状态码分组的请求统计
|
||
- `requests.top_endpoints`: 请求最多的前10个端点
|
||
- `tasks.total_created`: 创建的总任务数
|
||
- `tasks.total_completed`: 完成的任务数
|
||
- `tasks.total_failed`: 失败的任务数
|
||
- `tasks.success_rate_percent`: 任务成功率百分比
|
||
- `tasks.by_status`: 按状态分组的任务统计
|
||
- `performance.avg_response_time_ms`: 平均响应时间（毫秒）
|
||
- `performance.max_response_time_ms`: 最大响应时间（毫秒）
|
||
- `performance.min_response_time_ms`: 最小响应时间（毫秒）
|
||
- `system.memory_usage_percent`: 内存使用率百分比
|
||
- `system.memory_used_gb`: 已用内存（GB）
|
||
- `system.memory_total_gb`: 总内存（GB）
|
||
- `system.disk_usage_percent`: 磁盘使用率百分比（输出目录所在磁盘）
|
||
- `system.disk_used_gb`: 已用磁盘空间（GB）
|
||
- `system.disk_total_gb`: 总磁盘空间（GB）
|
||
- `recent_tasks[]`: 最近20个任务的状态信息
|
||
|
||
---
|
||
|
||
#### 3. 重置统计信息
|
||
**端点**: `POST /stats/reset`
|
||
|
||
**描述**: 重置所有 API 统计数据（管理员功能）。
|
||
|
||
**响应示例** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "统计信息重置成功",
|
||
"data": {
|
||
"timestamp": 1705257600.123
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
#### 4. 获取配置信息
|
||
**端点**: `GET /config`
|
||
|
||
**描述**: 获取当前应用配置信息和支持的环境变量。
|
||
|
||
**响应示例** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "配置信息获取成功",
|
||
"data": {
|
||
"configuration": {
|
||
"host": "0.0.0.0",
|
||
"port": 5000,
|
||
"debug": false,
|
||
"base_dir": "/app",
|
||
"upload_folder": "/app/web_api_data/uploads",
|
||
"output_folder": "/app/web_api_data/outputs",
|
||
"max_content_length": 104857600,
|
||
"log_level": "INFO",
|
||
"log_file": "logs/gasflux_api.log",
|
||
"cors_origins": ["*"],
|
||
"task_cleanup_interval": 3600,
|
||
"max_task_age": 86400,
|
||
"threads": 8,
|
||
"connection_limit": 100,
|
||
"channel_timeout": 300
|
||
},
|
||
"environment_variables": {
|
||
"supported": [
|
||
"GASFLUX_HOST", "GASFLUX_PORT", "GASFLUX_DEBUG",
|
||
"GASFLUX_UPLOAD_FOLDER", "GASFLUX_OUTPUT_FOLDER",
|
||
"GASFLUX_MAX_CONTENT_LENGTH", "GASFLUX_LOG_LEVEL",
|
||
"GASFLUX_LOG_FILE", "GASFLUX_CORS_ORIGINS",
|
||
"GASFLUX_TASK_CLEANUP_INTERVAL", "GASFLUX_MAX_TASK_AGE",
|
||
"GASFLUX_THREADS", "GASFLUX_CONNECTION_LIMIT",
|
||
"GASFLUX_CHANNEL_TIMEOUT"
|
||
],
|
||
"current_values": {
|
||
"GASFLUX_HOST": "0.0.0.0",
|
||
"GASFLUX_PORT": "5000",
|
||
"GASFLUX_DEBUG": "false"
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### 📤 文件上传和管理
|
||
|
||
#### 5. 文件上传和处理
|
||
**端点**: `POST /upload`
|
||
|
||
**描述**: 上传数据文件并启动异步处理任务。
|
||
|
||
**请求参数** (multipart/form-data):
|
||
- `file` (必需): 数据文件 (.xlsx 或 .xls 格式)
|
||
- `config` (可选): 配置文件 (.yaml 或 .yml 格式)
|
||
|
||
**请求示例** (cURL):
|
||
```bash
|
||
curl -X POST \
|
||
-F "file=@data.xlsx" \
|
||
-F "config=@config.yaml" \
|
||
http://localhost:5000/upload
|
||
```
|
||
|
||
**请求示例** (Python):
|
||
```python
|
||
import requests
|
||
|
||
files = {'file': open('data.xlsx', 'rb')}
|
||
config = {'config': open('config.yaml', 'rb')} # 可选
|
||
|
||
response = requests.post('http://localhost:5000/upload', files={**files, **config})
|
||
result = response.json()
|
||
print(f"Task ID: {result['job_id']}")
|
||
```
|
||
|
||
**成功响应** (202):
|
||
```json
|
||
{
|
||
"code": 202,
|
||
"message": "任务已接受并加入处理队列",
|
||
"data": {
|
||
"status": "accepted",
|
||
"job_id": "abc123-def456-ghi789",
|
||
"task_status_url": "/task/abc123-def456-ghi789"
|
||
}
|
||
}
|
||
```
|
||
|
||
**错误响应示例**:
|
||
|
||
- 文件类型不支持 (400):
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "无效的数据文件类型。只允许 .xlsx 和 .xls 格式。",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
- 文件过大 (413):
|
||
```json
|
||
{
|
||
"code": 413,
|
||
"message": "文件过大。最大尺寸为 100MB。",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
- 配置文件格式错误 (400):
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "无效的配置文件类型。只允许 .yaml 和 .yml 格式。",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
### 📊 任务管理和监控
|
||
|
||
#### 6. 查询任务状态
|
||
**端点**: `GET /task/{task_id}`
|
||
|
||
**描述**: 查询异步处理任务的当前状态和进度信息。
|
||
|
||
**路径参数**:
|
||
- `task_id`: 任务 ID (UUID 格式)
|
||
|
||
**响应示例** - 处理中 (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "任务查询成功",
|
||
"data": {
|
||
"task_id": "abc123-def456-ghi789",
|
||
"status": "processing",
|
||
"message": "GasFlux 分析完成，正在生成报告...",
|
||
"updated_at": 1705257600.123,
|
||
"progress": {
|
||
"stage": "report_generation",
|
||
"completed_steps": 4,
|
||
"total_steps": 5,
|
||
"estimated_time_remaining": 45
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**响应示例** - 处理完成 (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "任务查询成功",
|
||
"data": {
|
||
"task_id": "abc123-def456-ghi789",
|
||
"status": "completed",
|
||
"message": "处理完成成功",
|
||
"updated_at": 1705257600.123,
|
||
"processing_time_seconds": 125.67,
|
||
"results": [
|
||
{
|
||
"name": "08_34_01_5m.processed_ch4_report.html",
|
||
"rel_path": "abc123-def456-ghi789/08_34_01_5m.processed_ch4_report.html",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed_ch4_report.html",
|
||
"size": 245760,
|
||
"type": "report"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_data.csv",
|
||
"rel_path": "abc123-def456-ghi789/08_34_01_5m.processed_data.csv",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed_data.csv",
|
||
"size": 153600,
|
||
"type": "data"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_config.yaml",
|
||
"rel_path": "abc123-def456-ghi789/08_34_01_5m.processed_config.yaml",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed_config.yaml",
|
||
"size": 2048,
|
||
"type": "config"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_output_vars.json",
|
||
"rel_path": "abc123-def456-ghi789/08_34_01_5m.processed_output_vars.json",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed_output_vars.json",
|
||
"size": 4096,
|
||
"type": "metadata"
|
||
}
|
||
]
|
||
}
|
||
}
|
||
```
|
||
|
||
**响应示例** - 处理失败 (200):
|
||
```json
|
||
{
|
||
"task_id": "abc123-def456-ghi789",
|
||
"status": "failed",
|
||
"message": "处理失败",
|
||
"updated_at": 1705257600.123,
|
||
"processing_time_seconds": 23.45,
|
||
"error": "处理失败: Invalid data format in column 'temperature'",
|
||
"error_details": {
|
||
"stage": "data_validation",
|
||
"error_code": "INVALID_DATA_FORMAT",
|
||
"traceback": "..."
|
||
}
|
||
}
|
||
```
|
||
|
||
**响应示例** - 任务不存在 (404):
|
||
```json
|
||
{
|
||
"code": 404,
|
||
"message": "任务未找到",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
**任务状态说明**:
|
||
- `pending`: 任务已排队，等待处理
|
||
- `processing`: 正在处理中（包含进度信息）
|
||
- `completed`: 处理完成（包含结果文件列表）
|
||
- `failed`: 处理失败（包含错误信息）
|
||
|
||
---
|
||
|
||
#### 7. 更新任务状态
|
||
**端点**: `PUT /task/{task_id}`
|
||
|
||
**描述**: 更新任务的状态、信息或优先级。
|
||
|
||
**路径参数**:
|
||
- `task_id`: 任务 ID (UUID 格式)
|
||
|
||
**请求参数** (JSON):
|
||
- `status` (可选): 新的任务状态
|
||
- `pending`: 重新排队等待处理
|
||
- `processing`: 标记为处理中
|
||
- `completed`: 标记为完成
|
||
- `failed`: 标记为失败
|
||
- `message` (可选): 状态消息或错误描述
|
||
- `priority` (可选): 任务优先级 (normal/high/low)
|
||
|
||
**请求示例** (cURL):
|
||
```bash
|
||
# 标记任务为完成
|
||
curl -X PUT http://localhost:5000/task/abc123-def456-ghi789 \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"status": "completed", "message": "手动标记为完成"}'
|
||
|
||
# 更新任务消息
|
||
curl -X PUT http://localhost:5000/task/abc123-def456-ghi789 \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"message": "更新的状态消息"}'
|
||
|
||
# 设置高优先级
|
||
curl -X PUT http://localhost:5000/task/abc123-def456-ghi789 \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"priority": "high"}'
|
||
```
|
||
|
||
**请求示例** (Python):
|
||
```python
|
||
import requests
|
||
|
||
# 标记任务为失败
|
||
response = requests.put(
|
||
'http://localhost:5000/task/abc123-def456-ghi789',
|
||
json={
|
||
'status': 'failed',
|
||
'message': '处理失败 due to invalid input data'
|
||
}
|
||
)
|
||
|
||
# 更新任务优先级
|
||
response = requests.put(
|
||
'http://localhost:5000/task/abc123-def456-ghi789',
|
||
json={'priority': 'high'}
|
||
)
|
||
```
|
||
|
||
**成功响应** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "任务更新成功",
|
||
"data": {
|
||
"task_id": "abc123-def456-ghi789",
|
||
"status": "updated",
|
||
"task_info": {
|
||
"status": "completed",
|
||
"message": "手动标记为完成",
|
||
"updated_at": 1705257600.123,
|
||
"priority": "normal"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**错误响应示例**:
|
||
|
||
- 任务不存在 (404):
|
||
```json
|
||
{
|
||
"code": 404,
|
||
"message": "任务未找到",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
- 无效状态 (400):
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "无效状态。必须是以下之一: pending, processing, completed, failed",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
- 无效请求 (400):
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "请求体必须是 JSON 格式",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
#### 8. 删除任务
|
||
**端点**: `DELETE /task/{task_id}`
|
||
|
||
**描述**: 删除任务及其所有相关的文件和数据。
|
||
|
||
**路径参数**:
|
||
- `task_id`: 任务 ID (UUID 格式)
|
||
|
||
**请求示例** (cURL):
|
||
```bash
|
||
# 删除指定任务
|
||
curl -X DELETE http://localhost:5000/task/abc123-def456-ghi789
|
||
```
|
||
|
||
**请求示例** (Python):
|
||
```python
|
||
import requests
|
||
|
||
# 删除任务
|
||
response = requests.delete('http://localhost:5000/task/abc123-def456-ghi789')
|
||
|
||
if response.status_code == 200:
|
||
result = response.json()
|
||
print(f"Task {result['task_id']} deleted")
|
||
print(f"Files deleted: {result['details']['folders_deleted']}")
|
||
print(f"Size freed: {result['details']['total_size_deleted']} bytes")
|
||
else:
|
||
print(f"Failed to delete task: {response.json()}")
|
||
```
|
||
|
||
**成功响应** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "任务及相关文件删除成功",
|
||
"data": {
|
||
"task_id": "abc123-def456-ghi789",
|
||
"status": "deleted",
|
||
"details": {
|
||
"folders_deleted": 1,
|
||
"total_size_deleted": 307200,
|
||
"task_status": "completed"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**错误响应示例**:
|
||
|
||
- 任务不存在 (404):
|
||
```json
|
||
{
|
||
"code": 404,
|
||
"message": "任务未找到",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
- 任务正在处理 (409):
|
||
```json
|
||
{
|
||
"code": 409,
|
||
"message": "无法删除当前正在处理或等待处理的任务",
|
||
"data": {
|
||
"task_status": "processing"
|
||
}
|
||
}
|
||
```
|
||
|
||
- 删除文件失败 (500):
|
||
```json
|
||
{
|
||
"code": 500,
|
||
"message": "删除任务文件失败: Permission denied",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
**注意事项**:
|
||
- 只能删除已完成或失败的任务
|
||
- 无法删除正在处理或等待处理的任务
|
||
- 删除操作会同时删除任务记录和所有相关文件
|
||
- 删除操作不可逆，请谨慎使用
|
||
|
||
---
|
||
|
||
### 📋 报告管理和查询
|
||
|
||
#### 9. 分页查询已生成报告
|
||
**端点**: `GET /reports`
|
||
|
||
**描述**: 分页查询所有已生成的处理报告，支持排序和过滤。
|
||
|
||
**查询参数**:
|
||
- `page` (可选): 页码 (默认: 1)
|
||
- `per_page` (可选): 每页报告数量 (默认: 20, 最大: 100)
|
||
- `sort_by` (可选): 排序字段 (默认: created_at)
|
||
- `created_at`: 按创建时间排序
|
||
- `task_id`: 按任务ID排序
|
||
- `file_size`: 按文件总大小排序
|
||
- `processing_time`: 按处理时间排序
|
||
- `sort_order` (可选): 排序顺序 (默认: desc)
|
||
- `asc`: 升序
|
||
- `desc`: 降序
|
||
- `status` (可选): 按任务状态过滤
|
||
- `completed`: 只显示完成的任务
|
||
- `failed`: 只显示失败的任务
|
||
- 不指定: 显示所有任务
|
||
|
||
**请求示例** (cURL):
|
||
```bash
|
||
# 获取第一页，每页20个报告，按创建时间倒序
|
||
curl "http://localhost:5000/reports?page=1&per_page=20&sort_by=created_at&sort_order=desc"
|
||
|
||
# 获取第二页，只显示完成的任务
|
||
curl "http://localhost:5000/reports?page=2&status=completed"
|
||
|
||
# 按处理时间升序排序
|
||
curl "http://localhost:5000/reports?sort_by=processing_time&sort_order=asc"
|
||
```
|
||
|
||
**请求示例** (Python):
|
||
```python
|
||
import requests
|
||
|
||
# 基本查询
|
||
response = requests.get('http://localhost:5000/reports')
|
||
reports = response.json()
|
||
|
||
# 分页查询
|
||
params = {
|
||
'page': 1,
|
||
'per_page': 10,
|
||
'sort_by': 'created_at',
|
||
'sort_order': 'desc',
|
||
'status': 'completed'
|
||
}
|
||
response = requests.get('http://localhost:5000/reports', params=params)
|
||
data = response.json()
|
||
|
||
print(f"总报告数: {data['pagination']['total_reports']}")
|
||
print(f"当前页: {data['pagination']['page']}/{data['pagination']['total_pages']}")
|
||
|
||
for report in data['reports']:
|
||
print(f"任务: {report['task_id']}")
|
||
print(f"状态: {report['status']}")
|
||
print(f"创建时间: {report['created_at']}")
|
||
print(f"文件数量: {report['file_count']}")
|
||
if report['main_report']:
|
||
print(f"主报告: {report['main_report']['download_url']}")
|
||
```
|
||
|
||
**成功响应** (200):
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "报告列表获取成功",
|
||
"data": {
|
||
"reports": [
|
||
{
|
||
"task_id": "abc123-def456-ghi789",
|
||
"report_name": "08_34_01_5m",
|
||
"status": "completed",
|
||
"created_at": 1705257600.123,
|
||
"file_count": 4,
|
||
"total_size": 307200,
|
||
"processing_time_seconds": 125.67,
|
||
"main_report": {
|
||
"name": "08_34_01_5m.processed_ch4_report.html",
|
||
"size": 245760,
|
||
"type": "report",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run/08_34_01_5m.processed_ch4_report.html"
|
||
},
|
||
"all_files": [
|
||
{
|
||
"name": "08_34_01_5m.processed_ch4_report.html",
|
||
"size": 245760,
|
||
"type": "report",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run/08_34_01_5m.processed_ch4_report.html"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_data.csv",
|
||
"size": 153600,
|
||
"type": "data",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run/08_34_01_5m.processed_data.csv"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_config.yaml",
|
||
"size": 2048,
|
||
"type": "config",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run/08_34_01_5m.processed_config.yaml"
|
||
},
|
||
{
|
||
"name": "08_34_01_5m.processed_output_vars.json",
|
||
"size": 4096,
|
||
"type": "metadata",
|
||
"download_url": "/download/abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run/08_34_01_5m.processed_output_vars.json"
|
||
}
|
||
],
|
||
"run_directory": "abc123-def456-ghi789/08_34_01_5m.processed/2026-01-14_10-33-29-961698_processing_run"
|
||
}
|
||
],
|
||
"pagination": {
|
||
"page": 1,
|
||
"per_page": 20,
|
||
"total_reports": 45,
|
||
"total_pages": 3,
|
||
"has_next": true,
|
||
"has_prev": false
|
||
},
|
||
"filters": {
|
||
"sort_by": "created_at",
|
||
"sort_order": "desc",
|
||
"status": null
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**错误响应示例**:
|
||
|
||
- 参数无效 (400):
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "Invalid parameter: per_page must be between 1 and 100",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 📁 文件下载
|
||
|
||
#### 10. 下载处理结果
|
||
**端点**: `GET /download/{filename}`
|
||
|
||
**描述**: 下载处理后的结果文件。
|
||
|
||
**路径参数**:
|
||
- `filename`: 文件的相对路径 (包含任务ID)
|
||
|
||
**请求示例** (cURL):
|
||
```bash
|
||
# 下载 HTML 报告
|
||
curl -O http://localhost:5000/download/abc123-def456-ghi789/report.html
|
||
|
||
# 下载 CSV 数据
|
||
curl -O http://localhost:5000/download/abc123-def456-ghi789/data.csv
|
||
|
||
# 使用 Python 下载
|
||
import requests
|
||
|
||
response = requests.get('http://localhost:5000/download/abc123-def456-ghi789/report.html')
|
||
with open('report.html', 'wb') as f:
|
||
f.write(response.content)
|
||
```
|
||
|
||
**状态码**:
|
||
- `200`: 成功下载文件
|
||
- `403`: 访问被拒绝 (路径遍历攻击防护)
|
||
- `404`: 文件不存在
|
||
- `400`: 路径不是文件
|
||
|
||
---
|
||
|
||
### 🌐 Web 界面
|
||
|
||
#### 11. Web 管理界面
|
||
**端点**: `GET /`
|
||
|
||
**描述**: 访问用户友好的 Web 界面，支持文件上传、任务监控和结果下载。
|
||
|
||
**响应**: HTML 页面，包含：
|
||
- 文件上传表单
|
||
- 任务状态监控面板
|
||
- 结果文件下载链接
|
||
- 系统状态信息
|
||
|
||
---
|
||
|
||
## 🔄 处理流程详解
|
||
|
||
### 完整处理流程
|
||
|
||
1. **文件上传阶段**
|
||
- 客户端验证文件类型和大小
|
||
- 上传数据文件和可选的配置文件
|
||
- 服务器进行安全检查和文件存储
|
||
|
||
2. **任务队列管理**
|
||
- 服务器为上传任务分配唯一的 UUID
|
||
- 任务进入处理队列，根据系统负载进行调度
|
||
|
||
3. **异步数据处理**
|
||
- **数据预处理**: 格式转换、数据验证、单位转换
|
||
- **配置合并**: 默认配置 + 用户配置
|
||
- **GasFlux 核心分析**:
|
||
- 背景校正算法
|
||
- 气体通量计算
|
||
- 空间插值 (克里金插值)
|
||
- 统计分析和可视化
|
||
- **结果生成**: HTML 报告、CSV 数据、配置文件备份
|
||
|
||
4. **实时状态监控**
|
||
- 客户端通过任务 ID 轮询状态
|
||
- 支持进度跟踪和预计完成时间
|
||
|
||
5. **结果获取和清理**
|
||
- 处理完成后提供下载链接
|
||
- 定期清理过期任务和文件
|
||
|
||
### 处理时间估计
|
||
|
||
- 小文件 (< 10MB): 1-3 分钟
|
||
- 中等文件 (10-50MB): 3-10 分钟
|
||
- 大文件 (> 50MB): 10-30 分钟
|
||
- 受计算复杂度、数据质量和系统负载影响
|
||
|
||
## ⚠️ 错误处理
|
||
|
||
### HTTP 状态码
|
||
|
||
| 状态码 | 说明 | 处理建议 |
|
||
|--------|------|----------|
|
||
| 200 | 请求成功 | 正常处理响应数据 |
|
||
| 202 | 请求已接受 (异步处理) | 记录任务 ID，开始状态轮询 |
|
||
| 400 | 请求参数错误 | 检查请求格式和参数 |
|
||
| 403 | 访问被拒绝 | 检查文件路径和权限 |
|
||
| 404 | 资源不存在 | 验证任务 ID 或文件路径 |
|
||
| 413 | 文件过大 | 压缩文件或联系管理员 |
|
||
| 500 | 服务器内部错误 | 检查系统状态，重试请求 |
|
||
|
||
### 常见错误响应
|
||
|
||
**文件上传错误**:
|
||
```json
|
||
{
|
||
"code": 400,
|
||
"message": "无效的数据文件类型。只允许 .xlsx 和 .xls 格式。",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
```json
|
||
{
|
||
"code": 413,
|
||
"message": "文件过大。最大尺寸为 100MB。",
|
||
"data": {}
|
||
}
|
||
```
|
||
|
||
**任务查询错误**:
|
||
```json
|
||
{
|
||
"code": 404,
|
||
"message": "任务未找到",
|
||
"data": {
|
||
"task_id": "invalid-task-id"
|
||
}
|
||
}
|
||
```
|
||
|
||
**文件下载错误**:
|
||
```json
|
||
{
|
||
"code": 404,
|
||
"message": "文件未找到",
|
||
"data": {
|
||
"filename": "task-id/file.html"
|
||
}
|
||
}
|
||
```
|
||
|
||
## 📊 性能监控和日志
|
||
|
||
### 日志记录
|
||
|
||
所有 API 请求都会记录到 `gasflux_api.log` 文件，包含：
|
||
|
||
```
|
||
2026-01-14 10:30:15,123 - INFO - [task:abc123] POST /upload - 202 - 2.34s
|
||
2026-01-14 10:30:17,456 - INFO - [task:abc123] Processing started: data.xlsx (15.2MB)
|
||
2026-01-14 10:32:22,789 - INFO - [task:abc123] Processing completed: 4 files generated
|
||
2026-01-14 10:32:25,012 - INFO - [task:abc123] GET /download/abc123/report.html - 200 - 0.45s
|
||
```
|
||
|
||
### 性能指标
|
||
|
||
通过 `/stats` 端点获取：
|
||
- 请求响应时间统计
|
||
- 任务成功率
|
||
- 系统资源使用情况
|
||
- 错误率和热点端点
|
||
|
||
## 💻 编程示例
|
||
|
||
### Python 完整示例
|
||
|
||
#### 基础用法
|
||
```python
|
||
import requests
|
||
import time
|
||
import os
|
||
from pathlib import Path
|
||
|
||
class GasFluxClient:
|
||
def __init__(self, base_url="http://localhost:5000"):
|
||
self.base_url = base_url.rstrip('/')
|
||
|
||
def check_health(self):
|
||
"""检查 API 健康状态"""
|
||
response = requests.get(f"{self.base_url}/health")
|
||
return response.json()
|
||
|
||
def upload_and_process(self, data_file, config_file=None, output_dir="./results"):
|
||
"""上传文件并处理"""
|
||
files = {'file': open(data_file, 'rb')}
|
||
if config_file and os.path.exists(config_file):
|
||
files['config'] = open(config_file, 'rb')
|
||
|
||
# 上传文件
|
||
print(f"Uploading {data_file}...")
|
||
response = requests.post(f"{self.base_url}/upload", files=files)
|
||
result = response.json()
|
||
|
||
if response.status_code != 202:
|
||
raise Exception(f"Upload failed: {result.get('error', 'Unknown error')}")
|
||
|
||
task_id = result['job_id']
|
||
print(f"任务已创建: {task_id}")
|
||
|
||
# 监控处理状态
|
||
while True:
|
||
status_response = requests.get(f"{self.base_url}/task/{task_id}")
|
||
status = status_response.json()
|
||
|
||
print(f"Status: {status['status']} - {status['message']}")
|
||
|
||
if status['status'] == 'completed':
|
||
# 下载结果文件
|
||
os.makedirs(output_dir, exist_ok=True)
|
||
for result_file in status['results']:
|
||
download_url = f"{self.base_url}{result_file['download_url']}"
|
||
output_path = Path(output_dir) / result_file['name']
|
||
|
||
print(f"Downloading {result_file['name']}...")
|
||
download_response = requests.get(download_url)
|
||
with open(output_path, 'wb') as f:
|
||
f.write(download_response.content)
|
||
|
||
print(f"Processing completed! Results saved to {output_dir}")
|
||
return status
|
||
|
||
elif status['status'] == 'failed':
|
||
error_msg = status.get('error', 'Unknown error')
|
||
raise Exception(f"Task failed: {error_msg}")
|
||
|
||
time.sleep(3) # 每3秒检查一次状态
|
||
|
||
# 使用示例
|
||
client = GasFluxClient()
|
||
try:
|
||
# 检查 API 状态
|
||
health = client.check_health()
|
||
print(f"API Status: {health['status']}")
|
||
|
||
# 处理数据
|
||
result = client.upload_and_process(
|
||
data_file="data.xlsx",
|
||
config_file="config.yaml",
|
||
output_dir="./gasflux_results"
|
||
)
|
||
print("处理完成成功!")
|
||
|
||
except Exception as e:
|
||
print(f"Error: {e}")
|
||
```
|
||
|
||
#### 异步版本 (使用 asyncio)
|
||
```python
|
||
import asyncio
|
||
import aiohttp
|
||
import aiofiles
|
||
from pathlib import Path
|
||
|
||
class AsyncGasFluxClient:
|
||
def __init__(self, base_url="http://localhost:5000"):
|
||
self.base_url = base_url.rstrip('/')
|
||
|
||
async def upload_and_process(self, data_file, config_file=None, output_dir="./results"):
|
||
async with aiohttp.ClientSession() as session:
|
||
# 准备文件上传
|
||
data = aiohttp.FormData()
|
||
data.add_field('file', open(data_file, 'rb'), filename=Path(data_file).name)
|
||
if config_file and Path(config_file).exists():
|
||
data.add_field('config', open(config_file, 'rb'), filename=Path(config_file).name)
|
||
|
||
# 上传文件
|
||
print(f"Uploading {data_file}...")
|
||
async with session.post(f"{self.base_url}/upload", data=data) as response:
|
||
result = await response.json()
|
||
if response.status != 202:
|
||
raise Exception(f"Upload failed: {result.get('error', 'Unknown error')}")
|
||
|
||
task_id = result['job_id']
|
||
print(f"任务已创建: {task_id}")
|
||
|
||
# 监控处理状态
|
||
while True:
|
||
async with session.get(f"{self.base_url}/task/{task_id}") as response:
|
||
status = await response.json()
|
||
|
||
print(f"Status: {status['status']} - {status['message']}")
|
||
|
||
if status['status'] == 'completed':
|
||
# 下载结果文件
|
||
Path(output_dir).mkdir(exist_ok=True)
|
||
for result_file in status['results']:
|
||
download_url = f"{self.base_url}{result_file['download_url']}"
|
||
output_path = Path(output_dir) / result_file['name']
|
||
|
||
print(f"Downloading {result_file['name']}...")
|
||
async with session.get(download_url) as response:
|
||
async with aiofiles.open(output_path, 'wb') as f:
|
||
await f.write(await response.read())
|
||
|
||
print(f"Processing completed! Results saved to {output_dir}")
|
||
return status
|
||
|
||
elif status['status'] == 'failed':
|
||
error_msg = status.get('error', 'Unknown error')
|
||
raise Exception(f"Task failed: {error_msg}")
|
||
|
||
await asyncio.sleep(3)
|
||
|
||
# 使用异步客户端
|
||
async def main():
|
||
client = AsyncGasFluxClient()
|
||
try:
|
||
result = await client.upload_and_process(
|
||
data_file="large_dataset.xlsx",
|
||
config_file="config.yaml",
|
||
output_dir="./async_results"
|
||
)
|
||
print("Async processing completed!")
|
||
except Exception as e:
|
||
print(f"Error: {e}")
|
||
|
||
# 运行异步示例
|
||
# asyncio.run(main())
|
||
```
|
||
|
||
### JavaScript/Node.js 示例
|
||
|
||
#### 完整实现
|
||
```javascript
|
||
const axios = require('axios');
|
||
const FormData = require('form-data');
|
||
const fs = require('fs').promises;
|
||
const path = require('path');
|
||
|
||
class GasFluxAPI {
|
||
constructor(baseURL = 'http://localhost:5000') {
|
||
this.baseURL = baseURL.replace(/\/$/, '');
|
||
this.client = axios.create({
|
||
baseURL: this.baseURL,
|
||
timeout: 30000
|
||
});
|
||
}
|
||
|
||
async checkHealth() {
|
||
try {
|
||
const response = await this.client.get('/health');
|
||
return response.data;
|
||
} catch (error) {
|
||
throw new Error(`Health check failed: ${error.message}`);
|
||
}
|
||
}
|
||
|
||
async uploadFile(dataFilePath, configFilePath = null) {
|
||
const formData = new FormData();
|
||
|
||
// 添加数据文件
|
||
if (!await fs.access(dataFilePath).then(() => true).catch(() => false)) {
|
||
throw new Error(`Data file not found: ${dataFilePath}`);
|
||
}
|
||
formData.append('file', fs.createReadStream(dataFilePath), {
|
||
filename: path.basename(dataFilePath)
|
||
});
|
||
|
||
// 添加配置文件（如果提供）
|
||
if (configFilePath) {
|
||
if (!await fs.access(configFilePath).then(() => true).catch(() => false)) {
|
||
console.warn(`Config file not found: ${configFilePath}, skipping...`);
|
||
} else {
|
||
formData.append('config', fs.createReadStream(configFilePath), {
|
||
filename: path.basename(configFilePath)
|
||
});
|
||
}
|
||
}
|
||
|
||
try {
|
||
const response = await this.client.post('/upload', formData, {
|
||
headers: formData.getHeaders(),
|
||
maxContentLength: Infinity,
|
||
maxBodyLength: Infinity
|
||
});
|
||
|
||
return response.data;
|
||
} catch (error) {
|
||
if (error.response) {
|
||
throw new Error(`Upload failed: ${error.response.data.error}`);
|
||
}
|
||
throw error;
|
||
}
|
||
}
|
||
|
||
async getTaskStatus(taskId) {
|
||
try {
|
||
const response = await this.client.get(`/task/${taskId}`);
|
||
return response.data;
|
||
} catch (error) {
|
||
if (error.response && error.response.status === 404) {
|
||
throw new Error(`任务未找到: ${taskId}`);
|
||
}
|
||
throw error;
|
||
}
|
||
}
|
||
|
||
async downloadFile(downloadUrl, outputPath) {
|
||
try {
|
||
const response = await this.client.get(downloadUrl, {
|
||
responseType: 'stream'
|
||
});
|
||
|
||
const writer = fs.createWriteStream(outputPath);
|
||
response.data.pipe(writer);
|
||
|
||
return new Promise((resolve, reject) => {
|
||
writer.on('finish', resolve);
|
||
writer.on('error', reject);
|
||
});
|
||
} catch (error) {
|
||
throw new Error(`Download failed: ${error.message}`);
|
||
}
|
||
}
|
||
|
||
async processData(dataFilePath, configFilePath = null, outputDir = './results', pollInterval = 3000) {
|
||
try {
|
||
// 1. 检查 API 健康状态
|
||
console.log('Checking API health...');
|
||
const health = await this.checkHealth();
|
||
console.log(`API Status: ${health.status}`);
|
||
|
||
// 2. 上传文件
|
||
console.log(`Uploading ${dataFilePath}...`);
|
||
const uploadResult = await this.uploadFile(dataFilePath, configFilePath);
|
||
const taskId = uploadResult.job_id;
|
||
console.log(`任务已创建: ${taskId}`);
|
||
|
||
// 3. 监控处理状态
|
||
console.log('Monitoring processing status...');
|
||
while (true) {
|
||
const status = await this.getTaskStatus(taskId);
|
||
|
||
console.log(`[${new Date().toISOString()}] Status: ${status.status} - ${status.message}`);
|
||
|
||
if (status.status === 'completed') {
|
||
console.log('处理完成成功!');
|
||
|
||
// 4. 下载结果文件
|
||
console.log('Downloading result files...');
|
||
await fs.mkdir(outputDir, { recursive: true });
|
||
|
||
for (const resultFile of status.results) {
|
||
const downloadUrl = `${this.baseURL}${resultFile.download_url}`;
|
||
const outputPath = path.join(outputDir, resultFile.name);
|
||
|
||
console.log(`Downloading ${resultFile.name}...`);
|
||
await this.downloadFile(downloadUrl, outputPath);
|
||
console.log(`Saved to ${outputPath}`);
|
||
}
|
||
|
||
return {
|
||
taskId,
|
||
status: status.status,
|
||
results: status.results,
|
||
outputDir
|
||
};
|
||
|
||
} else if (status.status === 'failed') {
|
||
const errorMsg = status.error || 'Unknown error';
|
||
throw new Error(`处理失败: ${errorMsg}`);
|
||
}
|
||
|
||
// 等待后重试
|
||
await new Promise(resolve => setTimeout(resolve, pollInterval));
|
||
}
|
||
|
||
} catch (error) {
|
||
console.error(`Error in processData: ${error.message}`);
|
||
throw error;
|
||
}
|
||
}
|
||
}
|
||
|
||
// 使用示例
|
||
async function main() {
|
||
const api = new GasFluxAPI();
|
||
|
||
try {
|
||
const result = await api.processData(
|
||
'data.xlsx',
|
||
'config.yaml', // 可选
|
||
'./gasflux_results',
|
||
5000 // 5秒检查一次状态
|
||
);
|
||
|
||
console.log('All done!', result);
|
||
|
||
} catch (error) {
|
||
console.error('处理失败:', error.message);
|
||
process.exit(1);
|
||
}
|
||
}
|
||
|
||
// 如果直接运行此文件
|
||
if (require.main === module) {
|
||
main();
|
||
}
|
||
|
||
module.exports = GasFluxAPI;
|
||
```
|
||
|
||
### cURL 命令示例
|
||
|
||
#### 基本上传和监控
|
||
```bash
|
||
#!/bin/bash
|
||
|
||
# API 基础 URL
|
||
API_URL="http://localhost:5000"
|
||
|
||
# 检查健康状态
|
||
echo "Checking API health..."
|
||
curl -s "${API_URL}/health" | jq '.'
|
||
|
||
# 上传文件
|
||
echo "Uploading data file..."
|
||
UPLOAD_RESPONSE=$(curl -s -X POST \
|
||
-F "file=@data.xlsx" \
|
||
-F "config=@config.yaml" \
|
||
"${API_URL}/upload")
|
||
|
||
echo "Upload response: $UPLOAD_RESPONSE"
|
||
|
||
# 提取任务 ID
|
||
TASK_ID=$(echo "$UPLOAD_RESPONSE" | jq -r '.job_id')
|
||
|
||
if [ "$TASK_ID" = "null" ] || [ -z "$TASK_ID" ]; then
|
||
echo "Upload failed!"
|
||
exit 1
|
||
fi
|
||
|
||
echo "Task ID: $TASK_ID"
|
||
|
||
# 监控任务状态
|
||
echo "Monitoring task status..."
|
||
while true; do
|
||
STATUS_RESPONSE=$(curl -s "${API_URL}/task/${TASK_ID}")
|
||
STATUS=$(echo "$STATUS_RESPONSE" | jq -r '.status')
|
||
MESSAGE=$(echo "$STATUS_RESPONSE" | jq -r '.message')
|
||
|
||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Status: $STATUS - $MESSAGE"
|
||
|
||
if [ "$STATUS" = "completed" ]; then
|
||
echo "Processing completed!"
|
||
|
||
# 下载结果文件
|
||
echo "Downloading result files..."
|
||
mkdir -p results
|
||
|
||
echo "$STATUS_RESPONSE" | jq -r '.results[].download_url' | while read -r download_url; do
|
||
filename=$(basename "$download_url")
|
||
echo "Downloading $filename..."
|
||
curl -s -o "results/$filename" "${API_URL}${download_url}"
|
||
done
|
||
|
||
echo "All files downloaded to ./results/"
|
||
break
|
||
|
||
elif [ "$STATUS" = "failed" ]; then
|
||
ERROR=$(echo "$STATUS_RESPONSE" | jq -r '.error // "Unknown error"')
|
||
echo "Task failed: $ERROR"
|
||
exit 1
|
||
fi
|
||
|
||
sleep 3
|
||
done
|
||
```
|
||
|
||
## 🔧 故障排除指南
|
||
|
||
### 常见问题和解决方案
|
||
|
||
#### 1. 连接问题
|
||
**问题**: 无法连接到 API 服务器
|
||
```bash
|
||
curl: (7) 连接被拒绝，无法连接到 localhost 端口 5000
|
||
```
|
||
|
||
**解决方案**:
|
||
- 检查服务器是否正在运行：`ps aux | grep gasflux`
|
||
- 验证端口配置：检查环境变量 `GASFLUX_PORT`
|
||
- 确认防火墙设置：`sudo ufw status` 或 `sudo firewall-cmd --list-all`
|
||
|
||
#### 2. 文件上传失败
|
||
|
||
**问题**: 文件上传被拒绝
|
||
```json
|
||
{"error": "无效的数据文件类型。只允许 .xlsx 和 .xls 格式。"}
|
||
```
|
||
|
||
**解决方案**:
|
||
- 检查文件扩展名（必须是 .xlsx 或 .xls）
|
||
- 验证文件不是空的
|
||
- 确保文件大小不超过 100MB
|
||
|
||
**问题**: 文件过大
|
||
```json
|
||
{"error": "文件过大。最大尺寸为 100MB。"}
|
||
```
|
||
|
||
**解决方案**:
|
||
- 压缩数据文件
|
||
- 分割成多个较小的文件
|
||
- 联系管理员增加文件大小限制
|
||
|
||
#### 3. 任务处理问题
|
||
|
||
**问题**: 任务长时间处于 pending 状态
|
||
```json
|
||
{"status": "pending", "message": "Task queued for processing"}
|
||
```
|
||
|
||
**解决方案**:
|
||
- 检查系统负载：`GET /stats`
|
||
- 查看服务器资源使用情况
|
||
- 等待队列处理或联系管理员
|
||
|
||
**问题**: 处理失败
|
||
```json
|
||
{
|
||
"status": "failed",
|
||
"error": "处理失败: Invalid data format in column 'temperature'"
|
||
}
|
||
```
|
||
|
||
**解决方案**:
|
||
- 检查输入数据格式和列名
|
||
- 验证数据范围（温度、压力等）
|
||
- 查看详细的错误信息和日志
|
||
|
||
#### 4. 文件下载问题
|
||
|
||
**问题**: 下载失败
|
||
```json
|
||
{"error": "File not found or access denied"}
|
||
```
|
||
|
||
**解决方案**:
|
||
- 确认任务已完成（status: "completed"）
|
||
- 检查下载 URL 格式
|
||
- 验证文件路径是否存在
|
||
|
||
#### 5. 服务器性能问题
|
||
|
||
**问题**: 响应缓慢或超时
|
||
|
||
**解决方案**:
|
||
- 检查系统资源：`GET /stats`
|
||
- 查看并发任务数量
|
||
- 监控内存和 CPU 使用率
|
||
- 考虑增加服务器资源或优化配置
|
||
|
||
### 调试技巧
|
||
|
||
#### 启用详细日志
|
||
```bash
|
||
# 设置环境变量启用 DEBUG 模式
|
||
export GASFLUX_LOG_LEVEL=DEBUG
|
||
export GASFLUX_DEBUG=true
|
||
|
||
# 重启服务器
|
||
python server_waitress.py
|
||
```
|
||
|
||
#### 查看实时日志
|
||
```bash
|
||
# 监控日志文件
|
||
tail -f logs/gasflux_api.log
|
||
|
||
# 过滤特定任务的日志
|
||
tail -f logs/gasflux_api.log | grep "task:abc123"
|
||
```
|
||
|
||
#### 使用健康检查进行诊断
|
||
```bash
|
||
# 基本健康检查
|
||
curl -s http://localhost:5000/health | jq '.'
|
||
|
||
# 详细统计信息
|
||
curl -s http://localhost:5000/stats | jq '.'
|
||
|
||
# 配置信息
|
||
curl -s http://localhost:5000/config | jq '.'
|
||
```
|
||
|
||
## 🛡️ 安全考虑
|
||
|
||
### 数据保护
|
||
- **文件类型验证**: 只接受指定的文件类型 (.xlsx, .xls, .yaml, .yml)
|
||
- **路径遍历保护**: 防止通过 `../` 等路径访问敏感文件
|
||
- **文件大小限制**: 防止拒绝服务攻击
|
||
|
||
### 访问控制
|
||
- **无认证设计**: 适用于内部网络或受控环境
|
||
- **IP 白名单**: 可通过反向代理实现
|
||
- **HTTPS 推荐**: 在生产环境中使用 HTTPS
|
||
|
||
### 数据清理
|
||
- **自动清理**: 过期的任务和文件会被自动删除
|
||
- **配置选项**: 可通过环境变量调整清理间隔和过期时间
|
||
|
||
## 📈 最佳实践
|
||
|
||
### 客户端实现
|
||
|
||
#### 1. 错误处理
|
||
```python
|
||
def safe_api_call(url, max_retries=3, backoff_factor=2):
|
||
for attempt in range(max_retries):
|
||
try:
|
||
response = requests.get(url, timeout=30)
|
||
response.raise_for_status()
|
||
return response.json()
|
||
except requests.exceptions.RequestException as e:
|
||
if attempt == max_retries - 1:
|
||
raise e
|
||
wait_time = backoff_factor ** attempt
|
||
print(f"Request failed, retrying in {wait_time}s...")
|
||
time.sleep(wait_time)
|
||
```
|
||
|
||
#### 2. 状态轮询优化
|
||
```python
|
||
def monitor_task_efficiently(task_id, max_wait_time=3600):
|
||
start_time = time.time()
|
||
check_interval = 2 # 初始检查间隔
|
||
|
||
while time.time() - start_time < max_wait_time:
|
||
status = get_task_status(task_id)
|
||
|
||
if status['status'] in ['completed', 'failed']:
|
||
return status
|
||
|
||
# 根据任务阶段调整检查间隔
|
||
if 'progress' in status:
|
||
stage = status['progress'].get('stage', '')
|
||
if 'data_processing' in stage:
|
||
check_interval = 5 # 数据处理阶段检查频率降低
|
||
elif 'report_generation' in stage:
|
||
check_interval = 10 # 报告生成阶段进一步降低
|
||
|
||
time.sleep(check_interval)
|
||
|
||
raise TimeoutError(f"Task monitoring timed out after {max_wait_time}s")
|
||
```
|
||
|
||
#### 3. 大文件处理
|
||
```python
|
||
def upload_large_file(file_path, chunk_size=1024*1024): # 1MB chunks
|
||
file_size = os.path.getsize(file_path)
|
||
|
||
# 对于大文件，考虑压缩或分块上传
|
||
if file_size > 50*1024*1024: # 50MB
|
||
print(f"Large file detected ({file_size/1024/1024:.1f}MB)")
|
||
print("Consider compressing the data or splitting into smaller files")
|
||
|
||
# 标准上传
|
||
with open(file_path, 'rb') as f:
|
||
files = {'file': f}
|
||
response = requests.post('http://localhost:5000/upload', files=files)
|
||
return response.json()
|
||
```
|
||
|
||
### 服务器部署
|
||
|
||
#### 生产环境配置
|
||
```bash
|
||
# 环境变量配置
|
||
export GASFLUX_HOST=0.0.0.0
|
||
export GASFLUX_PORT=5000
|
||
export GASFLUX_LOG_LEVEL=INFO
|
||
export GASFLUX_MAX_CONTENT_LENGTH=104857600 # 100MB
|
||
|
||
# 使用进程管理器
|
||
# systemd 服务示例
|
||
cat > /etc/systemd/system/gasflux.service << EOF
|
||
[Unit]
|
||
Description=GasFlux Web API
|
||
After=network.target
|
||
|
||
[Service]
|
||
User=gasflux
|
||
Group=gasflux
|
||
WorkingDirectory=/opt/gasflux
|
||
ExecStart=/opt/gasflux/venv/bin/python server_waitress.py
|
||
Restart=always
|
||
RestartSec=5
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
EOF
|
||
|
||
# 启用和启动服务
|
||
sudo systemctl enable gasflux
|
||
sudo systemctl start gasflux
|
||
```
|
||
|
||
#### 监控和告警
|
||
```bash
|
||
#!/bin/bash
|
||
# 健康检查脚本
|
||
API_URL="http://localhost:5000"
|
||
|
||
# 检查健康状态
|
||
if ! curl -f -s "${API_URL}/health" > /dev/null; then
|
||
echo "API is unhealthy, sending alert..."
|
||
# 发送告警邮件、Slack 通知等
|
||
fi
|
||
|
||
# 检查队列长度
|
||
STATS=$(curl -s "${API_URL}/stats")
|
||
ACTIVE_TASKS=$(echo "$STATS" | jq '.summary.active_tasks')
|
||
|
||
if [ "$ACTIVE_TASKS" -gt 10 ]; then
|
||
echo "High task queue detected: $ACTIVE_TASKS active tasks"
|
||
# 发送告警
|
||
fi
|
||
```
|
||
|
||
## 📚 附录
|
||
|
||
### 支持的环境变量
|
||
|
||
| 变量名 | 默认值 | 描述 |
|
||
|--------|--------|------|
|
||
| `GASFLUX_HOST` | `0.0.0.0` | 服务器监听地址 |
|
||
| `GASFLUX_PORT` | `5000` | 服务器监听端口 |
|
||
| `GASFLUX_DEBUG` | `false` | 调试模式开关 |
|
||
| `GASFLUX_UPLOAD_FOLDER` | `web_api_data/uploads` | 上传文件存储目录 |
|
||
| `GASFLUX_OUTPUT_FOLDER` | `web_api_data/outputs` | 输出文件存储目录 |
|
||
| `GASFLUX_MAX_CONTENT_LENGTH` | `104857600` | 最大文件大小 (字节) |
|
||
| `GASFLUX_LOG_LEVEL` | `INFO` | 日志级别 |
|
||
| `GASFLUX_LOG_FILE` | `logs/gasflux_api.log` | 日志文件路径 |
|
||
| `GASFLUX_CORS_ORIGINS` | `["*"]` | 允许的 CORS 源 |
|
||
| `GASFLUX_TASK_CLEANUP_INTERVAL` | `3600` | 任务清理间隔 (秒) |
|
||
| `GASFLUX_MAX_TASK_AGE` | `86400` | 任务最大年龄 (秒) |
|
||
| `GASFLUX_THREADS` | `8` | Waitress 线程数 |
|
||
| `GASFLUX_CONNECTION_LIMIT` | `100` | 最大连接数 |
|
||
| `GASFLUX_CHANNEL_TIMEOUT` | `300` | 通道超时 (秒) |
|
||
|
||
### API 响应时间基准
|
||
|
||
- `/health`: < 100ms
|
||
- `/stats`: < 500ms
|
||
- `/config`: < 200ms
|
||
- `/upload`: < 2s (文件处理时间)
|
||
- `/task/{id}`: < 300ms
|
||
- `/download/{file}`: 根据文件大小 (通常 < 5s)
|
||
|
||
### 文件格式规范
|
||
|
||
#### 数据文件要求
|
||
- **格式**: Excel (.xlsx 或 .xls)
|
||
- **必需列**: latitude, longitude, height_ato, windspeed, winddir, temperature, pressure
|
||
- **可选列**: ch4, co2, c2h6 等气体浓度
|
||
- **数据类型**: 数值型 (float/int)
|
||
- **缺失值**: NaN 或空值
|
||
|
||
#### 配置文件格式
|
||
```yaml
|
||
output_dir: ~/gasflux_reports
|
||
|
||
required_cols:
|
||
latitude: [-90, 90]
|
||
longitude: [-180, 180]
|
||
height_ato: [-200, 500]
|
||
windspeed: [0, 50]
|
||
winddir: [0, 360]
|
||
temperature: [-50, 60]
|
||
pressure: [900, 1100]
|
||
|
||
gases:
|
||
ch4: [1.5, 500]
|
||
co2: [300, 5000]
|
||
c2h6: [-0.5, 10]
|
||
|
||
strategies:
|
||
background: "algorithmic"
|
||
sensor: "insitu"
|
||
spatial: "curtain"
|
||
interpolation: "kriging"
|
||
```
|
||
|
||
---
|
||
|
||
*最后更新: 2026年1月14日*
|
||
|
||
*GasFlux Web API 版本: 1.0.0*
|
||
*文档维护: API 开发团队* |