背景:系统此前没有操作审计。task_logs 的 task_id 是 NOT NULL 外键,只能挂在 任务上,且全项目仅 4 处写入点 —— 登录、导出、产品增删改、收编完全不留痕。 需求方整理的问题清单里「无审计日志查看页」正源于此:不是没有页面,是没数据。 设计参考 MOM(KCGL) 的 audit_logs / audit_listener,但按 Track 栈做了取舍: 1) 写入时机:MOM 用 SQLAlchemy event listener + 同事务写入,优点是零侵入, 缺点是**业务回滚时审计一起消失**,而失败/被拒的操作(越权尝试、参数错误) 恰恰最需要留痕。Track 改为响应生成后用**独立 session** 写入: - 业务回滚不影响审计(已验证 422/401 失败操作同样落库) - 审计写入失败也不影响业务(全包裹 try/except) - 代价:非原子提交,响应后进程立即被 kill 可能丢一条(已注释说明取舍) 2) 采集方式:中间件自动采集写操作 + 导出/下载/打印这类「读但敏感」的 GET。 路径段推导 module/action/target_id。不做手写埋点,因为手写必然漏 —— task_logs 只有 4 处写入点就是前车之鉴。 3) 增量价值:新增 request_id 字段,与 core/logging.py 的结构化日志打通, 凭一个 ID 就能从审计记录直接跳到那一次接口日志。MOM 无此字段。 4) 敏感信息:details 经 sanitize_details 递归剔除 password/token/secret 等键; 中间件不读请求体,登录明文密码不会落库(已断言表内无密码痕迹)。 配套改动: - core/roles.py:角色常量与 is_admin 收敛为单一事实来源。此前同一份 「管理员角色」规则散在 task_service、products.py 内联判断和前端 constants/task.ts 三处,已因此发生过「移动端漏判 SUPERVISOR 误挡主管」。 task_service 改为从 core.roles 导入同名常量,保持既有引用可用。 - core/deps.py:抽出 require_roles/require_admin 可复用依赖,替代内联判断。 - main.py:500 响应显式补 X-Request-ID 头 —— 该响应由 ServerErrorMiddleware 生成,位于 RequestContextMiddleware 外层,中间件没机会写头。 - auth.py:登录校验前把「尝试的账号」写入 request.state,使登录事件 (含失败登录)可归属到人,可用于追踪暴力破解。 验证:本地起 PostgreSQL 17 + 迁移后跑端到端测试,32/32 通过 (TestClient 每个请求新建事件循环,与模块级 asyncpg 连接池冲突会报 "got Future attached to a different loop",故改用 httpx.AsyncClient + ASGITransport 单循环;生产 uvicorn 单循环无此问题)。
201 lines
6.4 KiB
Python
201 lines
6.4 KiB
Python
"""审计服务 — 写入与检索
|
||
|
||
写入方案的取舍(与 MOM/KCGL 不同,理由如下)
|
||
--------------------------------------------------
|
||
MOM 用 SQLAlchemy event listener + **同事务**写入:优点是全自动、业务代码零改动;
|
||
缺点是业务事务回滚时审计记录一起被回滚掉 —— 而失败/被拒的操作恰恰是最需要
|
||
留痕的(比如越权尝试、参数错误导致的 4xx)。
|
||
|
||
Track 改为:响应生成后,用**独立 session** 写入审计。
|
||
- 业务回滚不影响审计,失败操作照样留痕
|
||
- 审计写入失败也不影响业务(全包裹 try/except,仅记日志)
|
||
- 代价:审计与业务不是原子提交,极端情况(响应后进程立即被 kill)可能丢一条。
|
||
对内部系统的操作审计,这个取舍划算。
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import logging
|
||
import uuid
|
||
from datetime import datetime
|
||
|
||
from sqlalchemy import func, select
|
||
from sqlalchemy.ext.asyncio import AsyncSession
|
||
|
||
from app.core.database import AsyncSessionLocal
|
||
from app.models.audit_log import AuditLog
|
||
|
||
logger = logging.getLogger("track.audit")
|
||
|
||
# 绝不落库的敏感字段名(命中即替换为 ***)
|
||
# 登录请求体含明文密码,一旦进审计表就成了长期泄露面
|
||
_SENSITIVE_KEYS = frozenset(
|
||
{"password", "passwd", "pwd", "token", "access_token", "refresh_token",
|
||
"secret", "api_key", "authorization", "password_hash"}
|
||
)
|
||
|
||
# 模块 / 动作 的中文标签(前端下拉与列表展示用)
|
||
MODULE_LABELS: dict[str, str] = {
|
||
"auth": "认证登录",
|
||
"product": "产品管理",
|
||
"task": "任务流转",
|
||
"order": "订单管理",
|
||
"record": "任务记录",
|
||
"print": "标签打印",
|
||
"material": "物料",
|
||
"user": "用户",
|
||
"notification": "消息通知",
|
||
"upload": "文件上传",
|
||
"dashboard": "看板统计",
|
||
"analytics": "效能分析",
|
||
"screen": "数据大屏",
|
||
"holiday": "节假日配置",
|
||
"app": "App版本",
|
||
"external": "外部系统对接",
|
||
"audit": "审计日志",
|
||
"other": "其它",
|
||
}
|
||
|
||
ACTION_LABELS: dict[str, str] = {
|
||
"create": "新增",
|
||
"update": "修改",
|
||
"delete": "删除",
|
||
"read": "查询",
|
||
"export": "导出",
|
||
"login": "登录",
|
||
"logout": "登出",
|
||
"refresh": "刷新令牌",
|
||
"print": "打印",
|
||
"upload": "上传",
|
||
"finalize": "收口",
|
||
"receive": "接收",
|
||
"transfer": "转交",
|
||
"reject": "驳回",
|
||
"recall": "撤回",
|
||
"spawn": "派发",
|
||
"end": "结束分支",
|
||
"complete": "完结",
|
||
}
|
||
|
||
|
||
def sanitize_details(details: dict | None) -> dict | None:
|
||
"""递归剔除敏感字段,避免密码/令牌落库"""
|
||
if not details:
|
||
return details
|
||
|
||
def _clean(value):
|
||
if isinstance(value, dict):
|
||
return {
|
||
k: ("***" if str(k).lower() in _SENSITIVE_KEYS else _clean(v))
|
||
for k, v in value.items()
|
||
}
|
||
if isinstance(value, list):
|
||
return [_clean(v) for v in value]
|
||
return value
|
||
|
||
return _clean(details)
|
||
|
||
|
||
async def record_audit(
|
||
*,
|
||
action: str,
|
||
module: str,
|
||
user_id: str | None = None,
|
||
display_name: str | None = None,
|
||
role: str | None = None,
|
||
target_type: str | None = None,
|
||
target_id: str | None = None,
|
||
target_name: str | None = None,
|
||
details: dict | None = None,
|
||
ip_address: str | None = None,
|
||
user_agent: str | None = None,
|
||
method: str | None = None,
|
||
url: str | None = None,
|
||
status_code: int | None = None,
|
||
error_message: str | None = None,
|
||
request_id: str | None = None,
|
||
) -> None:
|
||
"""写入一条审计记录。**绝不抛异常**:审计失败不能影响业务。"""
|
||
try:
|
||
async with AsyncSessionLocal() as session:
|
||
session.add(
|
||
AuditLog(
|
||
id=uuid.uuid4(),
|
||
user_id=user_id,
|
||
display_name=display_name,
|
||
role=role,
|
||
action=action,
|
||
module=module,
|
||
target_type=target_type,
|
||
target_id=str(target_id) if target_id is not None else None,
|
||
target_name=target_name,
|
||
details=sanitize_details(details),
|
||
ip_address=ip_address,
|
||
user_agent=user_agent[:500] if user_agent else None,
|
||
method=method,
|
||
url=url[:500] if url else None,
|
||
status_code=status_code,
|
||
error_message=error_message,
|
||
request_id=request_id,
|
||
)
|
||
)
|
||
await session.commit()
|
||
except Exception:
|
||
# 用 exception 级别但吞掉异常:保证调用方业务流程不受影响
|
||
logger.exception(
|
||
"审计写入失败(已忽略,不影响业务)",
|
||
extra={"extra_fields": {"action": action, "module": module, "url": url}},
|
||
)
|
||
|
||
|
||
async def list_audit_logs(
|
||
db: AsyncSession,
|
||
*,
|
||
user_id: str | None = None,
|
||
module: str | None = None,
|
||
action: str | None = None,
|
||
target_id: str | None = None,
|
||
request_id: str | None = None,
|
||
status_code: int | None = None,
|
||
start: datetime | None = None,
|
||
end: datetime | None = None,
|
||
skip: int = 0,
|
||
limit: int = 50,
|
||
) -> tuple[list[AuditLog], int]:
|
||
"""审计日志检索(按时间倒序)。返回 (当前页, 真实总数)。
|
||
|
||
真实总数走独立 COUNT —— 前端分页器依赖它,不能用 len(当前页)。
|
||
"""
|
||
filters = []
|
||
if user_id:
|
||
filters.append(AuditLog.user_id.ilike(f"%{user_id}%"))
|
||
if module:
|
||
filters.append(AuditLog.module == module)
|
||
if action:
|
||
filters.append(AuditLog.action == action)
|
||
if target_id:
|
||
filters.append(AuditLog.target_id == target_id)
|
||
if request_id:
|
||
filters.append(AuditLog.request_id == request_id)
|
||
if status_code is not None:
|
||
filters.append(AuditLog.status_code == status_code)
|
||
if start:
|
||
filters.append(AuditLog.created_at >= start)
|
||
if end:
|
||
filters.append(AuditLog.created_at <= end)
|
||
|
||
total = await db.scalar(
|
||
select(func.count()).select_from(AuditLog).where(*filters)
|
||
) or 0
|
||
|
||
rows = (
|
||
await db.execute(
|
||
select(AuditLog)
|
||
.where(*filters)
|
||
.order_by(AuditLog.created_at.desc())
|
||
.offset(skip)
|
||
.limit(limit)
|
||
)
|
||
).scalars().all()
|
||
|
||
return list(rows), total
|