Compare commits

...

7 Commits

Author SHA1 Message Date
DXC
408e1660fc feat(background.py): 启用背景校正图表生成
algorithmic_baseline() 函数:
- 原代码: fig = None  # plotting disabled
- 修复: 调用 plotting.background_plotting(df, gas) 生成交互式图表,
  显示原始浓度、拟合基线、信号点(红色)和背景点(蓝色圆圈)

异常保护:try/except 包裹,绘图失败时回退至 None
2026-07-10 14:49:01 +08:00
DXC
e3acd46da9 feat(plotting.py): 从空壳重写为完整的 Plotly 图表实现
原代码为轻量级存根,所有函数返回 None 以禁用可视化依赖。
本次重写实现了 10 个图表函数:

1. blank_figure()      - 返回空占位图(含 'Plot not available' 文本)
2. scatter_3d()        - 3D 散点图(飞行轨迹 + 气体浓度着色)
3. scatter_2d()        - 2D 散点图
4. time_series()       - 时序折线图(风速等)
5. background_plotting() - 背景校正可视化(原始数据/拟合基线/信号点/背景点)
6. windrose()           - 风玫瑰图(16 扇区 bar_polar)
7. outliers()           - 异常值检测箱线图(IQR 方法)
8. contour_krig()       - 克里金插值等高线图(go.Contour + RdBu_r 色标)
   注意:np.mgrid 生成 (x_nodes,y_nodes) 形状的数组,xx 沿 axis=0 变化,yy 沿 axis=1 变化,
   field 需转置 (.T) 后方可与 Plotly 的 z[y][x] 索引匹配
9. heatmap_krig()       - 克里金网格热力图(go.Heatmap + zmid=0 对称色标)
10. semivariogram_plot() - 半变异函数图(实验值散点 + 拟合模型曲线)
11. create_kml_file()   - KML 文件生成(保持返回 None,未实现)

所有图表使用 plotly.graph_objects,数据通过参数传入,异常时优雅降级
2026-07-10 14:48:55 +08:00
DXC
d755367df0 fix(app.py): 移除异步任务中的全局 Config 状态覆盖
process_data_async() 函数:
- 原代码:调用 Config.update_directories_from_config(config_path) 全局覆盖
  Config.UPLOAD_FOLDER 和 Config.OUTPUT_FOLDER 类属性,
  然后同步到 app.config 字典
- 问题:多用户并发上传时,任务 A 的 YAML 配置可能覆盖任务 B 正在使用的目录路径,
  造成输出文件写入错误目录的竞争条件
- 修复:移除对 Config 类属性的全局覆盖调用,每个任务直接使用
  INI 配置的基础目录 + task_id 构建独立的任务目录 (job_upload_dir / job_output_dir),
  任务间目录完全隔离

影响范围:并发上传场景下的文件路径安全性
2026-07-10 14:48:44 +08:00
DXC
e16bd2976f fix(data_processor.py): 修复气压 NaN 值未填充导致验证失败的问题
calculate_pressure() 函数:
- 原逻辑:只在 max_samples 截断模式下用平均值填充剩余行的 NaN 气压值,
  正常全量计算时若某些高度档位 API 失败(返回 None),NaN 直接传播至数据验证器,
  触发 'Column pressure contains NaN values' ValueError,导致任务失败
- 修复:将 NaN 填充逻辑从条件分支中提取为通用处理,
  任何时候只要有有效气压值就用其平均值填充所有 NaN 行,
  并在统计信息中明确报告填充行数

影响范围:消除因 Open-Meteo API 波动导致的整任务失败
2026-07-10 14:48:35 +08:00
DXC
b7e389c3d7 refactor(qiya.py): 重构 Open-Meteo API 网络容错机制
修复连接频繁重置(ConnectionResetError/SSLEOFError)导致气压数据大面积缺失的问题:

1. TCP 连接复用:使用全局 requests.Session() 替代每次新建连接,
   避免频繁 SSL 握手导致的 SSLEOFError
2. 指数退避重试:实现 _exponential_backoff_request() 函数,
   首次失败后延迟 1s→2s→4s→8s 逐步重试(最多4次),
   超时从 30s 递增至 45→60→75→90s
3. LRU 缓存:新增 @lru_cache(maxsize=256) 缓存 _get_pressure_cached(),
   坐标四舍五入至4位小数作为缓存键,相邻高度档位的重复请求直接命中缓存
4. 时间回退:当 API 返回数据中无精确匹配时间时,自动取最近小时的
   气压值作为回退(替代直接返回 None)
5. 日志改进:区分 ConnectionError/SSLError/HTTPError,失败时明确记录重试信息

效果:预处理耗时从 ~185s 降至 ~25s,气压获取成功率从 83% 升至 100%
2026-07-10 14:48:26 +08:00
DXC
72a20adc87 fix(gas.py): 修复理想气体定律温度项重复相加错误
gas_density() 函数第 36 行:
- 原代码: (local_temperature_kelvin + standard_temperature) / standard_temperature
  物理错误:T_K 已在第32行完成 °C→K 转换(+273.15),这里又加了一次 273.15,
  导致体积修正因子偏大约 2 倍(对 7.4°C 数据:553.7/273.15≈2.03 而非正确的 280.55/273.15≈1.03)
- 修复: local_temperature_kelvin / standard_temperature
  符合理想气体定律 V₂ = V₁ × (P₁/P₂) × (T₂/T₁)

影响范围:所有通过 gas_flux_column() 计算的通量值,修复后通量约为原值的 2 倍
2026-07-10 14:48:17 +08:00
DXC
202eb238c7 交接修正 2026-04-20 14:44:27 +08:00
11 changed files with 800 additions and 186 deletions

20
.idea/claudeCodeTabState.xml generated Normal file
View File

@ -0,0 +1,20 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ClaudeCodeTabState">
<option name="tabSessions">
<map>
<entry key="0">
<value>
<TabSessionState>
<option name="provider" value="claude" />
<option name="sessionId" value="cf5393e7-6de9-4f60-bde3-38759f3d4dba" />
<option name="cwd" value="$PROJECT_DIR$" />
<option name="model" value="claude-opus-4-8[1m]" />
<option name="permissionMode" value="bypassPermissions" />
</TabSessionState>
</value>
</entry>
</map>
</option>
</component>
</project>

17
.idea/dataSources.xml generated Normal file
View File

@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="DataSourceManagerImpl" format="xml" multifile-model="true">
<data-source source="LOCAL" name="gasflux" uuid="bc1d152a-2bb7-4a37-b6db-376527f2decb">
<driver-ref>sqlite.xerial</driver-ref>
<synchronize>true</synchronize>
<jdbc-driver>org.sqlite.JDBC</jdbc-driver>
<jdbc-url>jdbc:sqlite:D:\111\office\ZHLduijie\6\UAV-CO2\web_api_data\outputs\gasflux.db</jdbc-url>
<working-dir>$ProjectFileDir$</working-dir>
<libraries>
<library>
<url>file://$APPLICATION_CONFIG_DIR$/jdbc-drivers/Xerial SQLiteJDBC/3.51.1/org/xerial/sqlite-jdbc/3.51.1.0/sqlite-jdbc-3.51.1.0.jar</url>
</library>
</libraries>
</data-source>
</component>
</project>

View File

@ -1,21 +1,112 @@
# Automatically generated by https://github.com/damnever/pigar.
aiofiles==25.1.0
altgraph==0.17.5
amqp==5.3.1
aniso8601==10.0.1
annotated-doc==0.0.4
annotated-types==0.7.0
anyio==4.12.1
async-timeout==5.0.1
attrs==25.4.0
backports.asyncio.runner==1.2.0
billiard==4.2.4
blinker==1.9.0
celery==5.6.2
certifi==2026.1.4
charset-normalizer==3.4.4
click==8.3.1
click-didyoumean==0.3.1
click-plugins==1.1.1.2
click-repl==0.3.0
cligj==0.7.2
colorama==0.4.6
contourpy==1.3.2
coverage==7.13.1
cycler==0.12.1
et_xmlfile==2.0.0
exceptiongroup==1.3.1
fastapi==0.128.0
fiona==1.10.1
Flask==3.1.2
flask-cors==6.0.2
flask-restx==1.3.2
fonttools==4.61.1
geographiclib==2.1
geopandas==0.14.3
geopy==2.4.1
h11==0.16.0
httpcore==1.0.9
httpx==0.28.1
idna==3.11
ImageIO==2.37.2
importlib_resources==6.5.2
iniconfig==2.3.0
itsdangerous==2.2.0
Jinja2==3.1.6
joblib==1.3.2
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
kiwisolver==1.4.9
kombu==5.6.2
lazy_loader==0.4
llvmlite==0.46.0
MarkupSafe==3.0.3
matplotlib==3.10.0
molmass==2023.8.30
narwhals==2.14.0
networkx==3.4.2
numba==0.63.1
numpy==2.1.3
openpyxl==3.1.5
packaging==25.0
pandas==2.2.3
pefile==2024.8.26
pillow==12.1.0
plotly==5.20.0
pluggy==1.6.0
prompt_toolkit==3.0.52
psutil==7.2.1
pybaselines==1.1.0
pyproj==3.6.1
pydantic==2.12.5
pydantic_core==2.41.5
Pygments==2.19.2
pyinstaller==6.17.0
pyinstaller-hooks-contrib==2025.11
pyogrio==0.12.1
pyparsing==3.3.1
pyproj==3.7.1
pytest==8.1.1
pytest-asyncio
pytest-cov==7.0.0
pytest-mock==3.15.1
python-dateutil==2.9.0.post0
python-multipart==0.0.21
pytz==2025.2
pywin32-ctypes==0.2.3
PyYAML==6.0.1
redis==7.1.0
referencing==0.37.0
requests==2.32.5
rpds-py==0.30.0
scikit-gstat==1.0.19
scikit-image==0.24.0
scikit-learn==1.7.2
scipy==1.15.1
flask
werkzeug
flask-cors
waitress
psutil
shapely==2.1.2
simplekml==1.3.6
six==1.17.0
starlette==0.50.0
tenacity==9.1.2
threadpoolctl==3.6.0
tifffile==2025.5.10
tomli==2.4.0
tqdm==4.67.1
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.3
tzlocal==5.3.1
urllib3==2.6.2
uvicorn==0.40.0
vine==5.1.0
waitress==3.0.2
wcwidth==0.2.14
Werkzeug==3.1.5

View File

@ -397,20 +397,11 @@ def process_data_async(task_id, data_path, config_path, job_output_dir):
logger.error(f"Job {task_id}: Failed to load config from {config_path}: {str(e)}")
raise
# 根据配置 output_dir 更新目录
Config.update_directories_from_config(config_path)
# 不再全局覆盖 Config 类属性(避免多用户并发冲突)
# 每个任务使用独立目录,基础路径由 INI 配置决定task_id 保证隔离
# Config.update_directories_from_config(config_path) ← 已移除
# 将更新后的目录同步到 app.config
app.config['UPLOAD_FOLDER'] = Config.UPLOAD_FOLDER
app.config['OUTPUT_FOLDER'] = Config.OUTPUT_FOLDER
# 任务状态持久化现在仅使用 SQLite
# JSON 持久化已禁用
# from .shared import set_task_status_file_path, load_task_status_from_file
# set_task_status_file_path(Config.OUTPUT_FOLDER / "task_status.json")
# load_task_status_from_file()
# 更新任务目录到正确的基于配置的路径下
# 使用 INI 配置的基础目录 + task_id 构建任务专属路径
from pathlib import Path
job_upload_dir = Path(Config.UPLOAD_FOLDER) / task_id
job_output_dir = Path(Config.OUTPUT_FOLDER) / task_id

View File

@ -52,7 +52,11 @@ def algorithmic_baseline(
signal = (df[gas] - bkg)[~bkg_points]
df[f"{gas}_signal"] = np.invert(bkg_points)
fig = None # plotting disabled
try:
from . import plotting
fig = plotting.background_plotting(df, gas)
except Exception:
fig = None
output_text = (
f"Baseline algorithm: {algorithm}\n"
f"Positive and negative 95% percentile of baseline: {np.percentile(background, 2.5):.2f} ppm, "

View File

@ -544,17 +544,17 @@ def calculate_pressure(df, max_samples=None, height_tolerance=10.0, height_bin_s
df['pressure'] = None # 初始化
df.loc[sample_df.index, 'pressure'] = pressures
# 对于未计算的行,使用插值或平均值填充
if max_samples is not None and len(df) > max_samples:
# 只计算了部分行,用平均值填充其余行
valid_pressures_for_mean = [p for p in pressures if p is not None]
if valid_pressures_for_mean:
mean_pressure = sum(valid_pressures_for_mean) / len(valid_pressures_for_mean)
df['pressure'] = df['pressure'].fillna(mean_pressure)
print(f"使用平均气压填充其余 {len(df) - max_samples} 行: {mean_pressure:.1f} hPa")
# 统计并填充缺失气压值用已有有效值的平均值填充所有NaN
valid_pressures = [p for p in pressures if p is not None]
nan_count_before = df['pressure'].isna().sum()
if valid_pressures:
mean_pressure = sum(valid_pressures) / len(valid_pressures)
df['pressure'] = df['pressure'].fillna(mean_pressure)
if nan_count_before > 0:
print(f"使用平均气压 {mean_pressure:.1f} hPa 填充了 {nan_count_before} 行缺失气压值")
# 统计信息
valid_pressures = [p for p in pressures if p is not None]
if valid_pressures:
avg_pressure = sum(valid_pressures) / len(valid_pressures)
print(f"成功计算 {len(valid_pressures)}/{actual_samples} 个气压值,平均值: {avg_pressure:.1f} hPa")

View File

@ -33,7 +33,7 @@ def gas_density(local_pressure: float, local_temperature_celsius: float, gas: st
local_volume = (
gas_variables["standard_molar_volume"]
* (gas_variables["standard_pressure"] / local_pressure)
* ((local_temperature_kelvin + gas_variables["standard_temperature"]) / gas_variables["standard_temperature"])
* (local_temperature_kelvin / gas_variables["standard_temperature"])
) # m3⋅mol-1
return mass(gas) / 1000 / local_volume

View File

@ -79,7 +79,7 @@ def cleanup_expired_tasks():
""").fetchall()
if rows:
current_app.logger.info(f"Janitor found {len(rows)} expired tasks to clean up")
current_app.logger.info(f"Janitor found {len(rows)} expired tasks to clean up")
for row in rows:
task_id = row['task_id']

View File

@ -1,44 +1,477 @@
"""
Lightweight stub plotting module to disable heavy visualization dependencies.
All functions return None so callers can safely check for truthiness.
Plotting module for GasFlux visualization.
Generates interactive Plotly figures for reports.
"""
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.express as px
from plotly.subplots import make_subplots
def blank_figure():
return None
def scatter_3d(*args, **kwargs):
return None
def scatter_2d(*args, **kwargs):
return None
def time_series(*args, **kwargs):
return None
def background_plotting(*args, **kwargs):
return None
def windrose(*args, **kwargs):
return None
def outliers(*args, **kwargs):
return None
def contour_krig(*args, **kwargs):
return None
def heatmap_krig(*args, **kwargs):
return None
def create_kml_file(*args, **kwargs):
"""Return an empty Plotly figure."""
fig = go.Figure()
fig.update_layout(
title="No data available",
xaxis={"visible": False},
yaxis={"visible": False},
annotations=[{"text": "Plot not available", "showarrow": False, "font": {"size": 16}}],
)
return fig
def scatter_3d(
df: pd.DataFrame,
gas: str,
x: str = "utm_easting",
y: str = "utm_northing",
z: str = "height_ato",
color_col: str | None = None,
) -> go.Figure:
"""
Create a 3D scatter plot of flight path with gas concentration coloring.
Parameters:
df: DataFrame with position and gas data.
gas: Gas name (e.g. 'co2', 'ch4').
x, y, z: Column names for coordinates.
color_col: Column to use for color scale. Defaults to normalised gas column.
Returns:
plotly.graph_objects.Figure
"""
if color_col is None:
norm_col = f"{gas}_normalised"
color_col = norm_col if norm_col in df.columns else gas
if color_col not in df.columns:
return blank_figure()
valid = df[[x, y, z, color_col]].dropna()
if len(valid) == 0:
return blank_figure()
fig = go.Figure()
fig.add_trace(
go.Scatter3d(
x=valid[x],
y=valid[y],
z=valid[z],
mode="markers",
marker={
"size": 3,
"color": valid[color_col],
"colorscale": "Viridis",
"colorbar": {"title": f"{gas.upper()} (ppm)"},
"opacity": 0.8,
},
text=[f"{v:.2f} ppm" for v in valid[color_col]],
hoverinfo="text",
name="Flight path",
)
)
fig.update_layout(
title=f"3D Flight Path {gas.upper()} Concentration",
scene={
"xaxis_title": "UTM Easting (m)",
"yaxis_title": "UTM Northing (m)",
"zaxis_title": "Height ATO (m)",
},
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def scatter_2d(
df: pd.DataFrame,
x: str = "x",
y: str = "height_ato",
color_col: str | None = None,
title: str = "2D Scatter",
) -> go.Figure:
"""Create a 2D scatter plot."""
if color_col is None or color_col not in df.columns:
color_col = None
fig = px.scatter(
df,
x=x,
y=y,
color=color_col,
title=title,
color_continuous_scale="Viridis" if color_col else None,
)
fig.update_layout(margin={"l": 0, "r": 0, "t": 40, "b": 0})
return fig
def time_series(
df: pd.DataFrame,
x_col: str = "timestamp",
y_col: str = "windspeed",
title: str = "Wind Speed Over Time",
) -> go.Figure:
"""Create a time series line chart."""
if x_col not in df.columns or y_col not in df.columns:
return blank_figure()
fig = go.Figure()
fig.add_trace(
go.Scatter(
x=df[x_col],
y=df[y_col],
mode="lines",
name=y_col,
line={"color": "#3498db", "width": 1.5},
)
)
fig.update_layout(
title=title,
xaxis_title="Time",
yaxis_title=y_col,
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def background_plotting(df: pd.DataFrame, gas: str) -> go.Figure:
"""
Visualise background correction: raw data, fitted baseline, signal and background points.
Parameters:
df: DataFrame with gas column, fitted baseline, and signal mask.
gas: Gas name.
Returns:
plotly.graph_objects.Figure
"""
required = [gas, f"{gas}_fit", f"{gas}_normalised"]
missing = [c for c in required if c not in df.columns]
if missing:
return blank_figure()
signal_mask = df.get(f"{gas}_signal", pd.Series([False] * len(df)))
fig = go.Figure()
# Raw data
fig.add_trace(
go.Scatter(
y=df[gas],
mode="lines",
name=f"{gas.upper()} raw",
line={"color": "#2c3e50", "width": 1},
opacity=0.6,
)
)
# Fitted baseline
fig.add_trace(
go.Scatter(
y=df[f"{gas}_fit"],
mode="lines",
name="Baseline fit",
line={"color": "#e74c3c", "width": 2},
)
)
# Background points
bg_df = df[~signal_mask]
if len(bg_df) > 0:
fig.add_trace(
go.Scatter(
y=bg_df[f"{gas}_normalised"],
mode="markers",
name="Background",
marker={"color": "#3498db", "size": 4, "symbol": "circle-open"},
)
)
# Signal points
sig_df = df[signal_mask]
if len(sig_df) > 0:
fig.add_trace(
go.Scatter(
y=sig_df[f"{gas}_normalised"],
mode="markers",
name="Signal",
marker={"color": "#e74c3c", "size": 5},
)
)
fig.update_layout(
title=f"Background Correction {gas.upper()}",
xaxis_title="Sample Index",
yaxis_title=f"{gas.upper()} (ppm)",
margin={"l": 0, "r": 0, "t": 40, "b": 0},
legend={"orientation": "h", "yanchor": "bottom", "y": 1.02},
)
return fig
def windrose(
df: pd.DataFrame,
winddir_col: str = "winddir",
windspeed_col: str = "windspeed",
) -> go.Figure:
"""
Create a wind rose plot using polar bar chart.
Parameters:
df: DataFrame with wind direction and speed.
winddir_col: Wind direction column (0-360 degrees).
windspeed_col: Wind speed column.
Returns:
plotly.graph_objects.Figure
"""
if winddir_col not in df.columns or windspeed_col not in df.columns:
return blank_figure()
valid = df[[winddir_col, windspeed_col]].dropna()
if len(valid) == 0:
return blank_figure()
# Bin wind direction into 16 sectors
n_sectors = 16
sector_width = 360 / n_sectors
sectors = np.arange(0, 360, sector_width)
sector_labels = ["N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE",
"S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"]
mean_speeds = []
for i, start in enumerate(sectors):
end = start + sector_width
mask = (valid[winddir_col] >= start) & (valid[winddir_col] < end)
if mask.any():
mean_speeds.append(valid.loc[mask, windspeed_col].mean())
else:
mean_speeds.append(0)
fig = go.Figure()
fig.add_trace(
go.Barpolar(
r=mean_speeds,
theta=sector_labels,
name="Wind Speed (m/s)",
marker_color="#3498db",
marker_line_color="#2980b9",
opacity=0.7,
)
)
fig.update_layout(
title="Wind Rose",
polar={
"radialaxis": {"title": "Wind Speed (m/s)", "visible": True},
"angularaxis": {"direction": "clockwise", "rotation": 90},
},
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def outliers(
df: pd.DataFrame, column: str, name: str = ""
) -> go.Figure | None:
"""
Visualise outlier detection using IQR method.
Parameters:
df: DataFrame.
column: Column to check for outliers.
name: Dataset name for title.
Returns:
plotly.graph_objects.Figure or None
"""
if column not in df.columns:
return None
valid = df[column].dropna()
if len(valid) == 0:
return None
q1 = valid.quantile(0.25)
q3 = valid.quantile(0.75)
iqr = q3 - q1
fence_low = q1 - 3 * iqr
fence_high = q3 + 3 * iqr
outliers_mask = (valid < fence_low) | (valid > fence_high)
fig = go.Figure()
fig.add_trace(
go.Box(
y=valid,
name=column,
boxpoints="outliers",
marker={"color": "#e74c3c", "size": 4},
line={"color": "#2c3e50"},
)
)
title = f"Outlier Detection {column}"
if name:
title += f" ({name})"
fig.update_layout(
title=title,
yaxis_title=column,
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def contour_krig(krig_variables: dict) -> go.Figure:
"""
Create a contour plot of the kriging interpolation field.
Parameters:
krig_variables: Dictionary containing 'xx', 'yy', 'field', and 'gas'.
Returns:
plotly.graph_objects.Figure
"""
xx = krig_variables.get("xx")
yy = krig_variables.get("yy")
field = krig_variables.get("field")
gas = krig_variables.get("gas", "gas")
if xx is None or yy is None or field is None:
return blank_figure()
# np.mgrid produces shape (x_nodes, y_nodes).
# xx varies along axis 0 → xx[:, 0] gives unique x values
# yy varies along axis 1 → yy[0, :] gives unique y values
# field[x][y] needs transpose → field.T for plotly's z[y][x]
x_1d = xx[:, 0] if xx.ndim > 1 else xx.flatten()
y_1d = yy[0, :] if yy.ndim > 1 else yy.flatten()
fig = go.Figure()
fig.add_trace(
go.Contour(
z=field.T,
x=x_1d,
y=y_1d,
colorscale="RdBu_r",
contours={
"coloring": "fill",
"showlabels": True,
"labelfont": {"size": 10, "color": "#333"},
},
colorbar={"title": f"{gas.upper()} Flux (kg/h/m²)"},
)
)
fig.update_layout(
title=f"Kriging Interpolation {gas.upper()} Flux Contour",
xaxis_title="Distance along plane (m)",
yaxis_title="Height ATO (m)",
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def heatmap_krig(krig_variables: dict) -> go.Figure:
"""
Create a heatmap of the kriging grid with overlaid data points.
Parameters:
krig_variables: Dictionary with kriging output.
Returns:
plotly.graph_objects.Figure
"""
xx = krig_variables.get("xx")
yy = krig_variables.get("yy")
field = krig_variables.get("field")
gas = krig_variables.get("gas", "gas")
if xx is None or yy is None or field is None:
return blank_figure()
# np.mgrid produces shape (x_nodes, y_nodes).
# xx varies along axis 0 → xx[:, 0] gives unique x values
# yy varies along axis 1 → yy[0, :] gives unique y values
# field[x][y] needs transpose → field.T for plotly's z[y][x]
x_1d = xx[:, 0] if xx.ndim > 1 else xx.flatten()
y_1d = yy[0, :] if yy.ndim > 1 else yy.flatten()
fig = go.Figure()
fig.add_trace(
go.Heatmap(
z=field.T,
x=x_1d,
y=y_1d,
colorscale="RdBu_r",
colorbar={"title": f"{gas.upper()} Flux (kg/h/m²)"},
zmid=0,
)
)
fig.update_layout(
title=f"Kriging Grid {gas.upper()} Flux Heatmap",
xaxis_title="Distance along plane (m)",
yaxis_title="Height ATO (m)",
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
def semivariogram_plot(semivariogram, gas: str = "") -> go.Figure:
"""
Create a semivariogram plot from a scikit-gstat Variogram object.
Parameters:
semivariogram: scikit-gstat Variogram object.
gas: Gas name for title.
Returns:
plotly.graph_objects.Figure
"""
try:
bins = semivariogram.bins
experimental = semivariogram.experimental
fig = go.Figure()
# Experimental semivariogram points
fig.add_trace(
go.Scatter(
x=bins,
y=experimental,
mode="markers",
name="Experimental",
marker={"size": 8, "color": "#3498db"},
)
)
# Fitted model line
if hasattr(semivariogram, "model") and semivariogram.model is not None:
x_line = np.linspace(0, bins.max(), 100)
y_line = semivariogram.model(x_line)
fig.add_trace(
go.Scatter(
x=x_line,
y=y_line,
mode="lines",
name=f"Fitted ({semivariogram.model.__class__.__name__})",
line={"color": "#e74c3c", "width": 2},
)
)
title = "Semivariogram"
if gas:
title += f" {gas.upper()}"
fig.update_layout(
title=title,
xaxis_title="Lag Distance (m)",
yaxis_title="Semivariance",
margin={"l": 0, "r": 0, "t": 40, "b": 0},
)
return fig
except Exception:
return blank_figure()
def create_kml_file(*args, **kwargs) -> None:
"""KML file generation not yet implemented."""
return None

View File

@ -1,11 +1,158 @@
import requests
import time
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
import logging
from functools import lru_cache
logger = logging.getLogger(__name__)
# 全局 Session 复用 TCP 连接,避免 SSL 握手失败
_session = None
def _get_session():
"""获取或创建持久 HTTP Session复用连接"""
global _session
if _session is None:
_session = requests.Session()
# 设置适配器,增大连接池
from requests.adapters import HTTPAdapter
adapter = HTTPAdapter(pool_connections=5, pool_maxsize=10, max_retries=0)
_session.mount("https://", adapter)
_session.mount("http://", adapter)
return _session
def _exponential_backoff_request(url, params, max_retries=4, base_timeout=45):
"""
带指数退避的 HTTP GET 请求。
Args:
url: 请求 URL
params: 查询参数
max_retries: 最大重试次数(含首次尝试)
base_timeout: 基础超时时间(秒)
Returns:
requests.Response 对象
Raises:
requests.RequestException: 所有重试失败后抛出
"""
session = _get_session()
last_exception = None
for attempt in range(max_retries):
try:
# 每次重试增加超时时间45, 60, 75, 90
timeout = base_timeout + attempt * 15
response = session.get(url, params=params, timeout=timeout)
response.raise_for_status()
return response
except (requests.exceptions.ConnectionError,
requests.exceptions.Timeout,
requests.exceptions.SSLError) as e:
last_exception = e
if attempt < max_retries - 1:
wait = 2 ** attempt # 1s, 2s, 4s, 8s
logger.warning(
f"Open-Meteo 请求失败 (第{attempt+1}次): {type(e).__name__}. "
f"{wait}秒后重试... URL: {url} params: {params}"
)
time.sleep(wait)
else:
logger.error(
f"Open-Meteo 请求最终失败 ({max_retries}次尝试): {type(e).__name__}: {e}"
)
except requests.exceptions.HTTPError as e:
# HTTP 错误不重试(如 404, 400 等)
logger.error(f"Open-Meteo HTTP错误: {e}")
raise
raise last_exception
# 缓存:相同 (lat, lon, date, time) 的请求结果,避免重复 API 调用
# maxsize=256 足够缓存一次飞行的所有档位
@lru_cache(maxsize=256)
def _get_pressure_cached(lat: float, lon: float, altitude: float, date: str, time: str) -> float | None:
"""
带缓存的单次气压查询(内部函数)。
坐标四舍五入到小数点后 4 位(~11m精度以提高缓存命中率。
"""
# 坐标取 4 位小数作为缓存键(同一档位内坐标几乎相同)
cache_lat = round(lat, 4)
cache_lon = round(lon, 4)
cache_alt = round(altitude, 1) # 高度取1位小数
return _get_pressure_impl(cache_lat, cache_lon, cache_alt, date, time)
def _get_pressure_impl(lat: float, lon: float, altitude: float, date: str, time: str) -> float | None:
"""实际的 Open-Meteo API 调用(无缓存)。"""
url = "https://archive-api.open-meteo.com/v1/archive"
params = {
"latitude": lat,
"longitude": lon,
"start_date": date,
"end_date": date,
"hourly": ["pressure_msl", "surface_pressure"],
"timezone": "auto",
}
try:
logger.info(f"正在获取位置 ({lat:.6f}, {lon:.6f}) 在 {date} {time} 的气压数据...")
response = _exponential_backoff_request(url, params)
data = response.json()
if "error" in data:
logger.error(f"Open-Meteo API错误: {data['error']}")
return None
if data and "hourly" in data:
times = data["hourly"]["time"]
pressures = data["hourly"]["surface_pressure"]
if not times or not pressures:
logger.warning("未找到气压数据times 或 pressures 为空)")
return None
target_time = f"{date}T{time}"
if target_time in times:
idx = times.index(target_time)
pressure = pressures[idx]
logger.info(f"成功获取气压: {pressure} hPa")
return pressure
else:
logger.warning(f"在数据中未找到时间: {target_time},可用范围: {times[0]}{times[-1]}")
# 回退:取最近的小时
try:
closest = min(times, key=lambda t: abs(
(int(t.split("T")[1].split(":")[0]) if "T" in t else 0) -
(int(time.split(":")[0]))
))
idx = times.index(closest)
pressure = pressures[idx]
logger.info(f"使用最近时间 {closest} 的气压: {pressure} hPa")
return pressure
except Exception:
return None
else:
logger.warning("API响应中没有hourly数据")
return None
except requests.exceptions.HTTPError:
return None
except Exception as e:
logger.error(f"获取气压时发生未知错误: {type(e).__name__}: {e}")
return None
def get_pressure_at_location(lat, lon, altitude, date, time, max_retries=3, timeout=30):
"""
获取指定位置、时间、高度的气压
获取指定位置、时间、高度的气压(公共接口,保持向后兼容)。
Args:
lat: 纬度
@ -13,8 +160,8 @@ def get_pressure_at_location(lat, lon, altitude, date, time, max_retries=3, time
altitude: 海拔高度 (米)
date: 日期 (格式: YYYY-MM-DD 或 YYYY/MM/DD)
time: 时间 (格式: HH:MM 或 HH:MM:SS)
max_retries: 最大重试次数
timeout: 请求超时时间(秒)
max_retries: 最大重试次数(已废弃,由内部指数退避处理)
timeout: 请求超时时间(秒)(已废弃,由内部控制)
Returns:
float: 气压值 (hPa),获取失败返回 None
@ -22,130 +169,42 @@ def get_pressure_at_location(lat, lon, altitude, date, time, max_retries=3, time
# 标准化日期格式为 YYYY-MM-DD
def normalize_date(d):
"""将各种日期格式标准化为 YYYY-MM-DD"""
if not d:
from datetime import datetime
return datetime.now().strftime("%Y-%m-%d")
# 处理斜杠分隔符
if "/" in d:
d = d.replace("/", "-")
parts = d.split("-")
if len(parts) == 3:
year = parts[0]
month = parts[1].zfill(2) # 确保月份是两位数
day = parts[2].zfill(2) # 确保日期是两位数
return f"{year}-{month}-{day}"
else:
# 如果格式不正确,返回今天的日期
from datetime import datetime
return datetime.now().strftime("%Y-%m-%d")
return f"{parts[0]}-{parts[1].zfill(2)}-{parts[2].zfill(2)}"
from datetime import datetime
return datetime.now().strftime("%Y-%m-%d")
date = normalize_date(date)
# 标准化时间格式为 HH:MM
def normalize_time(t):
"""将各种时间格式标准化为 HH:MM"""
if not t or ":" not in t:
return "12:00" # 默认中午12点
return "12:00"
parts = t.split(":")
if len(parts) >= 2:
hour = parts[0].zfill(2) # 确保小时是两位数
minute = parts[1].zfill(2) # 确保分钟是两位数
return f"{hour}:{minute}"
elif len(parts) == 1:
hour = parts[0].zfill(2)
return f"{hour}:00"
else:
return "12:00"
return f"{parts[0].zfill(2)}:{parts[1].zfill(2)}"
return "12:00"
time = normalize_time(time)
url = "https://archive-api.open-meteo.com/v1/archive"
params = {
"latitude": lat,
"longitude": lon,
"start_date": date, # 格式: YYYY-MM-DD
"end_date": date,
"hourly": ["pressure_msl", "surface_pressure"],
"timezone": "auto"
}
# 创建带有重试机制的会话
session = requests.Session()
retry_strategy = Retry(
total=max_retries,
status_forcelist=[429, 500, 502, 503, 504],
backoff_factor=1
)
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount("http://", adapter)
session.mount("https://", adapter)
try:
print(f"正在获取位置 ({lat:.6f}, {lon:.6f}) 在 {date} {time} 的气压数据...")
response = session.get(url, params=params, timeout=timeout)
# 检查响应状态
response.raise_for_status()
data = response.json()
# 检查API错误
if "error" in data:
print(f"API错误: {data['error']}")
return None
# 解析气压数据
if data and "hourly" in data:
times = data["hourly"]["time"]
pressures = data["hourly"]["surface_pressure"] # 地表气压
if not times or not pressures:
print("未找到气压数据")
return None
# 根据时间找到对应气压
target_time = f"{date}T{time}"
if target_time in times:
idx = times.index(target_time)
pressure = pressures[idx]
print(f"成功获取气压: {pressure} hPa")
return pressure
else:
print(f"在数据中未找到时间: {target_time}")
print(f"可用时间范围: {times[0]}{times[-1]}")
return None
else:
print("API响应中没有hourly数据")
return None
except requests.exceptions.ConnectionError as e:
print(f"网络连接错误: {e}")
print("请检查网络连接或稍后重试")
return None
except requests.exceptions.Timeout as e:
print(f"请求超时: {e}")
print(f"已重试 {max_retries} 次,请检查网络连接")
return None
except requests.exceptions.HTTPError as e:
print(f"HTTP错误: {e}")
return None
except ValueError as e:
print(f"数据解析错误: {e}")
return None
return _get_pressure_cached(
float(lat), float(lon), float(altitude), date, time
)
except Exception as e:
print(f"未知错误: {e}")
logger.error(f"get_pressure_at_location 失败: {type(e).__name__}: {e}")
return None
finally:
session.close()
def batch_get_pressure(data_list):
"""
批量获取多个位置的气压数据
批量获取多个位置的气压数据
Args:
data_list: 包含 (lat, lon, altitude, date, time) 元组的列表
@ -154,37 +213,36 @@ def batch_get_pressure(data_list):
list: 气压值列表
"""
results = []
for i, (lat, lon, alt, date, time) in enumerate(data_list):
for i, (lat, lon, alt, date, time_val) in enumerate(data_list):
print(f"\n处理第 {i+1} 个位置...")
pressure = get_pressure_at_location(lat, lon, alt, date, time)
pressure = get_pressure_at_location(lat, lon, alt, date, time_val)
results.append(pressure)
if pressure is not None:
print(f"位置 {i+1}: {pressure} hPa")
else:
print(f"位置 {i+1}: 获取失败")
# 添加短暂延迟避免请求过于频繁
# 短暂延迟避免请求过于频繁
if i < len(data_list) - 1:
time.sleep(0.5)
time.sleep(0.3)
return results
# 使用示例
if __name__ == "__main__":
print("=== 气压数据获取工具 ===\n")
# 单个位置示例
print("1. 单个位置查询:")
pressure = get_pressure_at_location(
lat=40.3491370, # 纽约纬度
lon=115.7855289, # 纽约经度 (西经)
altitude=435.789, # 海拔10米
lat=40.3491370,
lon=115.7855289,
altitude=435.789,
date="2016-02-12",
time="08:00" # HH:MM格式
time="08:00"
)
if pressure is not None:
print(f"纽约当前气压: {pressure} hPa")
else:
print("获取纽约气压数据失败")