Compare commits

...

9 Commits

Author SHA1 Message Date
DXC
d5dd2ba1da chore: 移除 frontend/ 和图标资源目录;彻底清理遗留脚手架 2026-06-08 12:13:37 +08:00
DXC
1cbd38a8e0 chore: 从索引移除运行时产物、个人配置、旧脚手架;完善 .gitignore 2026-06-08 12:12:11 +08:00
DXC
e3debbcb15 fix(step8): 修复外部模型字典透传断链 + 规范化 loaded_model_data 防 Ridge subscriptable 崩溃 2026-06-08 11:36:36 +08:00
DXC
2b76d7908f feat(step8): 外部模型从单文件升级为母文件夹多模型字典扫描 2026-06-08 09:56:02 +08:00
DXC
4efe5b871e feat(gui): 一键运行智能预检
4 段预检彻底解决切换 PipelineRunner 后报 TypeError/静默跳过等问题, 并升级一键运行 UX:

- 预检 1: work_path + log + scan + auto_populate(无需弹窗, 静默回填)

- 预检 2: step3 波段越界 60s 倒计时弹窗(BandConfirmDialog) + gdal 主线程同步读 RasterCount, 越界时 SpinBox 回写 UI

- 预检 3: img_path 硬校验(warning + 跳 step1 + return)

- 预检 4: csv_path 软提示(information + 不 return, 让用户在 QMessageBox.question 二次确认时自己决定是否跳过训练)

新增 src/gui/dialogs.py: BandConfirmDialog(QDialog 子类, 60s 倒计时)
2026-06-04 10:38:46 +08:00
DXC
2139715829 fix(runner): step5 严格依赖 step4 产物 + 拒绝静默跳过
- step5.requires 加入 processed_csv_path(step4 产物) 并显式 parameter_map 到 csv_path 形参;step5.skip_when_missing=False 配合 Facade **kwargs 兜底

- parameter_map 双向映射规避 L2 顺序注入冲突: processed_csv_path→csv_path(主), csv_path→_raw_csv_ignored(占位, 落 **kwargs)

- PipelineRunner.run() skip_when_missing 块新增 _notify 通知, 让 GUI 知道具体缺了什么(拒绝静默跳过)
2026-06-04 10:38:33 +08:00
DXC
64aa5b8f40 fix(runner): 14 Facade kwargs 兜底 + 4 spec parameter_map 修正 + step6_75 路由切到 indices
- 14 个 stepX_... Facade 形参表末尾加 **kwargs,杜绝 Runner 注入未声明 key 时的 TypeError(典型:step3 收到 glint_mask_path)

- runner._invoke user_overrides 合并加 v is not None and v != '' 过滤,避免 GUI 面板空值覆盖 ctx 中已写入的有效路径

- PIPELINE_STEPS 加 4 个 parameter_map 修正 ctx 字段名→形参名错位:step6_5/6_75: training_csv_path→csv_path;step8_5: models_dir→non_empirical_models_dir;step8_75: models_dir→custom_regression_dir

- step6_75 路由从 training_csv_path 切到 indices_path(requires + parameter_map 同步);配合 skip_when_missing,未跑 step5_5 时自动 skip

- worker_thread.py: mode='full' 切到 PipelineRunner + PipelineContext 调度
2026-06-04 09:15:04 +08:00
DXC
343e316799 refactor(pipeline): 路径直接传输 — 统一 ctx 字段名/panel key/step 形参名 2026-06-03 17:29:41 +08:00
DXC
517bb28611 snapshot: 路线 B 重构前原状(pipeline 包首次入 git) 2026-06-03 16:31:45 +08:00
63 changed files with 3483 additions and 2829 deletions

40
.gitignore vendored
View File

@ -155,3 +155,43 @@ tmp/
*.bak
*.backup
*~
# ============================================================
# 不应进入版本控制的文件类型
# ============================================================
# Qwen Code 用户配置(个人环境,每次 clone 都不同)
.qwen/settings.json
.qwen/settings.json.orig
# Qwen Code 自动生成的 skill 文件(每次会话重新生成)
.qwen/skills/
# GUI 运行时生成的文件
src/gui/scaler_params.pkl
src/gui/crash_dump.txt
# 临时/调试脚本(根目录)
降采样光谱.py
1.py
tset.py
# 报告与文档(本地工作产物)
封装问题分析报告.md
软件说明.md
软件说明2.md
# 数据子目录中非 .gitkeep 的生成文件
data/sub/waterindex*.csv
data/sub/waterindex*.xlsx
data/sub/png/watermask.png
# 图标文件(仅需保留 vector/svg删除像素图标压缩包副本
data/icons-1/
data/icons/
# 旧版脚手架(遗留实验代码)
new/
# 前端脚手架(未集成的独立 Vue 项目)
frontend/

4
1.py
View File

@ -1,4 +0,0 @@
new_wavelengths = [np.mean(wavelengths[i:i+3]) for i in range(0, len(wavelengths), 3)]
print(new_wavelengths)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 950 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 978 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 300 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 250 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.9 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 884 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

View File

@ -1,46 +0,0 @@
Formula_Name,Category,Formula,Reference
BGA_Am09KBBI,Phycocyanin (BGA_PC),(w686 - w658) / (w686 + w658),"Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S.; Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery, Optics Express, 2009, 17, 11, 1-13."
BGA_Be162B643sub629,Phycocyanin (BGA_PC),w644 - w629,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be162B700sub601,Phycocyanin (BGA_PC),w700 - w601,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be162BsubPhy,Phycocyanin (BGA_PC),w715 - w615,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540."
BGA_Be16FLHBlueRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w458 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be16FLHGreenRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w558 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be16FLHVioletRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w444 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be16MPI,Phycocyanin (BGA_PC),(w615 - w601) - (w644 - w601),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be16NDPhyI,Phycocyanin (BGA_PC),(w700 - w622) / (w700 + w622),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540."
BGA_Be16NDPhyI644over615,Phycocyanin (BGA_PC),(w644 - w615) / (w644 + w615),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 541."
BGA_Be16NDPhyI644over629,Phycocyanin (BGA_PC),(w644 - w629) / (w644 + w629),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 542."
BGA_Be16Phy2BDA644over629,Phycocyanin (BGA_PC),w644 / w629,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 545."
BGA_Da052BDA,Phycocyanin (BGA_PC),w714 / w672,"Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672."
BGA_Go04MCI,Phycocyanin (BGA_PC),w709 - w681 - (w753 - w681),"Gower, J.F.R.; Brown,L.; Borstad, G.A.; Observation of chlorophyll fluorescence in west coast waters of Canada using the MODIS satellite sensor. Can. J. Remote Sens., 2004, 30 (1), 17<31><37>?5."
BGA_HU103BDA,Phycocyanin (BGA_PC),(((1 / w615) - (1 / w600)) - w725),"Hunter, P.D.; Tyler, A.N.; Willby, N.J.; Gilvear, D.J.; The spatial dynamics of vertical migration by Microcystis aeruginosa in a eutrophic shallow lake: A case study using high spatial resolution time-series airborne remote sensing. Limn. Oceanogr. 2008, 53, 2391-2406"
BGA_Ku15PhyCI,Phycocyanin (BGA_PC),(-1 * (W681 - W665 - (W709 - W665))),"Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-10."
BGA_Ku15SLH,Phycocyanin (BGA_PC),(w715 - w658) + (w715 - w658),"Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-11."
BGA_MI092BDA,Phycocyanin (BGA_PC),w700 / w600,"Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758<35><38>?75."
BGA_MM092BDA,Phycocyanin (BGA_PC),w724 / w600,"Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758<35><38>?76."
BGA_MM12NDCIalt,Phycocyanin (BGA_PC),(w700 - w658) / (w700 + w658),"Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114003"
BGA_MM143BDAopt,Phycocyanin (BGA_PC),((1 / w629) - (1 / w659)) * w724,"Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114004"
BGA_SI052BDA,Phycocyanin (BGA_PC),w709 / w620,"Simis, S. G. H.; Peters, S.W. M.; Gons, H. J.; Remote sensing of the cyanobacteria pigment phycocyanin in turbid inland water. Limn. Oceanogr., 2005, 50, 237<33><37>?45"
BGA_SM122BDA,Phycocyanin (BGA_PC),w709 / w600,"Mishra, S. Remote sensing of cyanobacteria in turbid productive waters, PhD Dissertation. Mississippi State University, USA. 2012."
BGA_SY002BDA,Phycocyanin (BGA_PC),w650 / w625,"Schalles, J.; Yacobi, Y. Remote detection and seasonal patterns of phycocyanin, carotenoid and chlorophyll-a pigments in eutrophic waters. Archiv fur Hydrobiologie, Special Issues Advances in Limnology, 2000, 55,153<35><33>?68"
BGA_Wy08CI,Phycocyanin (BGA_PC),(-1 * (W686 - W672 - (W715 - W672))),"Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672."
Chl_Al10SABI,chlorophyll_a,(w857 - w644) / (w458 + w529),"Alawadi, F. Detection of surface algal blooms using the newly developed algorithm surface algal bloom index (SABI). Proc. SPIE 2010, 7825."
Chl_Am092Bsub,chlorophyll_a,w681 - w665,"Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S. Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery. Opt. Express 2009, 17, 9126<32><36>?144."
Chl_Be16FLHblue,chlorophyll_a,w529 - (w644 + (w458 - w644)),"Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30."
Chl_Be16FLHviolet,chlorophyll_a,w529 - (w644 + (w429 - w644)),"Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30."
Chl_Be16NDTIblue,chlorophyll_a,(w658 - w458) / (w658 + w458),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 543."
Chl_Be16NDTIviolet,chlorophyll_a,(w658 - w444) / (w658 + w444),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 544."
Chl_De933BDA,chlorophyll_a,w600 - w648 - w625,"Dekker, A.; Detection of the optical water quality parameters for eutrophic waters by high resolution remote sensing, Ph.D. thesis, 1993, Free University, Amsterdam."
Chl_Gi033BDA,chlorophyll_a,((1 / w672) - (1 / w715)) * w757,"Gitelson, A.A.; U. Gritz, and M. N. Merzlyak.; Relationships between leaf chlorophyll content and spectral reflectance and algorithms for non-destructive chlorophyll assessment in higher plant leaves. J. Plant Phys. 2003, 160, 271-282."
Chl_Kn07KIVU,chlorophyll_a,(w458 - w644) / w529,"Kneubuhler, M.; Frank T.; Kellenberger, T.W; Pasche N.; Schmid M.; Mapping chlorophyll-a in Lake Kivu with remote sensing methods. 2007, Proceedings of the Envisat Symposium 2007, Montreux, Switzerland 23<32><33>?7 April 2007 (ESA SP-636, July 2007)."
Chl_MM12NDCI,chlorophyll_a,(w715 - w686) / (w715 + w686),"Mishra, S.; and Mishra, D.R. Normalized difference chlorophyll index: A novel model for remote estimation of chlorophyll-a concentration in turbid productive waters, Remote Sens. Environ., 2012, 117, 394-406"
Chl_Zh10FLH,chlorophyll_a,w686 - (w715 + (w672 - w751)),"Zhao, D.Z.; Xing, X.G.; Liu, Y.G.; Yang, J.H.; Wang, L. The relation of chlorophyll-a concentration with the reflectance peak near 700 nm in algae-dominated waters and sensitivity of fluorescence algorithms for detecting algal bloom. Int. J. Remote Sens. 2010, 31, 39-48"
Turb_Be16GreenPlusRedBothOverViolet,Turbidity,(w558 + w658) / w444,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538"
Turb_Be16RedOverViolet,Turbidity,w658 / w444,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539"
Turb_Bow06RedOverGreen,Turbidity,w658 / w558,"Bowers, D. G., and C. E. Binding. 2006. 闁炽儲缈籬e Optical Properties of Mineral Suspended Particles: A Review and Synthesis.<2E><>?Estuarine Coastal and Shelf Science 67 (1<><31>?): 219<31><39>?30. doi:10.1016/j.ecss.2005.11.010"
Turb_Chip09NIROverGreen,Turbidity,w857 / w558,"Chipman, J. W.; Olmanson, L.G.; Gitelson, A.A.; Remote sensing methods for lake management: A guide for resource managers and decision-makers. 2009."
Turb_Dox02NIRoverRed,Turbidity,w857 / w658,"Doxaran, D., Froidefond, J.-M.; Castaing, P. ; A reflectance band ratio used to estimate suspended matter concentrations in sediment-dominated coastal waters, Remote Sens., 2002, 23, 5079-5085"
Turb_Frohn09GreenPlusRedBothOverBlue,Turbidity,(w558 + w658) / w458,"Frohn, R. C., & Autrey, B. C. (2009). Water quality assessment in the Ohio River using new indices for turbidity and chlorophyll-a with Landsat-7 Imagery. Draft Internal Report, US Environmental Protection Agency."
Turb_Harr92NIR,Turbidity,w857,"Schiebe F.R., Harrington J.A., Ritchie J.C. Remote-Sensing of Suspended Sediments闁炽儲鏁刪e Lake Chicot, Arkansas Project. Int. J. Remote Sens. 1992;13:1487<38><37>?509"
Turb_Lath91RedOverBlue,Turbidity,w658 / w458,"Lathrop, R. G., Jr., T. M. Lillesand, and B. S. Yandell, 1991. Testing the utility of simple multi-date Thematic Mapper calibration algorithms for monitoring turbid inland waters. International Journal of Remote Sensing"
Turb_Moore80Red,Turbidity,w658,"Moore, G.K., Satellite remote sensing of water turbidity, Hydrological Sciences, 1980, 25, 4, 407-422"
1 Formula_Name Category Formula Reference
2 BGA_Am09KBBI Phycocyanin (BGA_PC) (w686 - w658) / (w686 + w658) Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S.; Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery, Optics Express, 2009, 17, 11, 1-13.
3 BGA_Be162B643sub629 Phycocyanin (BGA_PC) w644 - w629 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
4 BGA_Be162B700sub601 Phycocyanin (BGA_PC) w700 - w601 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
5 BGA_Be162BsubPhy Phycocyanin (BGA_PC) w715 - w615 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540.
6 BGA_Be16FLHBlueRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w458 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
7 BGA_Be16FLHGreenRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w558 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
8 BGA_Be16FLHVioletRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w444 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
9 BGA_Be16MPI Phycocyanin (BGA_PC) (w615 - w601) - (w644 - w601) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
10 BGA_Be16NDPhyI Phycocyanin (BGA_PC) (w700 - w622) / (w700 + w622) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540.
11 BGA_Be16NDPhyI644over615 Phycocyanin (BGA_PC) (w644 - w615) / (w644 + w615) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 541.
12 BGA_Be16NDPhyI644over629 Phycocyanin (BGA_PC) (w644 - w629) / (w644 + w629) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 542.
13 BGA_Be16Phy2BDA644over629 Phycocyanin (BGA_PC) w644 / w629 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 545.
14 BGA_Da052BDA Phycocyanin (BGA_PC) w714 / w672 Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672.
15 BGA_Go04MCI Phycocyanin (BGA_PC) w709 - w681 - (w753 - w681) Gower, J.F.R.; Brown,L.; Borstad, G.A.; Observation of chlorophyll fluorescence in west coast waters of Canada using the MODIS satellite sensor. Can. J. Remote Sens., 2004, 30 (1), 17?5.
16 BGA_HU103BDA Phycocyanin (BGA_PC) (((1 / w615) - (1 / w600)) - w725) Hunter, P.D.; Tyler, A.N.; Willby, N.J.; Gilvear, D.J.; The spatial dynamics of vertical migration by Microcystis aeruginosa in a eutrophic shallow lake: A case study using high spatial resolution time-series airborne remote sensing. Limn. Oceanogr. 2008, 53, 2391-2406
17 BGA_Ku15PhyCI Phycocyanin (BGA_PC) (-1 * (W681 - W665 - (W709 - W665))) Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-10.
18 BGA_Ku15SLH Phycocyanin (BGA_PC) (w715 - w658) + (w715 - w658) Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-11.
19 BGA_MI092BDA Phycocyanin (BGA_PC) w700 / w600 Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758?75.
20 BGA_MM092BDA Phycocyanin (BGA_PC) w724 / w600 Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758?76.
21 BGA_MM12NDCIalt Phycocyanin (BGA_PC) (w700 - w658) / (w700 + w658) Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114003
22 BGA_MM143BDAopt Phycocyanin (BGA_PC) ((1 / w629) - (1 / w659)) * w724 Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114004
23 BGA_SI052BDA Phycocyanin (BGA_PC) w709 / w620 Simis, S. G. H.; Peters, S.W. M.; Gons, H. J.; Remote sensing of the cyanobacteria pigment phycocyanin in turbid inland water. Limn. Oceanogr., 2005, 50, 237?45
24 BGA_SM122BDA Phycocyanin (BGA_PC) w709 / w600 Mishra, S. Remote sensing of cyanobacteria in turbid productive waters, PhD Dissertation. Mississippi State University, USA. 2012.
25 BGA_SY002BDA Phycocyanin (BGA_PC) w650 / w625 Schalles, J.; Yacobi, Y. Remote detection and seasonal patterns of phycocyanin, carotenoid and chlorophyll-a pigments in eutrophic waters. Archiv fur Hydrobiologie, Special Issues Advances in Limnology, 2000, 55,153?68
26 BGA_Wy08CI Phycocyanin (BGA_PC) (-1 * (W686 - W672 - (W715 - W672))) Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672.
27 Chl_Al10SABI chlorophyll_a (w857 - w644) / (w458 + w529) Alawadi, F. Detection of surface algal blooms using the newly developed algorithm surface algal bloom index (SABI). Proc. SPIE 2010, 7825.
28 Chl_Am092Bsub chlorophyll_a w681 - w665 Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S. Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery. Opt. Express 2009, 17, 9126?144.
29 Chl_Be16FLHblue chlorophyll_a w529 - (w644 + (w458 - w644)) Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30.
30 Chl_Be16FLHviolet chlorophyll_a w529 - (w644 + (w429 - w644)) Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30.
31 Chl_Be16NDTIblue chlorophyll_a (w658 - w458) / (w658 + w458) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 543.
32 Chl_Be16NDTIviolet chlorophyll_a (w658 - w444) / (w658 + w444) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 544.
33 Chl_De933BDA chlorophyll_a w600 - w648 - w625 Dekker, A.; Detection of the optical water quality parameters for eutrophic waters by high resolution remote sensing, Ph.D. thesis, 1993, Free University, Amsterdam.
34 Chl_Gi033BDA chlorophyll_a ((1 / w672) - (1 / w715)) * w757 Gitelson, A.A.; U. Gritz, and M. N. Merzlyak.; Relationships between leaf chlorophyll content and spectral reflectance and algorithms for non-destructive chlorophyll assessment in higher plant leaves. J. Plant Phys. 2003, 160, 271-282.
35 Chl_Kn07KIVU chlorophyll_a (w458 - w644) / w529 Kneubuhler, M.; Frank T.; Kellenberger, T.W; Pasche N.; Schmid M.; Mapping chlorophyll-a in Lake Kivu with remote sensing methods. 2007, Proceedings of the Envisat Symposium 2007, Montreux, Switzerland 23?7 April 2007 (ESA SP-636, July 2007).
36 Chl_MM12NDCI chlorophyll_a (w715 - w686) / (w715 + w686) Mishra, S.; and Mishra, D.R. Normalized difference chlorophyll index: A novel model for remote estimation of chlorophyll-a concentration in turbid productive waters, Remote Sens. Environ., 2012, 117, 394-406
37 Chl_Zh10FLH chlorophyll_a w686 - (w715 + (w672 - w751)) Zhao, D.Z.; Xing, X.G.; Liu, Y.G.; Yang, J.H.; Wang, L. The relation of chlorophyll-a concentration with the reflectance peak near 700 nm in algae-dominated waters and sensitivity of fluorescence algorithms for detecting algal bloom. Int. J. Remote Sens. 2010, 31, 39-48
38 Turb_Be16GreenPlusRedBothOverViolet Turbidity (w558 + w658) / w444 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538
39 Turb_Be16RedOverViolet Turbidity w658 / w444 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539
40 Turb_Bow06RedOverGreen Turbidity w658 / w558 Bowers, D. G., and C. E. Binding. 2006. 闁炽儲缈籬e Optical Properties of Mineral Suspended Particles: A Review and Synthesis.?Estuarine Coastal and Shelf Science 67 (1?): 219?30. doi:10.1016/j.ecss.2005.11.010
41 Turb_Chip09NIROverGreen Turbidity w857 / w558 Chipman, J. W.; Olmanson, L.G.; Gitelson, A.A.; Remote sensing methods for lake management: A guide for resource managers and decision-makers. 2009.
42 Turb_Dox02NIRoverRed Turbidity w857 / w658 Doxaran, D., Froidefond, J.-M.; Castaing, P. ; A reflectance band ratio used to estimate suspended matter concentrations in sediment-dominated coastal waters, Remote Sens., 2002, 23, 5079-5085
43 Turb_Frohn09GreenPlusRedBothOverBlue Turbidity (w558 + w658) / w458 Frohn, R. C., & Autrey, B. C. (2009). Water quality assessment in the Ohio River using new indices for turbidity and chlorophyll-a with Landsat-7 Imagery. Draft Internal Report, US Environmental Protection Agency.
44 Turb_Harr92NIR Turbidity w857 Schiebe F.R., Harrington J.A., Ritchie J.C. Remote-Sensing of Suspended Sediments闁炽儲鏁刪e Lake Chicot, Arkansas Project. Int. J. Remote Sens. 1992;13:1487?509
45 Turb_Lath91RedOverBlue Turbidity w658 / w458 Lathrop, R. G., Jr., T. M. Lillesand, and B. S. Yandell, 1991. Testing the utility of simple multi-date Thematic Mapper calibration algorithms for monitoring turbid inland waters. International Journal of Remote Sensing
46 Turb_Moore80Red Turbidity w658 Moore, G.K., Satellite remote sensing of water turbidity, Hydrological Sciences, 1980, 25, 4, 407-422

Binary file not shown.

View File

@ -1,46 +0,0 @@
Formula_Name,Category,Formula,Reference
BGA_Am09KBBI,Phycocyanin (BGA_PC),(w686 - w658) / (w686 + w658),"Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S.; Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery, Optics Express, 2009, 17, 11, 1-13."
BGA_Be162B643sub629,Phycocyanin (BGA_PC),w644 - w629,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be162B700sub601,Phycocyanin (BGA_PC),w700 - w601,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be162BsubPhy,Phycocyanin (BGA_PC),w715 - w615,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540."
BGA_Be16FLHBlueRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w458 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be16FLHGreenRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w558 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be16FLHVioletRedNIR,Phycocyanin (BGA_PC),w658 - (w857 + (w444 - w857)),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538."
BGA_Be16MPI,Phycocyanin (BGA_PC),(w615 - w601) - (w644 - w601),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539."
BGA_Be16NDPhyI,Phycocyanin (BGA_PC),(w700 - w622) / (w700 + w622),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540."
BGA_Be16NDPhyI644over615,Phycocyanin (BGA_PC),(w644 - w615) / (w644 + w615),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 541."
BGA_Be16NDPhyI644over629,Phycocyanin (BGA_PC),(w644 - w629) / (w644 + w629),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 542."
BGA_Be16Phy2BDA644over629,Phycocyanin (BGA_PC),w644 / w629,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 545."
BGA_Da052BDA,Phycocyanin (BGA_PC),w714 / w672,"Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672."
BGA_Go04MCI,Phycocyanin (BGA_PC),w709 - w681 - (w753 - w681),"Gower, J.F.R.; Brown,L.; Borstad, G.A.; Observation of chlorophyll fluorescence in west coast waters of Canada using the MODIS satellite sensor. Can. J. Remote Sens., 2004, 30 (1), 17<31><37>?5."
BGA_HU103BDA,Phycocyanin (BGA_PC),(((1 / w615) - (1 / w600)) - w725),"Hunter, P.D.; Tyler, A.N.; Willby, N.J.; Gilvear, D.J.; The spatial dynamics of vertical migration by Microcystis aeruginosa in a eutrophic shallow lake: A case study using high spatial resolution time-series airborne remote sensing. Limn. Oceanogr. 2008, 53, 2391-2406"
BGA_Ku15PhyCI,Phycocyanin (BGA_PC),-1 * (W681 - W665 - (W709 - W665)),"Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-10."
BGA_Ku15SLH,Phycocyanin (BGA_PC),(w715 - w658) + (w715 - w658),"Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-11."
BGA_MI092BDA,Phycocyanin (BGA_PC),w700 / w600,"Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758<35><38>?75."
BGA_MM092BDA,Phycocyanin (BGA_PC),w724 / w600,"Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758<35><38>?76."
BGA_MM12NDCIalt,Phycocyanin (BGA_PC),(w700 - w658) / (w700 + w658),"Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114003"
BGA_MM143BDAopt,Phycocyanin (BGA_PC),((1 / w629) - (1 / w659)) * w724,"Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114004"
BGA_SI052BDA,Phycocyanin (BGA_PC),w709 / w620,"Simis, S. G. H.; Peters, S.W. M.; Gons, H. J.; Remote sensing of the cyanobacteria pigment phycocyanin in turbid inland water. Limn. Oceanogr., 2005, 50, 237<33><37>?45"
BGA_SM122BDA,Phycocyanin (BGA_PC),w709 / w600,"Mishra, S. Remote sensing of cyanobacteria in turbid productive waters, PhD Dissertation. Mississippi State University, USA. 2012."
BGA_SY002BDA,Phycocyanin (BGA_PC),w650 / w625,"Schalles, J.; Yacobi, Y. Remote detection and seasonal patterns of phycocyanin, carotenoid and chlorophyll-a pigments in eutrophic waters. Archiv fur Hydrobiologie, Special Issues Advances in Limnology, 2000, 55,153<35><33>?68"
BGA_Wy08CI,Phycocyanin (BGA_PC),-1 * (W686 - W672 - (W715 - W672)),"Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672."
Chl_Al10SABI,chlorophyll_a,(w857 - w644) / (w458 + w529),"Alawadi, F. Detection of surface algal blooms using the newly developed algorithm surface algal bloom index (SABI). Proc. SPIE 2010, 7825."
Chl_Am092Bsub,chlorophyll_a,w681 - w665,"Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S. Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery. Opt. Express 2009, 17, 9126<32><36>?144."
Chl_Be16FLHblue,chlorophyll_a,w529 - (w644 + (w458 - w644)),"Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30."
Chl_Be16FLHviolet,chlorophyll_a,w529 - (w644 + (w429 - w644)),"Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30."
Chl_Be16NDTIblue,chlorophyll_a,(w658 - w458) / (w658 + w458),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 543."
Chl_Be16NDTIviolet,chlorophyll_a,(w658 - w444) / (w658 + w444),"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 544."
Chl_De933BDA,chlorophyll_a,w600 - w648 - w625,"Dekker, A.; Detection of the optical water quality parameters for eutrophic waters by high resolution remote sensing, Ph.D. thesis, 1993, Free University, Amsterdam."
Chl_Gi033BDA,chlorophyll_a,((1 / w672) - (1 / w715)) * w757,"Gitelson, A.A.; U. Gritz, and M. N. Merzlyak.; Relationships between leaf chlorophyll content and spectral reflectance and algorithms for non-destructive chlorophyll assessment in higher plant leaves. J. Plant Phys. 2003, 160, 271-282."
Chl_Kn07KIVU,chlorophyll_a,(w458 - w644) / w529,"Kneubuhler, M.; Frank T.; Kellenberger, T.W; Pasche N.; Schmid M.; Mapping chlorophyll-a in Lake Kivu with remote sensing methods. 2007, Proceedings of the Envisat Symposium 2007, Montreux, Switzerland 23<32><33>?7 April 2007 (ESA SP-636, July 2007)."
Chl_MM12NDCI,chlorophyll_a,(w715 - w686) / (w715 + w686),"Mishra, S.; and Mishra, D.R. Normalized difference chlorophyll index: A novel model for remote estimation of chlorophyll-a concentration in turbid productive waters, Remote Sens. Environ., 2012, 117, 394-406"
Chl_Zh10FLH,chlorophyll_a,w686 - (w715 + (w672 - w751)),"Zhao, D.Z.; Xing, X.G.; Liu, Y.G.; Yang, J.H.; Wang, L. The relation of chlorophyll-a concentration with the reflectance peak near 700 nm in algae-dominated waters and sensitivity of fluorescence algorithms for detecting algal bloom. Int. J. Remote Sens. 2010, 31, 39-48"
Turb_Be16GreenPlusRedBothOverViolet,Turbidity,(w558 + w658) / w444,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538"
Turb_Be16RedOverViolet,Turbidity,w658 / w444,"Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539"
Turb_Bow06RedOverGreen,Turbidity,w658 / w558,"Bowers, D. G., and C. E. Binding. 2006. 鈥淭he Optical Properties of Mineral Suspended Particles: A Review and Synthesis.<2E><>?Estuarine Coastal and Shelf Science 67 (1<><31>?): 219<31><39>?30. doi:10.1016/j.ecss.2005.11.010"
Turb_Chip09NIROverGreen,Turbidity,w857 / w558,"Chipman, J. W.; Olmanson, L.G.; Gitelson, A.A.; Remote sensing methods for lake management: A guide for resource managers and decision-makers. 2009."
Turb_Dox02NIRoverRed,Turbidity,w857 / w658,"Doxaran, D., Froidefond, J.-M.; Castaing, P. ; A reflectance band ratio used to estimate suspended matter concentrations in sediment-dominated coastal waters, Remote Sens., 2002, 23, 5079-5085"
Turb_Frohn09GreenPlusRedBothOverBlue,Turbidity,(w558 + w658) / w458,"Frohn, R. C., & Autrey, B. C. (2009). Water quality assessment in the Ohio River using new indices for turbidity and chlorophyll-a with Landsat-7 Imagery. Draft Internal Report, US Environmental Protection Agency."
Turb_Harr92NIR,Turbidity,w857,"Schiebe F.R., Harrington J.A., Ritchie J.C. Remote-Sensing of Suspended Sediments鈥攖he Lake Chicot, Arkansas Project. Int. J. Remote Sens. 1992;13:1487<38><37>?509"
Turb_Lath91RedOverBlue,Turbidity,w658 / w458,"Lathrop, R. G., Jr., T. M. Lillesand, and B. S. Yandell, 1991. Testing the utility of simple multi-date Thematic Mapper calibration algorithms for monitoring turbid inland waters. International Journal of Remote Sensing"
Turb_Moore80Red,Turbidity,w658,"Moore, G.K., Satellite remote sensing of water turbidity, Hydrological Sciences, 1980, 25, 4, 407-422"
1 Formula_Name Category Formula Reference
2 BGA_Am09KBBI Phycocyanin (BGA_PC) (w686 - w658) / (w686 + w658) Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S.; Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery, Optics Express, 2009, 17, 11, 1-13.
3 BGA_Be162B643sub629 Phycocyanin (BGA_PC) w644 - w629 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
4 BGA_Be162B700sub601 Phycocyanin (BGA_PC) w700 - w601 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
5 BGA_Be162BsubPhy Phycocyanin (BGA_PC) w715 - w615 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540.
6 BGA_Be16FLHBlueRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w458 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
7 BGA_Be16FLHGreenRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w558 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
8 BGA_Be16FLHVioletRedNIR Phycocyanin (BGA_PC) w658 - (w857 + (w444 - w857)) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538.
9 BGA_Be16MPI Phycocyanin (BGA_PC) (w615 - w601) - (w644 - w601) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539.
10 BGA_Be16NDPhyI Phycocyanin (BGA_PC) (w700 - w622) / (w700 + w622) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 540.
11 BGA_Be16NDPhyI644over615 Phycocyanin (BGA_PC) (w644 - w615) / (w644 + w615) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 541.
12 BGA_Be16NDPhyI644over629 Phycocyanin (BGA_PC) (w644 - w629) / (w644 + w629) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 542.
13 BGA_Be16Phy2BDA644over629 Phycocyanin (BGA_PC) w644 / w629 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 545.
14 BGA_Da052BDA Phycocyanin (BGA_PC) w714 / w672 Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672.
15 BGA_Go04MCI Phycocyanin (BGA_PC) w709 - w681 - (w753 - w681) Gower, J.F.R.; Brown,L.; Borstad, G.A.; Observation of chlorophyll fluorescence in west coast waters of Canada using the MODIS satellite sensor. Can. J. Remote Sens., 2004, 30 (1), 17?5.
16 BGA_HU103BDA Phycocyanin (BGA_PC) (((1 / w615) - (1 / w600)) - w725) Hunter, P.D.; Tyler, A.N.; Willby, N.J.; Gilvear, D.J.; The spatial dynamics of vertical migration by Microcystis aeruginosa in a eutrophic shallow lake: A case study using high spatial resolution time-series airborne remote sensing. Limn. Oceanogr. 2008, 53, 2391-2406
17 BGA_Ku15PhyCI Phycocyanin (BGA_PC) -1 * (W681 - W665 - (W709 - W665)) Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-10.
18 BGA_Ku15SLH Phycocyanin (BGA_PC) (w715 - w658) + (w715 - w658) Kudela, R.M., Palacios, S.L., Austerberry, D.C., Accorsi, E.K., Guild, L.S.; Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters, Torres-Perez, J., 2015, Remote Sens. Environ., 2015, 167, 1-11.
19 BGA_MI092BDA Phycocyanin (BGA_PC) w700 / w600 Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758?75.
20 BGA_MM092BDA Phycocyanin (BGA_PC) w724 / w600 Mishra, S.; Mishra, D.R.; Schluchter, W. M., A novel algorithm for predicting PC concentrations in cyanobacteria: A proximal hyperspectral remote sensing approach. Remote Sens., 2009, 1, 758?76.
21 BGA_MM12NDCIalt Phycocyanin (BGA_PC) (w700 - w658) / (w700 + w658) Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114003
22 BGA_MM143BDAopt Phycocyanin (BGA_PC) ((1 / w629) - (1 / w659)) * w724 Mishra, S.; Mishra, D.R.; A novel remote sensing algorithm to quantify phycocyanin in cyanobacterial algal blooms, Env. Res. Lett., 2014, 9 (11), DOI:10.1088/1748-9326/9/11/114004
23 BGA_SI052BDA Phycocyanin (BGA_PC) w709 / w620 Simis, S. G. H.; Peters, S.W. M.; Gons, H. J.; Remote sensing of the cyanobacteria pigment phycocyanin in turbid inland water. Limn. Oceanogr., 2005, 50, 237?45
24 BGA_SM122BDA Phycocyanin (BGA_PC) w709 / w600 Mishra, S. Remote sensing of cyanobacteria in turbid productive waters, PhD Dissertation. Mississippi State University, USA. 2012.
25 BGA_SY002BDA Phycocyanin (BGA_PC) w650 / w625 Schalles, J.; Yacobi, Y. Remote detection and seasonal patterns of phycocyanin, carotenoid and chlorophyll-a pigments in eutrophic waters. Archiv fur Hydrobiologie, Special Issues Advances in Limnology, 2000, 55,153?68
26 BGA_Wy08CI Phycocyanin (BGA_PC) -1 * (W686 - W672 - (W715 - W672)) Wynne, T. T., Stumpf, R. P., Tomlinson, M. C., Warner, R. A., Tester, P. A., Dyble, J.; Relating spectral shape to cyanobacterial blooms in the Laurentian Great Lakes. Int. J. Remote Sens., 2008, 29, 3665-3672.
27 Chl_Al10SABI chlorophyll_a (w857 - w644) / (w458 + w529) Alawadi, F. Detection of surface algal blooms using the newly developed algorithm surface algal bloom index (SABI). Proc. SPIE 2010, 7825.
28 Chl_Am092Bsub chlorophyll_a w681 - w665 Amin, R.; Zhou, J.; Gilerson, A.; Gross, B.; Moshary, F.; Ahmed, S. Novel optical techniques for detecting and classifying toxic dinoflagellate Karenia brevis blooms using satellite imagery. Opt. Express 2009, 17, 9126?144.
29 Chl_Be16FLHblue chlorophyll_a w529 - (w644 + (w458 - w644)) Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30.
30 Chl_Be16FLHviolet chlorophyll_a w529 - (w644 + (w429 - w644)) Beck, R.A. and 22 others; Comparison of satellite reflectance algorithms for estimating chlorophyll-a in a temperate reservoir using coincident hyperspectral aircraft imagery and dense coincident surface observations, Remote Sens. Environ., 2016, 178, 15-30.
31 Chl_Be16NDTIblue chlorophyll_a (w658 - w458) / (w658 + w458) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 543.
32 Chl_Be16NDTIviolet chlorophyll_a (w658 - w444) / (w658 + w444) Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 544.
33 Chl_De933BDA chlorophyll_a w600 - w648 - w625 Dekker, A.; Detection of the optical water quality parameters for eutrophic waters by high resolution remote sensing, Ph.D. thesis, 1993, Free University, Amsterdam.
34 Chl_Gi033BDA chlorophyll_a ((1 / w672) - (1 / w715)) * w757 Gitelson, A.A.; U. Gritz, and M. N. Merzlyak.; Relationships between leaf chlorophyll content and spectral reflectance and algorithms for non-destructive chlorophyll assessment in higher plant leaves. J. Plant Phys. 2003, 160, 271-282.
35 Chl_Kn07KIVU chlorophyll_a (w458 - w644) / w529 Kneubuhler, M.; Frank T.; Kellenberger, T.W; Pasche N.; Schmid M.; Mapping chlorophyll-a in Lake Kivu with remote sensing methods. 2007, Proceedings of the Envisat Symposium 2007, Montreux, Switzerland 23?7 April 2007 (ESA SP-636, July 2007).
36 Chl_MM12NDCI chlorophyll_a (w715 - w686) / (w715 + w686) Mishra, S.; and Mishra, D.R. Normalized difference chlorophyll index: A novel model for remote estimation of chlorophyll-a concentration in turbid productive waters, Remote Sens. Environ., 2012, 117, 394-406
37 Chl_Zh10FLH chlorophyll_a w686 - (w715 + (w672 - w751)) Zhao, D.Z.; Xing, X.G.; Liu, Y.G.; Yang, J.H.; Wang, L. The relation of chlorophyll-a concentration with the reflectance peak near 700 nm in algae-dominated waters and sensitivity of fluorescence algorithms for detecting algal bloom. Int. J. Remote Sens. 2010, 31, 39-48
38 Turb_Be16GreenPlusRedBothOverViolet Turbidity (w558 + w658) / w444 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 538
39 Turb_Be16RedOverViolet Turbidity w658 / w444 Beck, R.; Xu, M.; Zhan, S.; Liu, H.; Johansen, R.A.; Tong, S.; Yang, B.; Shu, S.; Wu, Q.; Wang, S.; Berling, K.; Murray, A.; Emery, E.; Reif, M.; Harwood, J.; Young, J.; Martin, M.; Stillings, G.; Stumpf, R.; Su, H.; Ye, Z.; Huang, Y. Comparison of Satellite Reflectance Algorithms for Estimating Phycocyanin Values and Cyanobacterial Total Biovolume in a Temperate Reservoir Using Coincident Hyperspectral Aircraft Imagery and Dense Coincident Surface Observations. Remote Sens. 2017, 9, 539
40 Turb_Bow06RedOverGreen Turbidity w658 / w558 Bowers, D. G., and C. E. Binding. 2006. 鈥淭he Optical Properties of Mineral Suspended Particles: A Review and Synthesis.?Estuarine Coastal and Shelf Science 67 (1?): 219?30. doi:10.1016/j.ecss.2005.11.010
41 Turb_Chip09NIROverGreen Turbidity w857 / w558 Chipman, J. W.; Olmanson, L.G.; Gitelson, A.A.; Remote sensing methods for lake management: A guide for resource managers and decision-makers. 2009.
42 Turb_Dox02NIRoverRed Turbidity w857 / w658 Doxaran, D., Froidefond, J.-M.; Castaing, P. ; A reflectance band ratio used to estimate suspended matter concentrations in sediment-dominated coastal waters, Remote Sens., 2002, 23, 5079-5085
43 Turb_Frohn09GreenPlusRedBothOverBlue Turbidity (w558 + w658) / w458 Frohn, R. C., & Autrey, B. C. (2009). Water quality assessment in the Ohio River using new indices for turbidity and chlorophyll-a with Landsat-7 Imagery. Draft Internal Report, US Environmental Protection Agency.
44 Turb_Harr92NIR Turbidity w857 Schiebe F.R., Harrington J.A., Ritchie J.C. Remote-Sensing of Suspended Sediments鈥攖he Lake Chicot, Arkansas Project. Int. J. Remote Sens. 1992;13:1487?509
45 Turb_Lath91RedOverBlue Turbidity w658 / w458 Lathrop, R. G., Jr., T. M. Lillesand, and B. S. Yandell, 1991. Testing the utility of simple multi-date Thematic Mapper calibration algorithms for monitoring turbid inland waters. International Journal of Remote Sensing
46 Turb_Moore80Red Turbidity w658 Moore, G.K., Satellite remote sensing of water turbidity, Hydrological Sciences, 1980, 25, 4, 407-422

85
data/格式转化.py Normal file
View File

@ -0,0 +1,85 @@
import os
from pathlib import Path
from PIL import Image
def batch_convert_to_ico(source_dirs, output_dir, target_size=(256, 256)):
"""
批量将指定目录下的图像文件转换为 ICO 格式。
:param source_dirs: 包含源文件夹路径的列表
:param output_dir: 转换后 ICO 文件的保存目录
:param target_size: 输出 ICO 的尺寸,默认 256x256
"""
# 支持的常见输入图像后缀
supported_extensions = {'.png', '.jpg', '.jpeg', '.bmp', '.webp', '.tiff'}
# 确保输出目录存在,若无则自动创建
out_path = Path(output_dir)
out_path.mkdir(parents=True, exist_ok=True)
total_converted = 0
total_failed = 0
print("=" * 50)
print(f"🚀 开始批量转换 ICO 图标...")
print(f"📁 目标输出目录: {out_path}")
print("=" * 50)
# 遍历所有传入的源目录
for folder in source_dirs:
folder_path = Path(folder)
if not folder_path.exists():
print(f"⚠️ 警告: 源目录不存在,已跳过 -> {folder_path}")
continue
print(f"\n📂 正在扫描目录: {folder_path}")
# 遍历目录下的所有文件
for file_path in folder_path.iterdir():
# 仅处理普通文件且后缀在支持列表内(忽略大小写)
if file_path.is_file() and file_path.suffix.lower() in supported_extensions:
try:
with Image.open(file_path) as img:
# 处理透明通道问题:
# 如果图片支持透明通道 (RGBA/P/LA),转为 RGBA 确保透明背景不丢失
# 如果是普通 RGB (如 JPG),转为 RGB
if img.mode in ('RGBA', 'LA') or (img.mode == 'P' and 'transparency' in img.info):
img_clean = img.convert('RGBA')
else:
img_clean = img.convert('RGB')
# 构造输出文件名 (原文件名.ico)
new_filename = f"{file_path.stem}.ico"
save_path = out_path / new_filename
# 如果目标文件夹中已存在同名文件,为了防止覆盖,可以在文件名后加个标识
# 但通常图标库同名直接覆盖较符合需求,这里默认直接保存
img_clean.save(save_path, format="ICO", sizes=[target_size])
print(f" ✅ 成功: {file_path.name} -> {new_filename}")
total_converted += 1
except Exception as e:
print(f" ❌ 失败: 无法转换 {file_path.name},错误信息: {e}")
total_failed += 1
print("\n" + "=" * 50)
print("🎉 转换任务结束!")
print(f"统计: 成功转换 {total_converted} 个文件,失败 {total_failed} 个。")
print("=" * 50)
if __name__ == "__main__":
# 1. 定义你要读取的两个源文件夹路径列表
SOURCES = [
r"D:\111\office\ZHLduijie\1.WQ\WQ_GUI\data\icons",
r"D:\111\office\ZHLduijie\1.WQ\WQ_GUI\data\icons\word"
]
# 2. 定义统一输出的目标文件夹路径
OUTPUT = r"D:\111\office\ZHLduijie\1.WQ\WQ_GUI\data\icons-1"
# 执行转换
batch_convert_to_ico(SOURCES, OUTPUT)

View File

@ -0,0 +1,350 @@
# Smoke Test — 路线 B MVPPipelineContext + AutoML + 软取消 + GUI 缝合)
> 适用范围:路线 B 重构 4 部分pipeline 包 / AutoML 训练器 / WorkerThread 软取消 / GUI 一键全自动)落盘后的端到端点火试飞清单。
> 目标:**用最小数据集1 个 BSQ + 1 个 CSV在 1020 分钟内验证全链路打通**。
---
## 0. 前置准备5 分钟)
### 0.1 装 Optuna
`environment.yml` 当前**未列** optuna属于本次重构新增依赖。若不装Step 6 会自动降级到老 GridSearchCV仍能跑通但会触发 fallback 日志)。
```bash
call venv\Scripts\activate.bat
pip install "optuna>=3.6,<4.0"
```
写入 `environment.yml` 的 patch提交时改
```yaml
# 路线 B AutoML 防爆引擎(可选;未装时 Step 6 走老 GridSearchCV 降级路径)
- optuna>=3.6
```
### 0.2 准备最小数据集
```text
work_dir_smoke/
├── raw/
│ ├── sample.b # 假彩色 BSQ任意小分辨率都行建议 50×50×6 波段)
│ ├── sample_mask.tif # (可选)水域掩膜;不提供则 Step 1 自动生成 NDWI
│ └── sample.csv # 含 36 个水质参数目标列Chl-a / TSS / SD / TN / TP / COD…+ 6 列波段反射率
└── (其他文件由流程自动生成)
```
**CSV 模板示例**`feature_start_column` 默认为第一列;目标列必须**在特征列之前**
```csv
Chl-a,TSS,SD,B1,B2,B3,B4,B5,B6
12.3,15.1,0.8,0.045,0.052,0.038,0.061,0.072,0.085
11.8,14.2,0.9,0.044,0.051,0.037,0.060,0.071,0.084
... (≥ 200 行AutoML 智能子采样 N>5000 时才生效)
```
### 0.3 启动 venv
```bash
cd /d "D:\111\office\ZHLduijie\1.WQ\WQ_GUI"
call venv\Scripts\activate.bat
set PYTHONPATH=src;%PYTHONPATH%
```
---
## 1. CLI 烟雾最快路径3 分钟)— **A 级:必跑**
跳过 GUI直接验证 `automl_trainer.py` 自身可独立运行 + Optuna 子采样 + 降级路径:
```bash
python -m src.core.prediction.automl_trainer ^
--csv work_dir_smoke/raw/sample.csv ^
--feature-start 6 ^
--n-trials 5 ^
--timeout 60.0 ^
--out work_dir_smoke/7_Supervised_Model_Training_AutoML
```
**通过标准**
- [ ] 进程退出码 0
- [ ] 控制台打印 `AutoML: 目标列 X 共尝试 N 个 trial最佳 CV R²=…`
- [ ] `<out>/<preprocess>/<target>_<preprocess>_<model>_AUTOML.joblib` 存在
- [ ] `<out>/automl_summary.json` 存在且 `success=true`
**若 Optuna 未装**,期待看到:
```
[AutoML] optuna 未安装,全目标列回退老 GridSearchCV
```
产物文件名带 `_AUTOML` 后缀的逻辑此时**不会触发**fallback 走老路径),属正常。
---
## 2. GUI 端到端 9 步核心场景1020 分钟)— **S 级:必跑**
### 2.1 启动 GUI
```bash
call venv\Scripts\activate.bat
set PYTHONPATH=src;%PYTHONPATH%
python -m src.gui.water_quality_gui
```
### 2.2 UI 配置
| 步骤 | 操作 | 期望 |
| ----- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| 1/9 | 点"选择工作目录" → 选 `work_dir_smoke/` | 左侧步骤列表高亮UI 不报错 |
| 2/9 | 在 Step 1 面板选 `sample.b`**掩膜留空**(验证 NDWI 自动生成路径) | 掩膜文本框保持空白 |
| 3/9 | 在 Step 4 面板选 `sample.csv` | CSV 路径显示正确 |
| 4/9 | **关键**其他步骤2/3/5/5.5/6/7/8/9保持默认不改任何参数 | AutoML 默认开启use_automl=True |
| 5/9 | 点 **▶ 运行完整流程**(不要用老 `run_full_pipeline` 槽) | 弹出**二次确认窗**,文案显示:<br>• 掩膜:`未指定(将自动生成 NDWI 水域掩膜)`<br>• 去耀斑:开启<br>• AutoML开启Optuna 子采样寻优) |
| 6/9 | 点"是(Y)" | "运行"按钮变灰,"停止"按钮亮起;进度条归零 |
### 2.3 观察日志(重点 4 大检查点)
#### ✅ 检查点 1ctx 路径传递
启动后**第一秒**应看到类似:
```
[Runner] ctx 已构造14 路径字段4 目录字段
[Runner] 步骤 1/14step1_generate_water_maskrequires=['raw_img_path', 'water_mask_path']
[Runner] 步骤 2/14step2_find_glint_arearequires=['raw_img_path', 'water_mask_path', 'output_dir']
...
[Runner] ctx 路径校准water_mask_path = ...\work_dir_smoke\2_Glint_Area_Mask\glint_mask.tif
```
**若没有 `[Runner]` 日志**,说明 v1 旧路径被走到了,**`inspect.signature` duck-type 没探测到 v2**,回去检查 `worker_thread.py:run()`
#### ✅ 检查点 2Step 1 NDWI 自动生成
```
[Step1] 未指定 mask_path自动基于 NDWI 生成水域掩膜
[Step1] NDWI 阈值=0.4,写入 1_Water_Mask/water_mask.tif
```
→ 验证 `<work_dir>/1_Water_Mask/water_mask.tif` 文件存在且非空。
#### ✅ 检查点 3AutoML 启用
```
[Step6] AutoML 启用 Optuna 子采样寻优timeout=300s, n_trials=20, max_samples=5000
[Step6] 目标列 'Chl-a' 共 3 个候选模型,最佳 R²=0.812model=RandomForest
[Step6] 目标列 'TSS' 共 3 个候选模型,最佳 R²=0.745model=XGBoost
[Step6] 训练完成,产物写入 7_Supervised_Model_Training_AutoML/
[Step6] automl_summary.json 写入完成
```
→ 验证产物:
- [ ] `7_Supervised_Model_Training_AutoML/<preprocess>/<target>_<preprocess>_<model>_AUTOML.joblib` ≥ 1 个
- [ ] `7_Supervised_Model_Training_AutoML/automl_summary.json``automl: true` 字段
- [ ] 老目录 `7_Supervised_Model_Training/` **不应该被创建**AutoML 路径独立)
#### ✅ 检查点 4AutoML 降级(仅未装 Optuna 时)
```
[AutoML] optuna 未安装,全目标列回退老 GridSearchCV
[Step6] 降级路径:调用 WaterQualityModelingBatch.train_models_batch132 组 GridSearchCV
```
→ 跑通即可(仍能产生模型文件),但**降级**属于非优选路径。
### 2.4 9 步全程观察清单
| 步 | 期望产物(路径相对 `work_dir` | 期望耗时50×50 测试数据) |
| ---- | -------------------------------------------------------------- | -------------------------- |
| 1 | `1_Water_Mask/water_mask.tif` | < 5 s |
| 2 | `2_Glint_Area_Mask/glint_mask.tif` | < 5 s |
| 3 | `3_Remove_Glint_Image/deglint_image.tif` | < 5 s |
| 4 | `4_Process_CSV/processed_data.csv` | < 2 s |
| 5 | `5_Training_Sample/training_spectra.csv` | < 5 s |
| 5.5 | `5_5_Calculate_Indices/indices.csv`如启用 | < 2 s |
| **6**| `7_Supervised_Model_Training_AutoML/`**新路径** | **< 5 minOptuna 5 trial** |
| 6.5 | `6_5_Non_Empirical_Modeling/`如启用 | 12 min |
| 6.75 | `6_75_Custom_Regression/`如启用 | 12 min |
| 7 | `7_Sampling_Points/sampling_points.csv` | < 3 s |
| 8 | `8_Prediction/predicted_values.csv` | < 5 s |
| 8.5 | `8_5_Prediction_Non_Empirical/predicted.csv`如启用 | < 5 s |
| 8.75 | `8_75_Prediction_Custom/predicted.csv`如启用 | < 5 s |
| 9 | `9_Kriging_Distribution_Map/distribution_map.tif` | 530 s Python |
### 2.5 流程结束
- [ ] 进度条到 100%
- [ ] "运行"按钮恢复可点
- [ ] "停止"按钮变灰
- [ ] 日志末行出现 `=== 流程执行完成 ===` `=== 流程被取消 ===`取决于是否点过停止
- [ ] 控制台 `on_pipeline_finished` 触发UI 状态被统一恢复
---
## 3. 软取消测试3 分钟)— **A 级:必跑**
验证 `threading.Event` 软取消链路不再用 `terminate()`)。
### 3.1 启动完整流程
2.2 启动流程
### 3.2 中途点"停止"
**时机** Step 6 AutoML trials 的中途看到 `[Step6] 目标列 'Chl-a' 共 N 个候选模型` 之后任意时刻"停止"。
**期望看到**
```
[STOP] 用户请求软取消
[Step6] 检测到 cancel_event本 trial 完成后退出
[Step6] AutoML 在 trial #X 中止,已完成 5/20 trial
[Runner] 软取消:跳过剩余 8 个 step
=== 流程被取消 ===
```
UI 状态
- [ ] "运行"按钮重新亮起
- [ ] "停止"按钮变灰
- [ ] 进度条保留在中断时的百分比****归零
- [ ] `on_pipeline_finished` 触发 `success=False, cancelled=True` 区分
- [ ] **Python 进程不退出**GUI 仍可继续点"运行"开新流程
**反例(不应该发生)**
- `QThread: Destroyed while thread is still running` 警告
- Python 解释器直接崩溃
- UI 永远卡死`run_all_btn` 一直是灰的
### 3.3 旧 `stop()` 路径回归
为防老代码忘了改临时把 `water_quality_gui.py:stop_pipeline` 改回 `self.worker.stop()`跑一次完整流程看是否出现
```
[DEPRECATED] WorkerThread.stop() 已弃用,请改用 soft_stop()。
```
**这是预期行为**弃用方法保留但打 warning流程仍能完成即视为通过
---
## 4. 失败 / 降级场景5 分钟)— **B 级:选跑**
### 4.1 未填掩膜 + NDWI 阈值设极端值
NDWI 阈值设到 `0.9`几乎无水域Step 1 应给出 warning 但不崩
```
[Step1] NDWI 阈值=0.9,水域覆盖率 < 1%,请检查影像
```
### 4.2 CSV 完全无目标列
准备一个**没有目标列的 CSV**全特征列点运行
```
[AutoML] 训练 CSV 不存在或无目标列:未识别出目标列
[Step6] AutoML 全部失败,所有目标列返回 success=False
```
UI 不会崩会在 `automl_summary.json` `error: "未识别出目标列"`
### 4.3 Step 1 路径不存在
Step 1 选了一个**不存在的 .bsq 文件**
```
[Runner] step1_generate_water_mask 异常FileNotFoundError
[STOP] 流程中止在 step 1
```
UI 弹错误窗 + 把左侧步骤列表 `setCurrentRow(0)` 自动定位到 Step 1`_focus_step` 起效)。
### 4.4 Optuna 版本冲突
装一个 `optuna==2.10`API 大改 GUI
```
[AutoML] optuna API 不兼容(>=3.6 要求):<error>
[AutoML] 全目标列回退老 GridSearchCV
```
降级路径生效即视为通过
---
## 5. 验证矩阵 Checklist
复制以下到 PR 描述 / 验收单
```markdown
## 路线 B MVP 验证矩阵
### 代码落盘
- [ ] src/core/pipeline/__init__.py17 行4 export
- [ ] src/core/pipeline/context.pyPipelineContext dataclass
- [ ] src/core/pipeline/runner.pyStepSpec + PIPELINE_STEPS + PipelineRunner
- [ ] src/core/prediction/__init__.py追加 train_with_automl export
- [ ] src/core/prediction/automl_trainer.pyAutoMLResult + train_with_automl + CLI
- [ ] src/core/steps/modeling_step.pyuse_automl 分支 + _train_models_automl
- [ ] src/core/water_quality_inversion_pipeline_GUI.pyrun_full_pipeline_v2 + LEGACY_ATTR_MAP + _sync_legacy_attrs_from_context
- [ ] src/gui/core/worker_thread.pycancel_event + soft_stop + run() duck-type
- [ ] src/gui/water_quality_gui.pyon_run_all_clicked + _collect_minimal_config + 按钮重连)
### CLI 自测
- [ ] A.1 `python -m src.core.prediction.automl_trainer --csv ...` 退出码 0
- [ ] A.2 产物 .joblib 含 `_AUTOML` 后缀
- [ ] A.3 automl_summary.json 含 success=true
### GUI 端到端
- [ ] B.1 启动无 ImportError
- [ ] B.2 二次确认窗文案含 mask 提示 + AutoML 状态
- [ ] B.3 日志含 [Runner] 前缀v2 路径生效)
- [ ] B.4 Step 1 NDWI 自动生成路径生效
- [ ] B.5 9 步产物路径全部存在
- [ ] B.6 流程结束后 UI 状态恢复(运行按钮亮、停止按钮灰)
### 软取消
- [ ] C.1 流程中途点停止cancel_event 触发
- [ ] C.2 流程被取消而非崩溃
- [ ] C.3 UI 状态由 on_pipeline_finished 统一恢复
- [ ] C.4 旧 stop() 调用打 [DEPRECATED] warning
### 降级
- [ ] D.1 Optuna 未装 → 全目标列回退老 GridSearchCV
- [ ] D.2 无目标列 CSV → 写 error 到 summary不崩 UI
- [ ] D.3 不存在文件 → _focus_step 定位到对应 step
```
---
## 6. 已知未做(不在本次范围)
- [ ] Kriging 多进程并行当前 backend="loop" Python
- [ ] Step 5 radius==0 内存优化整波段读入
- [ ] 进度条 sub-step 粒度当前只到 step
- [ ] Step 8 全图预测当前只对采样点预测
- [ ] 全项目搜替换老 `self.worker.stop()` 调用仅本会话改了 `water_quality_gui.py` stop_pipeline
- [ ] `requirements.txt` 同步 Optuna `environment.yml`
- [ ] 单元测试套件`tests/` 目录为空建议用 pytest 覆盖 train_with_automl / PipelineRunner
---
## 7. 出问题找哪里
| 现象 | 看哪里 |
| --------------------------------------------- | ------------------------------------------------------- |
| `[Runner]` 日志没出来 | `worker_thread.py:run()` `inspect.signature` 探测 |
| `[AutoML]` 完全没打 | `modeling_step.py:170` `if use_automl` 是否进了 |
| AutoML `optuna API 不兼容` | `automl_trainer.py:236` `try import` |
| 软取消无反应 | `worker_thread.py:run()` 末尾的 `cancel_event.is_set()` |
| 二次确认窗没出来 | `water_quality_gui.py:on_run_all_clicked` line ~2848 |
| 9 步产物路径错位 | `pipeline/runner.py:PIPELINE_STEPS` `output` 字段 |
| v1 路径被走到 | `_sync_legacy_attrs_from_context` 没调 v2 异常 |
---
> **作者注**:本清单对应**路线 B 一键全自动重构 4 部分全部落盘**的验收场景,编号与 todo 8 同步。
> 跑通 §1 + §2 + §3 三段即视为 MVP 验收通过§4 用于鲁棒性抽查。

8
license.lic Normal file
View File

@ -0,0 +1,8 @@
{
"version": "1.0",
"product": "WaterQualityInversion",
"machine_code": "76E4992A5CF08BA570D6150908E04755",
"generated_at": "2026-05-28 14:21:35",
"expiry": "2099-12-31",
"signature": "DC9AB900D7033A281E54F41F3F76D026FFA75D635484D40C7F6FC1F6023E02AB"
}

6
run_smoke.bat Normal file
View File

@ -0,0 +1,6 @@
@echo off
cd /d "D:\111\office\ZHLduijie\1.WQ\WQ_GUI"
call venv\Scripts\activate.bat
set PYTHONPATH=new\app\api;%PYTHONPATH%
python -c "import _smoke_test_train; _smoke_test_train.test_load_train_df(); _smoke_test_train.test_get_model_pipeline_all_types(); _smoke_test_train.test_run_train_sync_linearregression_fast(); _smoke_test_train.test_run_train_sync_bad_csv(); _smoke_test_train.test_run_train_sync_bad_target(); print('OK')" > %TEMP%\smoke_log.txt 2>&1
type %TEMP%\smoke_log.txt

View File

@ -0,0 +1,14 @@
# -*- coding: utf-8 -*-
"""
Pipeline 调度核心:基于 Context 的内存级依赖注入。
设计目标:
- 用 PipelineContext 替代 dict 散落传参9 步主路径 + 14 个 step 共享同一份 ctx
- 14 个 step 声明式描述StepSpec便于 Web / 异步 / 单元测试复用
- 不绑定具体 Pipeline 实现duck-typedWorkerThread / Web API / 单测可共用
"""
from .context import PipelineContext
from .runner import StepSpec, PIPELINE_STEPS, PipelineRunner
__all__ = ["PipelineContext", "StepSpec", "PIPELINE_STEPS", "PipelineRunner"]

View File

@ -0,0 +1,100 @@
# -*- coding: utf-8 -*-
"""
PipelineContext内存级数据载体跨 14 个 step 传递路径与元信息。
设计原则:
- 所有路径字段以 `_path` 为后缀(与 step 方法形参命名约定一致)
- 字段值可缺省None由 StepSpec.requires 在调度时注入
- dataclass + field(default_factory=dict) 支持原地增删
- 不放 GUI 状态(避免循环依赖)
- 不绑具体 step 方法duck-typed cancellation / log append
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
@dataclass
class PipelineContext:
"""流水线运行上下文(在 14 个 step 之间传递的内存字典)
字段命名约定:
- 路径类字段名 = panel key 名 = step 形参名(全链路无翻译)
- 训练/产物 CSV 用 `_path` 后缀(如 training_csv_path / water_mask_path
- 入参影像/CSV 沿用 panel 原名img_path / csv_path无 `_path` 后缀
- 目录类字段无 `_path` 后缀(如 models_dir / prediction_dir
- 元信息字段无后缀(如 user_config / status / log
"""
# ── 11 个 step 的入参/产物(按 step 顺序排列;字段名 = panel key = step 形参) ──
img_path: Optional[str] = None # Step 1/2/3 入参:原始影像
water_mask_path: Optional[str] = None # Step 1 出 → Step 2/3/7 入
glint_mask_path: Optional[str] = None # Step 2 出 → Step 3/7 入
deglint_img_path: Optional[str] = None # Step 3 出 → Step 5/7 入
csv_path: Optional[str] = None # Step 4/5/6_5/6_75 入参:原始/训练 CSV
processed_csv_path: Optional[str] = None # Step 4 出 → Step 5 入
training_csv_path: Optional[str] = None # Step 5 出 → Step 5_5/6/6_5/6_75 入
boundary_path: Optional[str] = None # Step 5 入参:边界 SHPpanel step5 名)
indices_path: Optional[str] = None # Step 5.5 出
sampling_csv_path: Optional[str] = None # Step 7 出 → Step 8/8_5/8_75/9 入
prediction_csv_path: Optional[str] = None # Step 8 出 → Step 9 入
distribution_map_path: Optional[str] = None # Step 9 出
boundary_shp_path: Optional[str] = None # Step 9 入参:边界 SHPpanel step9 名)
formula_csv_path: Optional[str] = None # Step 8_75 入参:公式 CSV
# ── 目录类(命名不带 _path 以示区别) ──
models_dir: Optional[str] = None
prediction_dir: Optional[str] = None
work_dir: Optional[str] = None
# ── Step 6 训练产物AutoML 模式有,常规模式为空) ──
model_files: List[str] = field(default_factory=list)
# ── 元信息(三件套:用户传的配置 / 取消事件 / 状态) ──
user_config: Dict[str, Any] = field(default_factory=dict)
cancel_event: Optional[Any] = None # duck-typed threading.Event / asyncio.Event
status: Dict[str, str] = field(default_factory=dict) # {step_id: 'start'/'completed'/'skipped'/'error'}
log: List[str] = field(default_factory=list)
# ── 诊断 ──
step_timings: Dict[str, float] = field(default_factory=dict)
pipeline_start_time: Optional[float] = None
pipeline_end_time: Optional[float] = None
last_error: Optional[str] = None
# ============================================================
# 读写辅助
# ============================================================
def set(self, key: str, value: Any) -> None:
"""原地写入任意属性。
允许动态字段(如 'report_path')直接挂在 __dict__ 上,
避免因静态字段缺失而抛 AttributeError。
"""
object.__setattr__(self, key, value)
def get(self, key: str, default: Any = None) -> Any:
"""原地读出,缺 key 不抛错。"""
return getattr(self, key, default)
def is_cancelled(self) -> bool:
"""统一软取消检查入口duck-typed
支持:
- threading.Event.is_set()
- asyncio.Eventloop-boundis_set 同步接口存在)
- 自定义 .is_set() / .cancelled 属性
"""
ev = self.cancel_event
if ev is None:
return False
is_set = getattr(ev, "is_set", None)
if callable(is_set):
return bool(is_set())
return bool(getattr(ev, "cancelled", False))
def append_log(self, msg: str) -> None:
"""写入日志列表(也用于主进程 stdout 调试)。"""
self.log.append(msg)

274
src/core/pipeline/runner.py Normal file
View File

@ -0,0 +1,274 @@
# -*- coding: utf-8 -*-
"""
PipelineRunner基于 StepSpec 声明式调度 14 个 step。
设计要点:
- StepSpec 声明 requiresctx 字段名列表)+ producesctx 字段名列表)
- 命名约定ctx 字段名 == panel key 名 == step 形参名(全链路无翻译)
- 保留 spec.parameter_map 字段骨架供极少数特例覆盖(默认空 dict
- 调度顺序:按 PIPELINE_STEPS 列表顺序requires 缺则 skip
- 软取消:在每个 step 前检查 ctx.is_cancelled()
- duck-typed pipelinerunner 只调 getattr(pipeline, method_name),不强依赖类层级
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Sequence
from .context import PipelineContext
# ============================================================
# StepSpec 声明式描述
# ============================================================
@dataclass
class StepSpec:
"""单个 step 的元信息(声明式,避免硬编码)"""
step_id: str
method_name: str
requires: List[str] # PipelineContext 字段名列表
produces: List[str] = field(default_factory=list) # 写入 ctx 的字段名列表
enabled: bool = True
parameter_map: Dict[str, str] = field(default_factory=dict)
# 当 requires 中任一字段为 None 时是否跳过;默认 True缺输入就 skip
skip_when_missing: bool = True
# 备注(仅用于文档生成 / 调试输出)
description: str = ""
# ============================================================
# 14 个 step 的声明表(顺序即调度顺序)
# 注:本表是"权威描述",与 WorkerThread.step_method_map / 旧 run_full_pipeline 保持一致
# ============================================================
PIPELINE_STEPS: List[StepSpec] = [
StepSpec(
step_id="step1", method_name="step1_generate_water_mask",
requires=["img_path"], produces=["water_mask_path"],
description="水域掩膜生成NDWI 或 SHP",
),
StepSpec(
step_id="step2", method_name="step2_find_glint_area",
requires=["img_path", "water_mask_path"], produces=["glint_mask_path"],
description="耀斑区域检测",
),
StepSpec(
step_id="step3", method_name="step3_remove_glint",
requires=["img_path", "water_mask_path", "glint_mask_path"],
produces=["deglint_img_path"],
description="耀斑去除",
),
StepSpec(
step_id="step4", method_name="step4_process_csv",
requires=["csv_path"], produces=["processed_csv_path"],
description="CSV 异常值清洗",
),
StepSpec(
step_id="step5", method_name="step5_extract_training_spectra",
requires=["deglint_img_path", "processed_csv_path", "csv_path", "boundary_path", "glint_mask_path"],
produces=["training_csv_path"],
# processed_csv_path(step4 产物) 才是 step5 真正需要的主路径,
# 通过 parameter_map 显式映射到形参 csv_path。
# raw csv_path 也保留在 requires 中以备 user_config 覆盖,
# 但用占位名 _raw_csv_ignored 注入,落到 step5 形参列表末尾的 **kwargs 兜底。
# 这样可以避免 L2 顺序注入中"后注入的 csv_path=None 覆盖前面的 processed_csv_path"的冲突。
parameter_map={
"processed_csv_path": "csv_path",
"csv_path": "_raw_csv_ignored",
},
skip_when_missing=False,
description="实测样本点光谱提取",
),
StepSpec(
step_id="step5_5", method_name="step5_5_calculate_water_quality_indices",
requires=["training_csv_path"], produces=["indices_path"],
description="水质光谱指数计算optional",
),
StepSpec(
step_id="step6", method_name="step6_train_models",
requires=["training_csv_path"], produces=["models_dir"],
description="ML 建模GridSearchCV / AutoML",
),
StepSpec(
step_id="step6_5", method_name="step6_5_non_empirical_modeling",
requires=["training_csv_path"], produces=["models_dir"],
parameter_map={"training_csv_path": "csv_path"},
description="非经验统计回归",
),
StepSpec(
step_id="step6_75", method_name="step6_75_custom_regression",
requires=["indices_path"], produces=["models_dir"],
parameter_map={"indices_path": "csv_path"},
description="自定义回归分析",
),
StepSpec(
step_id="step7", method_name="step7_generate_sampling_points",
requires=["deglint_img_path", "water_mask_path"], produces=["sampling_csv_path"],
description="整景密集采样点生成 + 光谱提取",
),
StepSpec(
step_id="step8", method_name="step8_predict_water_quality",
requires=["sampling_csv_path", "models_dir"], produces=["prediction_csv_path"],
description="ML 模型预测(采样点)",
),
StepSpec(
step_id="step8_5", method_name="step8_5_predict_with_non_empirical_models",
requires=["sampling_csv_path", "models_dir"], produces=["prediction_dir"],
parameter_map={"models_dir": "non_empirical_models_dir"},
description="非经验模型预测",
),
StepSpec(
step_id="step8_75", method_name="step8_75_predict_with_custom_regression",
requires=["sampling_csv_path", "models_dir", "formula_csv_path"],
produces=["prediction_dir"],
parameter_map={"models_dir": "custom_regression_dir"},
description="自定义回归预测",
),
StepSpec(
step_id="step9", method_name="step9_generate_distribution_map",
requires=["prediction_csv_path", "boundary_shp_path"],
produces=["distribution_map_path"],
description="克里金插值成图",
),
]
# ============================================================
# PipelineRunner执行者
# ============================================================
class PipelineRunner:
"""按 StepSpec 调度 14 个 step 方法,支持软取消 + 路径 ctx 注入。
用法:
runner = PipelineRunner(pipeline_instance)
ctx = PipelineContext(img_path=..., ...)
result_ctx = runner.run(ctx)
"""
def __init__(self, pipeline, steps: Optional[Sequence[StepSpec]] = None):
self.pipeline = pipeline
self.steps: List[StepSpec] = list(steps) if steps else list(PIPELINE_STEPS)
def run(self, ctx: PipelineContext) -> PipelineContext:
"""主入口:按顺序执行 14 步。软取消时已完成的 step 保留结果。"""
ctx.pipeline_start_time = time.time()
for spec in self.steps:
if ctx.is_cancelled():
ctx.append_log(f"[RUNNER] 收到取消信号,提前终止 @ {spec.step_id}")
break
if not spec.enabled:
ctx.status[spec.step_id] = "skipped"
ctx.append_log(f"[RUNNER] {spec.step_id} 标记为 disabled跳过")
continue
if spec.skip_when_missing:
missing = [k for k in spec.requires if not ctx.get(k)]
if missing:
ctx.status[spec.step_id] = "skipped"
reason = f"缺少必要的上下文参数,自动跳过: {missing}"
ctx.append_log(f"[RUNNER] {spec.step_id} {reason}")
if hasattr(self.pipeline, "_notify"):
self.pipeline._notify(spec.description, "skipped", reason)
continue
self._invoke(spec, ctx)
ctx.pipeline_end_time = time.time()
return ctx
# ------------------------------------------------------------------
def _invoke(self, spec: StepSpec, ctx: PipelineContext) -> None:
"""调一个 step 方法ctx 路径 → 形参;产出 → ctx 字段。"""
# DEBUG: 诊断"停在 step4"问题——每步打印 requires + ctx 实际数据
# 看到 requires=[] 但 actual=[None,...] 就说明 ctx 缺料step 会被 skip
ctx.append_log(
f"[DEBUG] Step {spec.step_id} requires: {spec.requires}, "
f"actual ctx data: {[ctx.get(k) for k in spec.requires]}"
)
method = getattr(self.pipeline, spec.method_name, None)
if method is None:
ctx.append_log(f"[RUNNER] 步骤方法缺失: {spec.method_name}(跳过)")
ctx.status[spec.step_id] = "skipped"
return
# 1) 把 ctx 路径作为形参注入(默认约定:去 _path 后缀)
kwargs: Dict[str, Any] = {}
for ctx_key in spec.requires:
param_name = spec.parameter_map.get(ctx_key, self._default_param_name(ctx_key))
kwargs[param_name] = ctx.get(ctx_key)
# 2) 允许用户在 ctx.user_config[step_id] 覆盖/补充
user_overrides = ctx.user_config.get(spec.step_id) or {}
if isinstance(user_overrides, dict):
for k, v in user_overrides.items():
# ★ 关键防御:绝不用 GUI 的“空字符串”或 None 覆盖上游传来的有效路径
if v is not None and v != "":
kwargs[k] = v
# 3) 状态置 start
ctx.append_log(
f"[RUNNER] -> {spec.method_name}({list(kwargs.keys())})"
)
ctx.status[spec.step_id] = "start"
notify = getattr(self.pipeline, "_notify", None)
if callable(notify):
try:
notify(f"步骤{spec.step_id[-1]}", "start", spec.method_name)
except Exception:
pass
# 4) 执行 + 捕获异常(不让单步崩溃拖垮 runner
t0 = time.time()
try:
result = method(**kwargs)
ctx.status[spec.step_id] = "completed"
ctx.step_timings[spec.step_id] = time.time() - t0
# 5) 产出收割
self._harvest(spec, result, ctx)
if callable(notify):
try:
notify(
f"步骤{spec.step_id[-1]}",
"completed",
str(result)[:200] if result is not None else "",
)
except Exception:
pass
except Exception as exc:
ctx.status[spec.step_id] = "error"
ctx.last_error = f"{spec.step_id}: {exc!r}"
ctx.append_log(f"[RUNNER] {spec.step_id} 异常: {exc!r}")
if callable(notify):
try:
notify(f"步骤{spec.step_id[-1]}", "error", str(exc))
except Exception:
pass
# ------------------------------------------------------------------
def _harvest(self, spec: StepSpec, result: Any, ctx: PipelineContext) -> None:
"""把 step 方法返回值灌入 ctx 的 produces 字段。
规则:
- 若 result 是 dict 且 key 匹配 produce_keyctx.set(produce_key, result[key])
- 若 result 非 dict 且 produces 非空:第一个 produces 字段接 result
- 若 produces 为空result 仅记录到 log不写 ctx
"""
if not spec.produces:
return
if isinstance(result, dict):
for produce_key in spec.produces:
if produce_key in result:
ctx.set(produce_key, result[produce_key])
elif result is not None:
ctx.set(spec.produces[0], result)
# ------------------------------------------------------------------
@staticmethod
def _default_param_name(ctx_key: str) -> str:
"""
废弃有毒的去 _path 后缀逻辑。
默认原样返回 ctx 键名作为形参名。遇到特殊缩写时,由各个 step 的 parameter_map 显式处理。
"""
return ctx_key

View File

@ -0,0 +1,544 @@
# -*- coding: utf-8 -*-
"""
Optuna + 智能子采样 AutoML 训练器(路线 B 防爆引擎)。
为什么需要这个:
- 老路径11 预处理 × 4 模型 × 3 划分 = 132 组 GridSearchCV
对中小数据集 10 分钟+,对大数据集 5w+ 行 直接 OOM
- AutoML 路径1 预处理 × N 模型Optuna 调超参),用智能子采样避开 OOM
再用最优超参在**全量数据**上 refit最终保存单一模型
设计要点:
- 入口 train_with_automl(csv, feature_start_column, model_names, ...)
- AutoMLResult dataclass 返回(每个目标列一份)
- smart_subsampleN > max_samples 时随机下采样
- 失败兜底optuna 未装 / 全 trial 失败 → fallback 到 WaterQualityModelingBatch
- 文件命名规范:{target}_{preprocess}_{model}_AUTOML.joblib
- save_data["metadata"]["automl"] = True 标记
调用:
from src.core.prediction.automl_trainer import train_with_automl
results = train_with_automl(
training_csv_path=".../training_spectra.csv",
feature_start_column="374.285004",
model_names=["RF", "SVR", "Ridge"],
n_trials=20,
timeout_sec=300,
)
"""
from __future__ import annotations
import json
import time
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Tuple
import numpy as np
import pandas as pd
# ============================================================
# 常量
# ============================================================
# AutoML 寻优阶段允许的最大样本数(避免 OOM
# 5000 样本对 RF/SVR/Ridge 的 Optuna 寻优足够给出稳定 CV
DEFAULT_MAX_SAMPLES = 5000
# 单次 Optuna trial 的默认超时(秒)
DEFAULT_TIMEOUT = 300.0
# 默认 trial 数
DEFAULT_N_TRIALS = 20
# AutoML 输出目录名后缀
AUTOML_DIR_SUFFIX = "_AutoML"
# ============================================================
# 数据类
# ============================================================
@dataclass
class AutoMLResult:
"""单个目标列的 AutoML 训练结果"""
success: bool = False
model_path: Optional[str] = None
cv_score: float = -float("inf")
best_params: Optional[Dict[str, Any]] = None
target_column: str = ""
preprocessing: str = ""
model_name: str = ""
n_trials_done: int = 0
n_samples_used: int = 0
fallback_used: bool = False
elapsed_sec: float = 0.0
error: Optional[str] = None
metadata: Dict[str, Any] = field(default_factory=dict)
# ============================================================
# 智能子采样
# ============================================================
def smart_subsample(
X: np.ndarray,
y: np.ndarray,
max_samples: int = DEFAULT_MAX_SAMPLES,
random_state: int = 42,
) -> Tuple[np.ndarray, np.ndarray, bool]:
"""当 N > max_samples 时随机下采样;否则原样返回。
Returns:
(X_sub, y_sub, was_subsampled)
"""
n = X.shape[0]
if n <= max_samples:
return X, y, False
rng = np.random.default_rng(random_state)
idx = rng.choice(n, size=max_samples, replace=False)
return X[idx], y[idx], True
# ============================================================
# 模型工厂
# ============================================================
def _build_model(model_name: str, random_state: int = 42):
"""根据英文模型键名构造 sklearn-compatible 模型实例factory"""
from sklearn.ensemble import (
AdaBoostRegressor, ExtraTreesRegressor, GradientBoostingRegressor,
RandomForestRegressor,
)
from sklearn.linear_model import (
ElasticNet, Lasso, LinearRegression, Ridge,
)
from sklearn.neighbors import KNeighborsRegressor
from sklearn.neural_network import MLPRegressor
from sklearn.svm import SVR
from sklearn.tree import DecisionTreeRegressor
factory = {
"RF": lambda **kw: RandomForestRegressor(random_state=random_state, n_jobs=1, **kw),
"ET": lambda **kw: ExtraTreesRegressor(random_state=random_state, n_jobs=1, **kw),
"GradientBoosting": lambda **kw: GradientBoostingRegressor(random_state=random_state, **kw),
"AdaBoost": lambda **kw: AdaBoostRegressor(random_state=random_state, **kw),
"Ridge": lambda **kw: Ridge(**kw),
"Lasso": lambda **kw: Lasso(max_iter=5000, **kw),
"ElasticNet": lambda **kw: ElasticNet(max_iter=5000, **kw),
"LinearRegression": lambda **kw: LinearRegression(**kw),
"SVR": lambda **kw: SVR(**kw),
"KNN": lambda **kw: KNeighborsRegressor(n_jobs=1, **kw),
"MLP": lambda **kw: MLPRegressor(max_iter=500, random_state=random_state, **kw),
"DecisionTree": lambda **kw: DecisionTreeRegressor(random_state=random_state, **kw),
"PLS": None, # sklearn.cross_decomposition.PLSRegression 暂未集成
}
builder = factory.get(model_name)
if builder is None:
return None
return builder
# ============================================================
# Optuna 超参 search space
# ============================================================
def _get_search_space(model_name: str, trial) -> Dict[str, Any]:
"""按模型名返回 Optuna 超参 search space。"""
sp: Dict[str, Any] = {}
if model_name == "RF":
sp["n_estimators"] = trial.suggest_int("n_estimators", 50, 300, step=50)
sp["max_depth"] = trial.suggest_int("max_depth", 3, 20)
sp["min_samples_split"] = trial.suggest_int("min_samples_split", 2, 10)
sp["min_samples_leaf"] = trial.suggest_int("min_samples_leaf", 1, 5)
elif model_name == "ET":
sp["n_estimators"] = trial.suggest_int("n_estimators", 50, 300, step=50)
sp["max_depth"] = trial.suggest_int("max_depth", 3, 20)
elif model_name == "GradientBoosting":
sp["n_estimators"] = trial.suggest_int("n_estimators", 50, 300, step=50)
sp["max_depth"] = trial.suggest_int("max_depth", 3, 8)
sp["learning_rate"] = trial.suggest_float("learning_rate", 0.01, 0.3, log=True)
elif model_name == "SVR":
sp["C"] = trial.suggest_float("C", 0.1, 100.0, log=True)
sp["epsilon"] = trial.suggest_float("epsilon", 0.001, 1.0, log=True)
sp["kernel"] = trial.suggest_categorical("kernel", ["rbf", "linear"])
elif model_name == "KNN":
sp["n_neighbors"] = trial.suggest_int("n_neighbors", 3, 20)
sp["weights"] = trial.suggest_categorical("weights", ["uniform", "distance"])
elif model_name in ("Ridge", "Lasso", "ElasticNet"):
sp["alpha"] = trial.suggest_float("alpha", 0.01, 100.0, log=True)
if model_name == "ElasticNet":
sp["l1_ratio"] = trial.suggest_float("l1_ratio", 0.0, 1.0)
elif model_name == "MLP":
sp["hidden_layer_sizes"] = trial.suggest_categorical(
"hidden_layer_sizes", [(50,), (100,), (50, 50), (100, 50)]
)
sp["alpha"] = trial.suggest_float("alpha", 1e-5, 1e-1, log=True)
sp["learning_rate_init"] = trial.suggest_float("learning_rate_init", 1e-4, 1e-2, log=True)
elif model_name == "DecisionTree":
sp["max_depth"] = trial.suggest_int("max_depth", 3, 20)
sp["min_samples_split"] = trial.suggest_int("min_samples_split", 2, 10)
elif model_name == "AdaBoost":
sp["n_estimators"] = trial.suggest_int("n_estimators", 30, 200, step=30)
sp["learning_rate"] = trial.suggest_float("learning_rate", 0.01, 1.0, log=True)
else:
sp["n_estimators"] = trial.suggest_int("n_estimators", 50, 200, step=50)
return sp
def _make_objective(model_name: str, X: np.ndarray, y: np.ndarray,
cv_folds: int, random_state: int):
"""构造 Optuna objective5 折 CV R²"""
from sklearn.model_selection import KFold, cross_val_score
def objective(trial):
params = _get_search_space(model_name, trial)
try:
builder = _build_model(model_name, random_state=random_state)
if builder is None:
return -1.0
model = builder(**params)
kf = KFold(n_splits=cv_folds, shuffle=True, random_state=random_state)
scores = cross_val_score(model, X, y, cv=kf, scoring="r2", n_jobs=1)
return float(np.mean(scores))
except Exception:
return -1.0
return objective
def _refit_full(model_name: str, best_params: Dict[str, Any],
X: np.ndarray, y: np.ndarray, random_state: int):
"""用 best params 在**全量数据**上 refit。"""
builder = _build_model(model_name, random_state=random_state)
if builder is None:
return None
model = builder(**best_params)
model.fit(X, y)
return model
# ============================================================
# 失败兜底(回退到老 GridSearchCV 路径)
# ============================================================
def _fallback_train(
training_csv_path: str,
feature_start_column,
preprocessing: str,
model_name: str,
split_method: str,
cv_folds: int,
output_dir: Path,
target_column: str,
) -> AutoMLResult:
"""AutoML 失败时调老 WaterQualityModelingBatch。
返回的 AutoMLResult.fallback_used=True。
"""
try:
from src.core.modeling.modeling_batch import WaterQualityModelingBatch
except ImportError as e:
return AutoMLResult(
success=False, error=f"fallback 导入失败: {e!r}", fallback_used=True,
target_column=target_column, preprocessing=preprocessing, model_name=model_name,
)
try:
out_dir = output_dir / preprocessing
out_dir.mkdir(parents=True, exist_ok=True)
modeler = WaterQualityModelingBatch(str(out_dir))
modeler.train_models_batch(
csv_path=training_csv_path,
feature_start_column=feature_start_column,
preprocessing_methods=[preprocessing],
model_names=[model_name],
split_methods=[split_method],
cv_folds=cv_folds,
)
# 找产出
candidates = list(out_dir.rglob(f"{target_column}_{preprocessing}_{model_name}.joblib"))
model_path = str(candidates[0]) if candidates else None
return AutoMLResult(
success=model_path is not None,
model_path=model_path,
target_column=target_column, preprocessing=preprocessing, model_name=model_name,
fallback_used=True,
metadata={"source": "WaterQualityModelingBatch"},
)
except Exception as e:
return AutoMLResult(
success=False, error=f"fallback 失败: {e!r}", fallback_used=True,
target_column=target_column, preprocessing=preprocessing, model_name=model_name,
)
# ============================================================
# 主入口
# ============================================================
def train_with_automl(
training_csv_path: str,
feature_start_column,
preprocessing_methods: Optional[List[str]] = None,
model_names: Optional[List[str]] = None,
split_methods: Optional[List[str]] = None,
cv_folds: int = 5,
output_dir: Optional[str] = None,
n_trials: int = DEFAULT_N_TRIALS,
timeout_sec: float = DEFAULT_TIMEOUT,
max_samples: int = DEFAULT_MAX_SAMPLES,
random_state: int = 42,
callback: Optional[Callable[[str, str, str], None]] = None,
) -> List[AutoMLResult]:
"""用 Optuna + 子采样跑 AutoML。失败时自动回退到 GridSearchCV。
Args:
training_csv_path: 训练用 CSVStep 5 产物 training_spectra.csv
feature_start_column: 特征起始列名或索引(之前所有列视为目标 y
preprocessing_methods: 候选预处理列表(**仅用第 1 个**,避免笛卡尔爆炸)
model_names: 候选模型列表(每个都会跑一遍 Optuna
split_methods: 候选数据划分列表AutoML 仅用第 1 个)
cv_folds: 交叉验证折数
output_dir: 输出目录(默认 <models_dir>_AutoML
n_trials: 单模型 Optuna trial 数
timeout_sec: 单模型超时(秒),到时强制停止
max_samples: 寻优阶段允许的最大样本数
callback: 状态回调 callback(step_name, status, message)
Returns:
List[AutoMLResult],每个目标列一份结果
"""
def notify(status: str, msg: str = "") -> None:
if callback:
callback("步骤6_AutoML", status, msg)
# ---- 1) 参数默认值 ----
if preprocessing_methods is None:
preprocessing_methods = ["MMS"]
if model_names is None:
model_names = ["RF", "SVR", "Ridge"]
if split_methods is None:
split_methods = ["spxy"]
# 决策:仅用第一个预处理 + 第一个划分,避免笛卡尔爆炸
preproc = preprocessing_methods[0]
split_method = split_methods[0]
if output_dir is None:
output_dir = "./7_Supervised_Model_Training_AutoML"
out_dir = Path(output_dir)
out_dir.mkdir(parents=True, exist_ok=True)
preproc_dir = out_dir / preproc
preproc_dir.mkdir(parents=True, exist_ok=True)
# ---- 2) 加载数据 ----
notify("start", f"AutoML 训练开始 (n_trials={n_trials}, timeout={timeout_sec}s, max_samples={max_samples})")
if not Path(training_csv_path).exists():
return [AutoMLResult(success=False, error=f"训练 CSV 不存在: {training_csv_path}")]
df = pd.read_csv(training_csv_path)
# 提取目标列feature_start_column 之前所有数值列)
if isinstance(feature_start_column, int):
y_cols = [c for c in df.columns[:feature_start_column]
if pd.api.types.is_numeric_dtype(df[c])]
else:
try:
idx = list(df.columns).index(feature_start_column)
y_cols = [c for c in df.columns[:idx]
if pd.api.types.is_numeric_dtype(df[c])]
except ValueError:
y_cols = []
if not y_cols:
notify("error", "AutoML: 未识别出目标列feature_start_column 之前的所有数值列)")
return [AutoMLResult(success=False, error="未识别出目标列")]
feat_cols = [c for c in df.columns if c not in y_cols]
X_all = df[feat_cols].values.astype(np.float64)
# ---- 3) 预处理(仅第一项) ----
if preproc != "None":
try:
from src.preprocessing.spectral_Preprocessing import Preprocessing
processed = Preprocessing(preproc, df[feat_cols])
if isinstance(processed, pd.DataFrame):
X_all = processed.values.astype(np.float64)
else:
X_all = np.asarray(processed, dtype=np.float64)
except Exception as e:
notify("warning", f"预处理 {preproc} 失败: {e!r},改用 None")
preproc = "None"
# ---- 4) 检查 Optuna 是否可用 ----
try:
import optuna
optuna.logging.set_verbosity(optuna.logging.WARNING)
optuna_available = True
except ImportError:
optuna_available = False
notify("warning", "optuna 未安装,全目标列回退到 GridSearchCVpip install \"optuna>=3.6\"")
# ---- 5) 逐 target 跑 ----
results: List[AutoMLResult] = []
total = len(y_cols)
per_model_timeout = max(10.0, timeout_sec / max(1, len(model_names)))
for ti, tgt in enumerate(y_cols, 1):
t0 = time.time()
yv = df[tgt].values.astype(np.float64)
mask = ~np.isnan(yv)
X_t = X_all[mask]
y_t = yv[mask]
if X_t.shape[0] < cv_folds * 2:
notify("warning", f"目标 {tgt}: 有效样本 {X_t.shape[0]} 不足,跳过")
results.append(AutoMLResult(
success=False, target_column=tgt, error=f"样本不足({X_t.shape[0]})",
preprocessing=preproc,
))
continue
X_sub, y_sub, was_sub = smart_subsample(X_t, y_t, max_samples=max_samples, random_state=random_state)
if was_sub:
notify("info", f"目标 {tgt}: {X_t.shape[0]} 样本 → 子采样 {X_sub.shape[0]}(寻优用)")
best_overall = AutoMLResult(success=False, target_column=tgt, preprocessing=preproc)
if not optuna_available:
# 全目标列一次性 fallback
best_overall = _fallback_train(
training_csv_path, feature_start_column, preproc, model_names[0], split_method,
cv_folds, out_dir, tgt,
)
else:
for model_name in model_names:
try:
builder = _build_model(model_name, random_state=random_state)
if builder is None:
notify("warning", f"模型 {model_name} 暂不支持 AutoML 寻优")
continue
study = optuna.create_study(
direction="maximize",
sampler=optuna.samplers.TPESampler(seed=random_state),
)
study.optimize(
_make_objective(model_name, X_sub, y_sub, cv_folds, random_state),
n_trials=n_trials,
timeout=per_model_timeout,
show_progress_bar=False,
)
if study.best_value is None or study.best_value <= -1.0:
notify("warning", f"{tgt}/{model_name}: 全部 trial 失败CV 全部 <= -1")
continue
# refit on FULL
final_model = _refit_full(model_name, study.best_params, X_t, y_t, random_state)
if final_model is None:
continue
# 保存
import joblib
fname = f"{tgt}_{preproc}_{model_name}_AUTOML.joblib"
fpath = preproc_dir / fname
joblib.dump({
"model": final_model,
"target_column_name": tgt,
"preprocess_method": preproc,
"model_name": model_name,
"metadata": {
"automl": True,
"best_params": study.best_params,
"cv_score": float(study.best_value),
"n_trials_done": len(study.trials),
"n_samples_used_full": int(X_t.shape[0]),
"n_samples_used_for_search": int(X_sub.shape[0]),
"was_subsampled": was_sub,
"split_method": split_method,
},
}, fpath)
cand = AutoMLResult(
success=True,
model_path=str(fpath),
cv_score=float(study.best_value),
best_params=study.best_params,
target_column=tgt,
preprocessing=preproc,
model_name=model_name,
n_trials_done=len(study.trials),
n_samples_used=int(X_sub.shape[0]),
metadata={"refit_on_full": True, "n_samples_full": int(X_t.shape[0])},
)
if cand.cv_score > best_overall.cv_score:
best_overall = cand
except Exception as e:
notify("warning", f"目标 {tgt} / 模型 {model_name} 失败: {e!r}")
continue
if not best_overall.success:
notify("warning", f"目标 {tgt} 全部 Optuna trial 失败,回退 GridSearchCV")
best_overall = _fallback_train(
training_csv_path, feature_start_column, preproc, model_names[0], split_method,
cv_folds, out_dir, tgt,
)
best_overall.elapsed_sec = time.time() - t0
results.append(best_overall)
notify("info", f"AutoML 目标 {tgt} 完成 ({ti}/{total}) cv={best_overall.cv_score:.4f}")
# ---- 6) 汇总 json ----
summary_path = out_dir / "automl_summary.json"
try:
with open(summary_path, "w", encoding="utf-8") as f:
json.dump([asdict(r) for r in results], f, ensure_ascii=False, indent=2, default=str)
except Exception as e:
notify("warning", f"写 automl_summary.json 失败: {e!r}")
success_n = sum(1 for r in results if r.success)
fallback_n = sum(1 for r in results if r.fallback_used)
notify("completed", f"AutoML 训练完成 {success_n}/{len(results)} 成功({fallback_n} 走 fallback汇总 {summary_path}")
return results
# ============================================================
# CLI 自测
# ============================================================
if __name__ == "__main__":
import argparse
p = argparse.ArgumentParser(description="AutoML 训练器 CLI 自测")
p.add_argument("--csv", required=True, help="训练用 CSVfeature_start_column 之前的列为目标 y")
p.add_argument("--feature-start", default="0", help="特征起始列名或索引(默认 0")
p.add_argument("--n-trials", type=int, default=DEFAULT_N_TRIALS)
p.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT)
p.add_argument("--max-samples", type=int, default=DEFAULT_MAX_SAMPLES)
p.add_argument("--out", default="./7_Supervised_Model_Training_AutoML")
args = p.parse_args()
# 智能推断 feature_start_column 类型
fsc: Any = args.feature_start
try:
fsc = int(fsc)
except ValueError:
pass
res = train_with_automl(
training_csv_path=args.csv,
feature_start_column=fsc,
n_trials=args.n_trials,
timeout_sec=args.timeout,
max_samples=args.max_samples,
output_dir=args.out,
)
print(f"\n训练完成 {len(res)} 个目标")
for r in res:
marker = "" if r.success else ""
fb = " [fallback]" if r.fallback_used else ""
print(f" {marker} {r.target_column}: cv={r.cv_score:.4f} path={r.model_path}{fb}")

View File

@ -26,19 +26,31 @@ from sklearn.model_selection import train_test_split
class WaterQualityInference:
"""水质参数反演推理类"""
def __init__(self, artifacts_dir: str = "models/artifacts"):
def __init__(self, artifacts_dir: str = "models/artifacts",
external_model=None, external_model_path=None):
"""
初始化推理类
Args:
artifacts_dir: 模型保存目录
external_model: 外部预训练模型对象(来自 GUI 导入,跳过磁盘加载)
external_model_path: 外部模型文件路径(仅用于日志)
"""
self.artifacts_dir = Path(artifacts_dir)
if not self.artifacts_dir.exists():
print(f"警告: 模型目录不存在: {artifacts_dir},将在需要时创建")
self.best_model_info = None
self.loaded_model_data = None
self.external_model = external_model
self.external_model_path = external_model_path
# 规范化 loaded_model_data始终为 dict确保 ['model'] 访问不崩溃
if external_model is not None:
# 外部传入的是裸模型对象 → 包装为 dict统一后续 .get('model') 访问
self.loaded_model_data = {'model': external_model, 'preprocess_method': 'None'}
print(f" 外部模型已规范化: type={type(external_model).__name__}")
else:
self.loaded_model_data = None
def load_sampling_data(self, csv_path: str) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
"""
@ -745,7 +757,10 @@ class WaterQualityInference:
# 1. 加载模型
print("\n步骤1: 加载模型")
print("-" * 40)
if model_file_path:
if self.external_model is not None:
# 已在 __init__ 中规范化,无需重复赋值
print(f" 使用外部预训练模型: type={type(self.external_model).__name__}")
elif model_file_path:
self.load_specific_model(model_file_path)
else:
self.load_best_model(metric=metric)
@ -793,8 +808,8 @@ class WaterQualityInference:
info = {
"status": "model_loaded",
"preprocess_method": self.loaded_model_data['preprocess_method'],
"model_name": self.loaded_model_data['model_name'],
"preprocess_method": self.loaded_model_data.get('preprocess_method', 'Unknown'),
"model_name": self.loaded_model_data.get('model_name', type(self.external_model).__name__ if self.external_model else 'Unknown'),
"model_type": str(type(self.loaded_model_data['model'])),
"metadata": self.loaded_model_data.get('metadata', {})
}
@ -863,10 +878,13 @@ class WaterQualityInference:
print(f"\n批量推理完成,共处理 {len(csv_files)} 个文件")
return results
def batch_inference_multi_models(self, models_root_dir: str, sampling_csv_path: str,
output_dir: str, metric: str = 'test_r2',
def batch_inference_multi_models(self, models_root_dir: str, sampling_csv_path: str,
output_dir: str, metric: str = 'test_r2',
prediction_column: str = 'prediction',
output_format: str = 'csv'):
output_format: str = 'csv',
external_model=None,
external_model_path=None,
external_models_dict=None):
"""
使用多个子文件夹中的模型进行批量推理
@ -881,28 +899,62 @@ class WaterQualityInference:
models_root = Path(models_root_dir)
output_path = Path(output_dir)
output_path.mkdir(parents=True, exist_ok=True)
# 查找所有子文件夹
subdirs = [d for d in models_root.iterdir() if d.is_dir()]
if not subdirs:
print(f"在目录 {models_root_dir} 中未找到子文件夹")
return
print(f"找到 {len(subdirs)} 个模型子文件夹进行批量推理")
print(f"输出格式: {output_format.upper()}")
all_results = {}
for subdir in subdirs:
# 优先级 1_external_models_dict 非空 → 直接用字典的 keys 作为 targets不扫描磁盘
print(f"[BatchInference] 终于收到字典啦!包含模型: {list(external_models_dict.keys()) if external_models_dict else 'None'}")
if external_models_dict is not None and len(external_models_dict) > 0:
targets = list(external_models_dict.keys())
print(f"\n使用外部导入模型字典({len(targets)} 个模型)")
print(f"检测到外部导入模型,将预测以下参数: {targets}")
elif external_model is not None:
print(f"\n使用外部预训练模型: {external_model_path or 'unknown'}")
subdirs = [d for d in models_root.iterdir() if d.is_dir()]
if not subdirs:
print(f"在目录 {models_root_dir} 中未找到子文件夹")
return {}
print(f"找到 {len(subdirs)} 个模型子文件夹进行批量推理")
targets = [d.name for d in subdirs]
else:
subdirs = [d for d in models_root.iterdir() if d.is_dir()]
if not subdirs:
print(f"在目录 {models_root_dir} 中未找到子文件夹")
return {}
print(f"找到 {len(subdirs)} 个模型子文件夹进行批量推理")
targets = [d.name for d in subdirs]
print(f"输出格式: {output_format.upper()}")
for subdir_name in targets:
try:
subdir_name = subdir.name
print(f"\n{'='*60}")
print(f"处理模型文件夹: {subdir_name}")
print(f"处理模型: {subdir_name}")
print(f"{'='*60}")
# 创建新的推理实例使用当前子文件夹作为artifacts_dir
model_inferencer = WaterQualityInference(str(subdir))
# 优先级:字典中该 target 的模型 > 共享单模型 > 磁盘加载
effective_model = None
if external_models_dict and subdir_name in external_models_dict:
effective_model = external_models_dict[subdir_name]
print(f" → 使用字典中模型: {type(effective_model).__name__}")
elif external_model is not None:
effective_model = external_model
print(f" → 使用共享外部模型: {type(effective_model).__name__}")
# artifacts_dir字典模式优先用 placeholder "./",否则用真实子目录
artifacts_dir = (
str(models_root / subdir_name)
if (models_root / subdir_name).is_dir()
else str(models_root)
)
if effective_model is not None:
model_inferencer = WaterQualityInference(
artifacts_dir,
external_model=effective_model,
external_model_path=external_model_path or "",
)
else:
model_inferencer = WaterQualityInference(artifacts_dir)
# 根据输出格式设置文件扩展名
file_ext = f".{output_format}"
@ -931,10 +983,10 @@ class WaterQualityInference:
}
}
print(f"子文件夹 {subdir_name} 处理完成")
print(f"模型 {subdir_name} 处理完成")
except Exception as e:
print(f"处理子文件夹 {subdir_name} 失败: {e}")
print(f"处理模型 {subdir_name} 失败: {e}")
all_results[subdir_name] = {
'status': 'error',
'error': str(e)

View File

@ -126,7 +126,7 @@ class DataPreparationStep:
@staticmethod
def calculate_water_quality_indices(
training_spectra_path: Optional[str] = None,
training_csv_path: Optional[str] = None,
formula_csv_file: Optional[str] = None,
formula_names: Optional[List[str]] = None,
output_file: Optional[str] = None,
@ -153,8 +153,8 @@ class DataPreparationStep:
notify("skipped", "跳过水质指数计算")
return None
if training_spectra_path is None:
raise ValueError("必须提供 training_spectra_path 参数")
if training_csv_path is None:
raise ValueError("必须提供 training_csv_path 参数")
if formula_csv_file is None:
raise ValueError("必须提供 formula_csv_file 参数")
@ -170,7 +170,7 @@ class DataPreparationStep:
from src.utils.band_math import BandMathCalculator
calculator = BandMathCalculator(training_spectra_path)
calculator = BandMathCalculator(training_csv_path)
result_df = calculator.process_formulas_from_csv(
formula_csv_file=formula_csv_file,
formula_names=formula_names,

View File

@ -103,6 +103,10 @@ class PredictionStep:
output_dir: Union[str, Path] = "./11_12_13_predictions/Machine_Learning_Prediction",
callback: Optional[Callable] = None,
_report_generator=None,
_external_model=None,
_external_model_path=None,
_external_models_dict=None,
_external_model_dir=None,
) -> Dict[str, str]:
"""将训练好的最佳机器学习模型应用到采样点光谱上,预测水质参数"""
from src.core.prediction.inference_batch import WaterQualityInference
@ -114,6 +118,8 @@ class PredictionStep:
print("\n" + "=" * 80)
print("步骤8: 预测水质参数")
print("=" * 80)
print(f"[PredictionStep] 准备执行预测,字典状态: {'Yes' if _external_models_dict else 'No'}"
f", 单模型状态: {'Yes' if _external_model else 'No'}")
step_start_time = time.time()
@ -149,19 +155,60 @@ class PredictionStep:
else:
print(f"检测到部分预测结果文件,缺少: {missing_targets},将继续生成...")
inferencer = WaterQualityInference(models_dir)
all_results = inferencer.batch_inference_multi_models(
models_root_dir=models_dir,
sampling_csv_path=sampling_csv_path,
output_dir=str(ml_prediction_dir),
metric=metric,
prediction_column=prediction_column,
output_format="csv",
)
all_results = {}
for target_name, result in all_results.items():
if result.get("status") == "success":
prediction_files[target_name] = result["output_file"]
if _external_models_dict:
# 外部模型字典优先:直接用字典的 keys 作为 targets 列表,
# 手动为每个模型创建 inference 实例并调用 inference_pipeline。
print(f"\n使用外部导入模型字典({len(_external_models_dict)} 个模型)...")
for target_name, model_obj in _external_models_dict.items():
try:
output_file = ml_prediction_dir / f"{target_name}.csv"
model_inferencer = WaterQualityInference(
models_dir or "./",
external_model=model_obj,
external_model_path=_external_model_dir or "",
)
predictions, result_df = model_inferencer.inference_pipeline(
sampling_csv_path=sampling_csv_path,
output_csv_path=str(output_file),
metric=metric,
prediction_column=prediction_column,
)
prediction_files[target_name] = str(output_file)
all_results[target_name] = {
"status": "success",
"output_file": str(output_file),
"sample_count": len(predictions),
}
print(f"{target_name}: {len(predictions)} 个预测值")
except Exception as e:
print(f"{target_name}: 失败 — {type(e).__name__}: {e}")
prediction_files[target_name] = None
all_results[target_name] = {"status": "error", "error": str(e)}
else:
# 字典为空或不存在:回退到扫描 models_dir 子目录的传统逻辑
inferencer = WaterQualityInference(
models_dir,
external_model=_external_model,
external_model_path=_external_model_path,
)
all_results = inferencer.batch_inference_multi_models(
models_root_dir=models_dir,
sampling_csv_path=sampling_csv_path,
output_dir=str(ml_prediction_dir),
metric=metric,
prediction_column=prediction_column,
output_format="csv",
external_model=_external_model,
external_model_path=_external_model_path,
external_models_dict=_external_models_dict,
)
# batch_inference_multi_models 已确保返回字典,永不返回 None
if all_results:
for target_name, result in all_results.items():
if result.get("status") == "success":
prediction_files[target_name] = result["output_file"]
print(f"预测完成,结果保存在: {ml_prediction_dir}")

View File

@ -173,7 +173,7 @@ class WaterQualityInversionPipeline:
self.interpolated_img_path = None # 存储插值后的影像路径
self.deglint_img_path = None
self.processed_csv_path = None
self.training_spectra_path = None
self.training_csv_path = None
self.indices_path = None
self.custom_regression_path = None
@ -267,7 +267,7 @@ class WaterQualityInversionPipeline:
use_ndwi: bool = False,
skip_dependency_check: bool = False,
generate_png: bool = True,
output_path: Optional[str] = None) -> str:
output_path: Optional[str] = None, **kwargs) -> str:
"""步骤1: 生成或设置水域maskFacade"""
step_start_time = time.time()
try:
@ -392,7 +392,7 @@ class WaterQualityInversionPipeline:
max_area: Optional[int] = None,
buffer_size: Optional[int] = None,
water_mask_path: Optional[str] = None,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""步骤2: 找到耀斑区域Facade"""
step_start_time = time.time()
try:
@ -511,7 +511,7 @@ class WaterQualityInversionPipeline:
left_shoulder_wave: Optional[float] = None,
valley_wave: Optional[float] = None,
right_shoulder_wave: Optional[float] = None,
water_mask: Optional[Union[str, np.ndarray]] = None,
water_mask_path: Optional[Union[str, np.ndarray]] = None,
interpolate_zeros: bool = False,
interpolation_method: str = 'nearest',
enabled: bool = True,
@ -533,7 +533,7 @@ class WaterQualityInversionPipeline:
sugar_iter: Optional[int] = 3,
sugar_termination_thresh: float = 20.0,
output_path: Optional[str] = None,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""步骤3: 去除耀斑Facade"""
step_start_time = time.time()
try:
@ -546,7 +546,7 @@ class WaterQualityInversionPipeline:
left_shoulder_wave=left_shoulder_wave,
valley_wave=valley_wave,
right_shoulder_wave=right_shoulder_wave,
water_mask=water_mask,
water_mask=water_mask_path,
interpolate_zeros=interpolate_zeros,
interpolation_method=interpolation_method,
enabled=enabled,
@ -588,7 +588,7 @@ class WaterQualityInversionPipeline:
status="failed", error=str(e))
raise
def step4_process_csv(self, csv_path: str, skip_dependency_check: bool = False) -> str:
def step4_process_csv(self, csv_path: str, skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤4: 对csv文件进行处理筛选剔除异常值
@ -615,7 +615,7 @@ class WaterQualityInversionPipeline:
csv_path: Optional[str] = None,
boundary_path: Optional[str] = None,
glint_mask_path: Optional[str] = None,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤5: 根据csv文件的采样点坐标在去除耀斑的文件中统计采样点的平均光谱
@ -655,43 +655,43 @@ class WaterQualityInversionPipeline:
water_mask_path=self.water_mask_path,
output_dir=str(self.training_spectra_dir),
)
self.training_spectra_path = result
self.training_csv_path = result
self._record_step_time("步骤5: 提取训练样本点光谱", 0, 0)
self._notify("completed", f"训练光谱数据已保存: {result}")
return result
def step5_5_calculate_water_quality_indices(self,
training_spectra_path: Optional[str] = None,
training_csv_path: Optional[str] = None,
formula_csv_file: Optional[str] = None,
formula_names: Optional[List[str]] = None,
output_file: Optional[str] = None,
enabled: bool = True,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤5.5: 根据训练光谱计算水质光谱指数
使用band_math.py中的方法实现支持从公式CSV文件中批量计算指定公式
Args:
training_spectra_path: 训练光谱数据CSV路径如果为None使用步骤5的结果
training_csv_path: 训练光谱数据CSV路径如果为None使用步骤5的结果
formula_csv_file: 公式CSV文件路径包含公式名称和具体公式
formula_names: 要计算的公式名称列表如果为None则计算所有公式
output_file: 输出文件完整路径支持绝对路径如果为None则使用默认路径
Returns:
包含计算结果的新CSV文件路径
"""
# 参数解析(保留原逻辑)
if training_spectra_path is not None:
csv_path = training_spectra_path
elif self.training_spectra_path is not None:
csv_path = self.training_spectra_path
if training_csv_path is not None:
csv_path = training_csv_path
elif self.training_csv_path is not None:
csv_path = self.training_csv_path
else:
csv_path = None
self._notify("started", "步骤5.5: 计算水质光谱指数")
result = DataPreparationStep.calculate_water_quality_indices(
training_spectra_path=csv_path,
training_csv_path=csv_path,
formula_csv_file=formula_csv_file,
formula_names=formula_names,
output_file=output_file,
@ -710,7 +710,7 @@ class WaterQualityInversionPipeline:
split_methods: List[str] = None,
cv_folds: int = 5,
training_csv_path: Optional[str] = None,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤6: 使用采样点的平均光谱和对应的实测值建立机器学习模型,保存模型权重
@ -727,8 +727,8 @@ class WaterQualityInversionPipeline:
# 参数解析(保留原逻辑)
if training_csv_path is not None:
final_csv_path = training_csv_path
elif self.training_spectra_path is not None:
final_csv_path = self.training_spectra_path
elif self.training_csv_path is not None:
final_csv_path = self.training_csv_path
else:
final_csv_path = None
@ -753,7 +753,7 @@ class WaterQualityInversionPipeline:
chunk_size: int = 1000,
water_mask_path: Optional[str] = None,
glint_mask_path: Optional[str] = None,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤7: 生成根据水域掩膜内且耀斑掩膜外的采样点,统计采样点的平均光谱
@ -795,7 +795,7 @@ class WaterQualityInversionPipeline:
models_dir: Optional[str] = None,
metric: str = 'test_r2',
prediction_column: str = 'prediction',
skip_dependency_check: bool = False) -> Dict[str, str]:
skip_dependency_check: bool = False, **kwargs) -> Dict[str, str]:
"""
步骤8: 将训练好的最佳机器学习模型应用到采样点的平均光谱上,预测水质参数
@ -808,6 +808,13 @@ class WaterQualityInversionPipeline:
Returns:
预测结果文件路径字典(键为目标列名)
"""
_external_models_dict = kwargs.get('_external_models_dict')
_external_model = kwargs.get('_external_model')
_external_model_path = kwargs.get('_external_model_path')
_external_model_dir = kwargs.get('_external_model_dir')
print(f"[Pipeline] 收到字典: {'Yes' if _external_models_dict else 'No'}"
f", 收到单模型: {'Yes' if _external_model else 'No'}")
self._notify("started", "步骤8: 预测水质参数")
result = PredictionStep.predict_water_quality(
sampling_csv_path=sampling_csv_path,
@ -816,11 +823,15 @@ class WaterQualityInversionPipeline:
prediction_column=prediction_column,
output_dir=str(self.prediction_dir / "Machine_Learning_Prediction"),
_report_generator=self.report_generator,
_external_model=_external_model,
_external_model_path=_external_model_path,
_external_models_dict=_external_models_dict,
_external_model_dir=_external_model_dir,
)
self._record_step_time("步骤8: 预测水质参数", 0, 0)
self._notify("completed", f"预测完成,结果保存在: {self.prediction_dir}")
return result
def step9_generate_distribution_map(self, prediction_csv_path: str,
boundary_shp_path: str,
output_image_path: Optional[str] = None,
@ -835,7 +846,7 @@ class WaterQualityInversionPipeline:
diffusion_n_neighbors: int = 15,
cmap: Optional[str] = None,
expand_ratio: float = 0.05,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤9: 根据采样点的坐标和反演的实测参数,以及水域掩膜,通过插值的方法,得到水质参数的可视化分布图
@ -911,7 +922,7 @@ class WaterQualityInversionPipeline:
print("="*80)
if training_csv_path is None:
training_csv_path = self.training_spectra_path
training_csv_path = self.training_csv_path
if training_csv_path is None:
raise ValueError("请提供训练数据CSV路径或先执行步骤5")
@ -1033,7 +1044,7 @@ class WaterQualityInversionPipeline:
print("="*80)
if csv_path is None:
csv_path = self.training_spectra_path
csv_path = self.training_csv_path
if csv_path is None:
raise ValueError("请提供CSV文件路径或先执行步骤5")
@ -1506,7 +1517,7 @@ class WaterQualityInversionPipeline:
if 'step5' in config:
self._notify("步骤5: 光谱提取", "start")
self.step5_extract_training_spectra(**config['step5'])
self._notify("步骤5: 光谱提取", "completed", f"(输出: {self.training_spectra_path})")
self._notify("步骤5: 光谱提取", "completed", f"(输出: {self.training_csv_path})")
else:
self._notify("步骤5: 光谱提取", "skipped", "未配置")
@ -1615,7 +1626,7 @@ class WaterQualityInversionPipeline:
# 生成散点图
if 'visualization' in config and config['visualization'].get('generate_scatter', True):
if self.training_spectra_path and self.models_dir.exists():
if self.training_csv_path and self.models_dir.exists():
try:
self._notify("可视化", "info", "生成模型评估散点图...")
scatter_config = config['visualization'].get('scatter_config', {})
@ -1653,7 +1664,7 @@ class WaterQualityInversionPipeline:
# 生成光谱曲线图
if 'visualization' in config and config['visualization'].get('generate_spectrum', True):
if self.training_spectra_path:
if self.training_csv_path:
try:
self._notify("可视化", "info", "生成光谱曲线对比图...")
spectrum_paths = self.generate_spectrum_comparison_plots(
@ -1701,7 +1712,7 @@ class WaterQualityInversionPipeline:
pipeline_info['step2'] = {'status': 'completed', 'output_file': str(self.glint_mask_path) if self.glint_mask_path else 'N/A'}
pipeline_info['step3'] = {'status': 'completed', 'output_file': str(self.deglint_img_path) if self.deglint_img_path else 'N/A'}
pipeline_info['step4'] = {'status': 'completed', 'output_file': str(self.processed_csv_path) if self.processed_csv_path else 'N/A'}
pipeline_info['step5'] = {'status': 'completed', 'output_file': str(self.training_spectra_path) if self.training_spectra_path else 'N/A'}
pipeline_info['step5'] = {'status': 'completed', 'output_file': str(self.training_csv_path) if self.training_csv_path else 'N/A'}
pipeline_info['step5_5'] = {'status': 'completed', 'output_file': str(self.indices_path) if self.indices_path else 'N/A'}
pipeline_info['step6'] = {'status': 'completed', 'output_file': str(self.models_dir)}
pipeline_info['step6_75'] = {'status': 'completed', 'output_file': str(self.custom_regression_path) if self.custom_regression_path else 'N/A'}
@ -1764,7 +1775,7 @@ class WaterQualityInversionPipeline:
window: int = 5,
output_dir: Optional[str] = None,
enabled: bool = True,
skip_dependency_check: bool = False) -> Dict[str, str]:
skip_dependency_check: bool = False, **kwargs) -> Dict[str, str]:
"""
步骤6.5: 非经验统计回归模型训练
@ -1784,8 +1795,8 @@ class WaterQualityInversionPipeline:
# 参数解析(保留原逻辑)
if csv_path is not None:
final_csv_path = csv_path
elif self.training_spectra_path is not None:
final_csv_path = self.training_spectra_path
elif self.training_csv_path is not None:
final_csv_path = self.training_csv_path
else:
final_csv_path = None
@ -1812,7 +1823,7 @@ class WaterQualityInversionPipeline:
methods: Union[str, List[str]] = 'all',
output_dir: Optional[str] = None,
enabled: bool = True,
skip_dependency_check: bool = False) -> str:
skip_dependency_check: bool = False, **kwargs) -> str:
"""
步骤6.75: 使用自定义回归方法分析指标与目标参数之间的关系
"""
@ -1991,7 +2002,7 @@ class WaterQualityInversionPipeline:
metric: str = 'Average Accuracy(%)',
prediction_column: str = 'prediction',
enabled: bool = True,
skip_dependency_check: bool = False) -> Dict[str, str]:
skip_dependency_check: bool = False, **kwargs) -> Dict[str, str]:
"""
步骤8.5: 使用非经验统计回归模型进行参数预测
@ -2028,7 +2039,7 @@ class WaterQualityInversionPipeline:
output_dir: Optional[str] = None,
filename_prefix: str = "custom_regression_prediction",
enabled: bool = True,
skip_dependency_check: bool = False) -> Dict[str, str]:
skip_dependency_check: bool = False, **kwargs) -> Dict[str, str]:
"""
步骤8.75: 使用自定义回归模型进行参数预测
@ -2109,7 +2120,7 @@ def main():
'interpolation_method': 'bilinear', # 插值方法: 'nearest'(邻近), 'bilinear'(双线性),
# 'spline'(样条), 'kriging'(克里金)
# 水域掩膜参数(可选):
'water_mask':r"D:\BaiduNetdiskDownload\yaobao\roi\roi.shp", # None表示自动使用步骤1生成的掩膜也可以提供
'water_mask_path':r"D:\BaiduNetdiskDownload\yaobao\roi\roi.shp", # None表示自动使用步骤1生成的掩膜也可以提供
# # - numpy数组
# # - 栅格文件路径(.dat/.tif)
# # - shapefile路径(.shp)

View File

@ -0,0 +1,430 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
图表与交互弹窗模块
包含 ChartViewerDialog、ChartBrowserDialog 和 InteractiveViewerDialog 类。
"""
import numpy as np
from PyQt5.QtWidgets import (
QDialog, QVBoxLayout, QHBoxLayout, QPushButton,
QSizePolicy, QFileDialog, QMessageBox, QGroupBox,
QListWidget, QLabel, QComboBox, QCheckBox,
)
from PyQt5.QtCore import Qt
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT as NavigationToolbar
from matplotlib.figure import Figure
class ChartViewerDialog(QDialog):
"""图表查看器对话框"""
def __init__(self, title="图表查看器", parent=None):
super().__init__(parent)
self.setWindowTitle(title)
self.resize(1000, 700)
self.init_ui()
def init_ui(self):
layout = QVBoxLayout()
self.figure = Figure(figsize=(10, 7))
self.canvas = FigureCanvas(self.figure)
self.canvas.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Expanding)
self.toolbar = NavigationToolbar(self.canvas, self)
layout.addWidget(self.toolbar)
layout.addWidget(self.canvas)
btn_layout = QHBoxLayout()
self.save_btn = QPushButton("保存图表")
self.save_btn.clicked.connect(self.save_chart)
btn_layout.addWidget(self.save_btn)
btn_layout.addStretch()
self.close_btn = QPushButton("关闭")
self.close_btn.clicked.connect(self.close)
btn_layout.addWidget(self.close_btn)
layout.addLayout(btn_layout)
self.setLayout(layout)
def display_image(self, image_path):
"""显示图片"""
self.figure.clear()
ax = self.figure.add_subplot(111)
try:
import matplotlib.image as mpimg
img = mpimg.imread(image_path)
ax.imshow(img)
ax.axis('off')
self.figure.tight_layout()
self.canvas.draw()
self.current_image_path = image_path
except Exception as e:
ax.text(0.5, 0.5, f'加载图片失败:\n{str(e)}',
ha='center', va='center', transform=ax.transAxes)
self.canvas.draw()
def display_custom_plot(self, plot_func):
"""显示自定义绘图函数"""
self.figure.clear()
try:
plot_func(self.figure)
self.canvas.draw()
except Exception as e:
ax = self.figure.add_subplot(111)
ax.text(0.5, 0.5, f'绘图失败:\n{str(e)}',
ha='center', va='center', transform=ax.transAxes)
self.canvas.draw()
def save_chart(self):
"""保存图表"""
file_path, _ = QFileDialog.getSaveFileName(
self, "保存图表", "",
"PNG图片 (*.png);;JPG图片 (*.jpg);;PDF文件 (*.pdf);;所有文件 (*.*)"
)
if file_path:
try:
self.figure.savefig(file_path, dpi=300, bbox_inches='tight')
QMessageBox.information(self, "成功", f"图表已保存到:\n{file_path}")
except Exception as e:
QMessageBox.critical(self, "错误", f"保存失败:\n{str(e)}")
class ChartBrowserDialog(QDialog):
"""图表浏览器对话框"""
def __init__(self, chart_files, parent=None):
super().__init__(parent)
self.chart_files = sorted(chart_files, key=lambda x: x.stat().st_mtime, reverse=True)
self.current_index = 0
self.setWindowTitle("图表浏览器")
self.resize(1200, 800)
self.init_ui()
self.show_chart(0)
def init_ui(self):
layout = QVBoxLayout()
list_group = QGroupBox(f"图表列表 (共 {len(self.chart_files)} 个)")
list_layout = QHBoxLayout()
self.chart_list = QListWidget()
self.chart_list.setMaximumHeight(150)
for chart_file in self.chart_files:
self.chart_list.addItem(chart_file.name)
self.chart_list.currentRowChanged.connect(self.show_chart)
list_layout.addWidget(self.chart_list)
list_group.setLayout(list_layout)
layout.addWidget(list_group)
self.figure = Figure(figsize=(12, 8))
self.canvas = FigureCanvas(self.figure)
self.canvas.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Expanding)
self.toolbar = NavigationToolbar(self.canvas, self)
layout.addWidget(self.toolbar)
layout.addWidget(self.canvas, 1)
btn_layout = QHBoxLayout()
self.prev_btn = QPushButton("◀ 上一个")
self.prev_btn.clicked.connect(self.prev_chart)
btn_layout.addWidget(self.prev_btn)
self.next_btn = QPushButton("下一个 >")
self.next_btn.clicked.connect(self.next_chart)
btn_layout.addWidget(self.next_btn)
btn_layout.addStretch()
self.save_btn = QPushButton("💾 保存当前图表")
self.save_btn.clicked.connect(self.save_current_chart)
btn_layout.addWidget(self.save_btn)
self.close_btn = QPushButton("关闭")
self.close_btn.clicked.connect(self.close)
btn_layout.addWidget(self.close_btn)
layout.addLayout(btn_layout)
self.setLayout(layout)
def show_chart(self, index):
"""显示指定索引的图表"""
if 0 <= index < len(self.chart_files):
self.current_index = index
self.chart_list.setCurrentRow(index)
chart_file = self.chart_files[index]
self.figure.clear()
ax = self.figure.add_subplot(111)
try:
import matplotlib.image as mpimg
img = mpimg.imread(str(chart_file))
ax.imshow(img)
ax.axis('off')
ax.set_title(chart_file.name, fontsize=12, pad=10)
self.figure.tight_layout()
self.canvas.draw()
except Exception as e:
ax.text(0.5, 0.5, f'加载图片失败:\n{str(e)}',
ha='center', va='center', transform=ax.transAxes)
self.canvas.draw()
self.prev_btn.setEnabled(index > 0)
self.next_btn.setEnabled(index < len(self.chart_files) - 1)
def prev_chart(self):
"""上一个图表"""
if self.current_index > 0:
self.show_chart(self.current_index - 1)
def next_chart(self):
"""下一个图表"""
if self.current_index < len(self.chart_files) - 1:
self.show_chart(self.current_index + 1)
def save_current_chart(self):
"""保存当前图表"""
if 0 <= self.current_index < len(self.chart_files):
current_file = self.chart_files[self.current_index]
file_path, _ = QFileDialog.getSaveFileName(
self, "保存图表", current_file.name,
"PNG图片 (*.png);;JPG图片 (*.jpg);;所有文件 (*.*)"
)
if file_path:
try:
import shutil
shutil.copy(str(current_file), file_path)
QMessageBox.information(self, "成功", f"图表已保存到:\n{file_path}")
except Exception as e:
QMessageBox.critical(self, "错误", f"保存失败:\n{str(e)}")
class InteractiveViewerDialog(QDialog):
"""交互式影像预览对话框:显示影像、参考点散点图、点击查询坐标/值"""
def __init__(self, parent, img_path, ref_csv=None):
super().__init__(parent)
self.img_path = img_path
self.ref_csv = ref_csv
self.geotransform = None
self.fig = None
self.canvas = None
self.ax = None
self.status_label = None
self.init_ui()
def init_ui(self):
self.setWindowTitle("👁️ 交互式影像预览")
self.setMinimumSize(900, 700)
layout = QVBoxLayout()
toolbar = QHBoxLayout()
self.band_combo = QComboBox()
self.band_combo.currentIndexChanged.connect(self.on_band_changed)
toolbar.addWidget(QLabel("显示波段:"))
toolbar.addWidget(self.band_combo)
self.gray_check = QCheckBox("灰度显示")
self.gray_check.stateChanged.connect(self.on_band_changed)
toolbar.addWidget(self.gray_check)
toolbar.addStretch()
layout.addLayout(toolbar)
try:
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.figure import Figure
import matplotlib
matplotlib.use('Qt5Agg')
self.fig = Figure(figsize=(10, 8))
self.canvas = FigureCanvas(self.fig)
self.ax = self.fig.add_subplot(111)
self.fig.tight_layout()
layout.addWidget(self.canvas)
self.load_and_display()
except ImportError as e:
layout.addWidget(QLabel(f"Matplotlib 未安装: {e}"))
self.status_label = QLabel("点击影像查看像素坐标和经纬度")
self.status_label.setStyleSheet("background:#f0f0f0;padding:4px;font-size:12px;")
self.status_label.setWordWrap(True)
layout.addWidget(self.status_label)
close_btn = QPushButton("关闭")
close_btn.clicked.connect(self.close)
layout.addWidget(close_btn)
self.setLayout(layout)
def load_and_display(self):
"""加载影像并显示"""
from osgeo import gdal
dataset = gdal.Open(self.img_path)
if dataset is None:
self.status_label.setText(f"无法打开影像: {self.img_path}")
return
self.geotransform = dataset.GetGeoTransform()
self.projection = dataset.GetProjection()
n_bands = dataset.RasterCount
self.height = dataset.RasterYSize
self.width = dataset.RasterXSize
self.band_combo.clear()
if n_bands >= 3:
for i in range(1, n_bands + 1):
self.band_combo.addItem(f"RGB (B{i-0}, G{i-1}, R{i-2})" if i >= 3 else f"波段 {i}", i)
self.band_combo.addItem(f"单波段 (B1)", 0)
else:
for i in range(1, n_bands + 1):
self.band_combo.addItem(f"波段 {i}", i - 1)
self.band_combo.setCurrentIndex(0)
self.dataset = dataset
self.display_band(0, is_gray=False)
self.load_ref_points()
def display_band(self, band_idx, is_gray=False):
"""显示指定波段组合"""
from osgeo import gdal
import numpy as np
dataset = self.dataset
self.ax.clear()
if is_gray or (self.band_combo.currentData() == 0 and dataset.RasterCount == 1):
band = dataset.GetRasterBand(1 if band_idx == 0 else band_idx + 1)
data = band.ReadAsArray()
data = np.nan_to_num(data, nan=0.0)
self.ax.imshow(data, cmap='gray')
self.ax.set_title(f"波段 {band_idx + 1} (灰度)")
else:
n = min(3, dataset.RasterCount)
bands_data = []
for i in range(n):
b = dataset.GetRasterBand(i + 1)
bd = b.ReadAsArray()
bd = np.nan_to_num(bd, nan=0.0)
bands_data.append(bd)
rgb = np.dstack(bands_data)
for i in range(rgb.shape[2]):
p2, p98 = np.percentile(rgb[:, :, i], [2, 98])
if p98 > p2:
rgb[:, :, i] = np.clip((rgb[:, :, i] - p2) / (p98 - p2), 0, 1)
else:
rgb[:, :, i] = np.clip(rgb[:, :, i] / (p98 + 1e-6), 0, 1)
self.ax.imshow(rgb)
self.ax.set_title(f"RGB 显示")
self.ax.set_xlabel("列 (Column)")
self.ax.set_ylabel("行 (Row)")
self.fig.tight_layout()
self.canvas.draw()
self.cid = self.canvas.mpl_connect('button_press_event', self.on_click)
def on_band_changed(self):
"""波段选择变化时更新显示"""
if not hasattr(self, 'dataset'):
return
is_gray = self.gray_check.isChecked()
band_data = self.band_combo.currentData()
self.display_band(band_data if band_data != 0 else 0, is_gray=is_gray)
def load_ref_points(self):
"""加载并显示参考点"""
import os
if not self.ref_csv or not os.path.isfile(self.ref_csv):
return
try:
import csv
lon_list, lat_list = [], []
with open(self.ref_csv, 'r', encoding='utf-8-sig') as f:
reader = csv.DictReader(f)
for row in reader:
try:
lon = float(row.get('Lon', row.get('lon', row.get('LON', 0))))
lat = float(row.get('Lat', row.get('lat', row.get('LAT', 0))))
if lon and lat:
lon_list.append(lon)
lat_list.append(lat)
except (ValueError, TypeError):
continue
if not lon_list:
return
px_list, py_list = [], []
gt = self.geotransform
if gt and (gt[1] != 0 or gt[5] != 0):
for lon, lat in zip(lon_list, lat_list):
px = (lon - gt[0]) / gt[1]
py = (lat - gt[3]) / gt[5]
if 0 <= px < self.width and 0 <= py < self.height:
px_list.append(px)
py_list.append(py)
if px_list:
self.ax.scatter(px_list, py_list, c='red', s=40, marker='o',
edgecolors='white', linewidths=0.8, zorder=5, alpha=0.9,
label=f'参考点 ({len(px_list)}个)')
self.ax.legend(loc='upper right', fontsize=9)
self.fig.tight_layout()
self.canvas.draw()
self.status_label.setText(
f"已加载 {len(px_list)} 个参考点(仅显示在影像范围内的点)"
)
except Exception as e:
self.status_label.setText(f"加载参考点失败: {e}")
def pixel_to_geo(self, px, py):
"""像素坐标转经纬度"""
gt = self.geotransform
if gt is None:
return None, None
lon = gt[0] + px * gt[1] + py * gt[2]
lat = gt[3] + px * gt[4] + py * gt[5]
return lon, lat
def on_click(self, event):
"""鼠标点击事件"""
if event.inaxes != self.ax or event.xdata is None or event.ydata is None:
return
px, py = int(round(event.xdata)), int(round(event.ydata))
if not (0 <= px < self.width and 0 <= py < self.height):
return
from osgeo import gdal
import numpy as np
dataset = self.dataset
n_bands = dataset.RasterCount
vals = []
for b in range(1, n_bands + 1):
val = dataset.GetRasterBand(b).ReadAsArray()[py, px]
vals.append(f"{val:.4f}" if isinstance(val, float) else str(val))
lon, lat = self.pixel_to_geo(px, py)
geo_str = f"Lon={lon:.6f}, Lat={lat:.6f}" if lon is not None else "无地理参考"
self.status_label.setText(
f"像素: (行={py}, 列={px}) | {geo_str} | "
f"波段值: {' | '.join(vals[:5])}" +
(f" ... ({n_bands}波段的更多信息)" if n_bands > 5 else "")
)

View File

@ -0,0 +1,50 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
数据模型模块
包含 PandasTableModel 等数据模型类。
"""
import pandas as pd
from PyQt5.QtCore import Qt, QAbstractTableModel
class PandasTableModel(QAbstractTableModel):
"""支持DataFrame的表格模型"""
def __init__(self, data_frame: pd.DataFrame):
super().__init__()
self._data = data_frame.copy()
if self._data.empty:
self._data = pd.DataFrame()
self._data.fillna("", inplace=True)
self._columns = [str(col) for col in self._data.columns]
def rowCount(self, parent=None):
return len(self._data)
def columnCount(self, parent=None):
return len(self._columns)
def data(self, index, role=Qt.DisplayRole):
if not index.isValid() or role != Qt.DisplayRole:
return None
value = self._data.iat[index.row(), index.column()]
if pd.isna(value):
return ""
return str(value)
def headerData(self, section, orientation, role=Qt.DisplayRole):
if role != Qt.DisplayRole:
return None
if orientation == Qt.Horizontal:
if section < len(self._columns):
return self._columns[section]
return str(section)
return str(section + 1)
def flags(self, index):
if not index.isValid():
return Qt.NoItemFlags
return Qt.ItemIsEnabled | Qt.ItemIsSelectable

View File

@ -0,0 +1,351 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
图像浏览组件模块
包含 ImageCategoryTree 和 ImageViewerWidget 类。
"""
import os
from pathlib import Path
from typing import List, Optional
from PyQt5.QtWidgets import (
QTreeWidget, QTreeWidgetItem, QWidget, QVBoxLayout, QHBoxLayout,
QPushButton, QLabel, QScrollArea, QFrame, QGroupBox,
QFileDialog, QMessageBox,
)
from PyQt5.QtCore import Qt, QTimer
from PyQt5.QtGui import QPixmap
class ImageCategoryTree(QTreeWidget):
"""图像分类目录树 - 按类别组织图像文件"""
CATEGORIES = [
("模型评估", ["scatter", "regression", "validation", "r2", "rmse"], "📊"),
("光谱分析", ["spectrum", "spectral", "band", "wavelength"], "📈"),
("统计图表", ["boxplot", "histogram", "heatmap", "statistics", "stats"], "📉"),
("处理结果", ["mask", "glint", "deglint", "preview", "overlay", "water_mask"], "🖼️"),
("含量分布图", [], "📁"),
]
def __init__(self, parent=None):
super().__init__(parent)
self.setHeaderLabel("图像目录")
self.setMaximumWidth(300)
self.setMinimumWidth(250)
self.setup_categories()
self.setStyleSheet("""
QTreeWidget {
border: 1px solid #ddd;
border-radius: 5px;
background-color: #f8f9fa;
}
QTreeWidget::item {
padding: 5px;
border-radius: 3px;
}
QTreeWidget::item:selected {
background-color: #0078D4;
color: white;
}
QTreeWidget::item:hover {
background-color: #e3f2fd;
}
""")
def setup_categories(self):
"""初始化类别节点"""
self.category_items = {}
for category_name, keywords, icon in self.CATEGORIES:
item = QTreeWidgetItem(self)
item.setText(0, f"{icon} {category_name}")
item.setData(0, Qt.UserRole, {"type": "category", "keywords": keywords, "name": category_name})
item.setExpanded(True)
self.category_items[category_name] = item
def clear_all_images(self):
"""清除所有图像项"""
for category_item in self.category_items.values():
while category_item.childCount() > 0:
category_item.removeChild(category_item.child(0))
def add_image(self, file_path: Path, display_name: str = None):
"""添加图像到对应的类别"""
if display_name is None:
display_name = file_path.stem
category = self._determine_category(file_path.name)
category_item = self.category_items.get(category, self.category_items["含量分布图"])
image_item = QTreeWidgetItem(category_item)
image_item.setText(0, f" └─ {display_name}")
image_item.setData(0, Qt.UserRole, {"type": "image", "path": str(file_path)})
image_item.setToolTip(0, str(file_path))
return image_item
def _determine_category(self, filename: str) -> str:
"""根据文件名确定类别"""
filename_lower = filename.lower()
for category_name, keywords, _ in self.CATEGORIES:
if any(keyword in filename_lower for keyword in keywords):
return category_name
return "含量分布图"
def scan_directory(self, work_dir: str):
"""扫描目录中的所有图像文件"""
self.clear_all_images()
work_path = Path(work_dir)
if not work_path.exists():
return
image_extensions = ['*.png', '*.jpg', '*.jpeg', '*.tif', '*.tiff', '*.bmp']
scan_roots: List[Path] = []
_viz = work_path / "14_visualization"
if _viz.is_dir():
scan_roots.append(_viz)
_wm = work_path / "1_water_mask"
if _wm.is_dir():
scan_roots.append(_wm)
if not scan_roots:
scan_roots.append(work_path)
seen_norm: set = set()
image_files: List[Path] = []
for root in scan_roots:
for ext in image_extensions:
for p in root.glob(f"**/{ext}"):
key = os.path.normcase(os.path.normpath(str(p.resolve())))
if key in seen_norm:
continue
seen_norm.add(key)
image_files.append(p)
for img_file in sorted(image_files):
if img_file.name.startswith('.') or 'thumb' in img_file.name.lower():
continue
self.add_image(img_file)
for category_name, item in self.category_items.items():
count = item.childCount()
if count > 0:
for cat_name, _, icon in self.CATEGORIES:
if cat_name == category_name:
item.setText(0, f"{icon} {category_name} ({count})")
break
def get_selected_image_path(self) -> Optional[str]:
"""获取当前选中的图像路径"""
selected_item = self.currentItem()
if not selected_item:
return None
data = selected_item.data(0, Qt.UserRole)
if data and data.get("type") == "image":
return data.get("path")
return None
class ImageViewerWidget(QWidget):
"""图像查看器组件 - 支持缩放、平移"""
def __init__(self, parent=None):
super().__init__(parent)
self.current_image_path = None
self.scale_factor = 1.0
self._update_timer = QTimer()
self._update_timer.setSingleShot(True)
self._update_timer.timeout.connect(self._do_update_display)
self._pending_scale = None
self.setup_ui()
def setup_ui(self):
layout = QVBoxLayout()
layout.setContentsMargins(0, 0, 0, 0)
toolbar = QHBoxLayout()
self.refresh_btn = QPushButton("🔄 刷新目录")
self.refresh_btn.setToolTip("重新扫描工作目录中的图像文件")
toolbar.addWidget(self.refresh_btn)
separator = QFrame()
separator.setFrameShape(QFrame.VLine)
separator.setFrameShadow(QFrame.Sunken)
toolbar.addWidget(separator)
self.zoom_in_btn = QPushButton("🔍+")
self.zoom_in_btn.setToolTip("放大")
self.zoom_in_btn.setMaximumWidth(50)
toolbar.addWidget(self.zoom_in_btn)
self.zoom_out_btn = QPushButton("🔍-")
self.zoom_out_btn.setToolTip("缩小")
self.zoom_out_btn.setMaximumWidth(50)
toolbar.addWidget(self.zoom_out_btn)
self.fit_btn = QPushButton("⬜ 适应窗口")
self.fit_btn.setToolTip("适应窗口大小")
toolbar.addWidget(self.fit_btn)
self.original_btn = QPushButton("1:1 原始大小")
self.original_btn.setToolTip("原始大小")
toolbar.addWidget(self.original_btn)
toolbar.addStretch()
self.save_btn = QPushButton("💾 保存")
self.save_btn.setToolTip("保存当前图像")
toolbar.addWidget(self.save_btn)
layout.addLayout(toolbar)
self.scroll_area = QScrollArea()
self.scroll_area.setWidgetResizable(True)
self.scroll_area.setStyleSheet("background-color: white;")
self.image_label = QLabel()
self.image_label.setAlignment(Qt.AlignCenter)
self.image_label.setStyleSheet("background-color: white;")
self.scroll_area.setWidget(self.image_label)
layout.addWidget(self.scroll_area, 1)
status_layout = QHBoxLayout()
self.status_label = QLabel("就绪")
self.status_label.setStyleSheet("color: #666; font-size: 11px;")
status_layout.addWidget(self.status_label)
status_layout.addStretch()
layout.addLayout(status_layout)
self.setLayout(layout)
self.zoom_in_btn.clicked.connect(self.zoom_in)
self.zoom_out_btn.clicked.connect(self.zoom_out)
self.fit_btn.clicked.connect(self.fit_to_window)
self.original_btn.clicked.connect(self.original_size)
self.save_btn.clicked.connect(self.save_image)
def load_image(self, image_path: str):
"""加载并显示图像"""
if not image_path or not Path(image_path).exists():
self.image_label.setText("图像不存在")
self.status_label.setText("图像加载失败")
return
self.current_image_path = image_path
self.scale_factor = 1.0
pixmap = QPixmap(image_path)
if pixmap.isNull():
self.image_label.setText("无法加载图像")
self.status_label.setText("图像格式不支持")
return
self.original_pixmap = pixmap
self.fit_to_window()
file_info = Path(image_path).stat()
size_mb = file_info.st_size / (1024 * 1024)
self.status_label.setText(f"{pixmap.width()}x{pixmap.height()} | {size_mb:.2f} MB | {Path(image_path).name} | 适应窗口")
def update_image_display(self):
"""更新图像显示 - 使用防抖避免频繁重绘卡顿"""
self._update_timer.stop()
self._pending_scale = self.scale_factor
self._update_timer.start(50)
def _do_update_display(self):
"""实际执行图像更新"""
if not hasattr(self, 'original_pixmap') or self.original_pixmap.isNull():
return
if self._pending_scale is None:
return
if self._pending_scale > 2.0 or self._pending_scale < 0.5:
transform = Qt.FastTransformation
else:
transform = Qt.SmoothTransformation
scaled_pixmap = self.original_pixmap.scaled(
int(self.original_pixmap.width() * self._pending_scale),
int(self.original_pixmap.height() * self._pending_scale),
Qt.KeepAspectRatio,
transform
)
self.image_label.setPixmap(scaled_pixmap)
self._pending_scale = None
def wheelEvent(self, event):
"""鼠标滚轮缩放 - 实时响应"""
delta = event.angleDelta().y()
if delta > 0:
if self.scale_factor < 5.0:
self.scale_factor = min(self.scale_factor * 1.1, 5.0)
self.update_image_display()
else:
if self.scale_factor > 0.1:
self.scale_factor = max(self.scale_factor / 1.1, 0.1)
self.update_image_display()
event.accept()
def zoom_in(self):
"""放大"""
if self.scale_factor < 5.0:
self.scale_factor = min(self.scale_factor * 1.25, 5.0)
self.update_image_display()
def zoom_out(self):
"""缩小"""
if self.scale_factor > 0.1:
self.scale_factor = max(self.scale_factor / 1.25, 0.1)
self.update_image_display()
def fit_to_window(self):
"""适应窗口"""
if not hasattr(self, 'original_pixmap') or self.original_pixmap.isNull():
return
view_size = self.scroll_area.viewport().size()
img_size = self.original_pixmap.size()
scale_w = view_size.width() / img_size.width()
scale_h = view_size.height() / img_size.height()
self._fit_scale = min(scale_w, scale_h)
self.scale_factor = self._fit_scale
self.update_image_display()
self.status_label.setText(f"适应窗口 | 缩放: {self.scale_factor:.1%}")
def original_size(self):
"""原始大小"""
self.scale_factor = 1.0
self._fit_scale = None
self.update_image_display()
self.status_label.setText("原始大小 | 缩放: 100%")
def save_image(self):
"""保存图像"""
if not self.current_image_path:
return
file_path, _ = QFileDialog.getSaveFileName(
self, "保存图像", Path(self.current_image_path).name,
"PNG图片 (*.png);;JPG图片 (*.jpg);;所有文件 (*.*)"
)
if file_path:
try:
import shutil
shutil.copy(self.current_image_path, file_path)
except Exception as e:
QMessageBox.critical(self, "错误", f"保存失败: {e}")

View File

@ -0,0 +1,112 @@
import time
import warnings
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_regression
# 屏蔽烦人的 sklearn 警告
warnings.filterwarnings("ignore")
print("====== 🚀 启动 Mega Water 模型终极体检脚本 ======")
# ---------------------------------------------------------
# 1. 完美复刻侦察报告中的 CSV 数据结构
# 报告指出: 目标值(y)在左边,光谱特征(X)在右边
# ---------------------------------------------------------
print("📦 正在生成符合系统结构的模拟测试数据...")
X_raw, y_raw = make_regression(n_samples=200, n_features=50, noise=0.1, random_state=42)
# 模拟真实的 CSV 列名前2列是水质参数后面是 50 个光谱波段
columns = ['Chla', 'SS'] + [f"Band_{i}" for i in range(50)]
# 拼装成一整张大表
data = pd.DataFrame(np.hstack((y_raw.reshape(-1, 1), (y_raw * 0.5).reshape(-1, 1), X_raw)), columns=columns)
# 按照 load_data_batch 的逻辑进行切割
feature_start_index = 2
X = data.iloc[:, feature_start_index:] # 截取光谱作为 X
y = data['Chla'] # 提取一个目标参数作为 y
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
print(f"✅ 数据切割完毕! 模拟波段数: {X.shape[1]}, 训练集样本数: {X_train.shape[0]}\n")
# ---------------------------------------------------------
# 2. 严格装载侦察报告中的 16 个真实模型
# ---------------------------------------------------------
print("🔍 正在加载底层真实配置库中的模型...")
from sklearn.svm import SVR
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor, AdaBoostRegressor, ExtraTreesRegressor
from sklearn.neighbors import KNeighborsRegressor
from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet
from sklearn.cross_decomposition import PLSRegression
from sklearn.tree import DecisionTreeRegressor
from sklearn.neural_network import MLPRegressor
# 将参数压至极低,实施“降维打击”,确保 1 秒内跑完
models = {
'SVR': SVR(),
'RF': RandomForestRegressor(n_estimators=10, max_depth=5, n_jobs=-1),
'KNN': KNeighborsRegressor(),
'LinearRegression': LinearRegression(),
'Ridge': Ridge(),
'Lasso': Lasso(),
'ElasticNet': ElasticNet(),
'PLS': PLSRegression(),
'GradientBoosting': GradientBoostingRegressor(n_estimators=10, max_depth=5),
'AdaBoost': AdaBoostRegressor(n_estimators=10),
'DecisionTree': DecisionTreeRegressor(max_depth=5),
'MLP': MLPRegressor(max_iter=50),
'ExtraTrees': ExtraTreesRegressor(n_estimators=10, max_depth=5, n_jobs=-1)
}
# 针对报告中发现的 3 个“被禁用”的第三方强力库,进行刺探测试
try:
from xgboost import XGBRegressor
models['XGBoost'] = XGBRegressor(n_estimators=10, max_depth=5, n_jobs=-1)
except ImportError:
models['XGBoost'] = "IMPORT_ERROR"
try:
from lightgbm import LGBMRegressor
models['LightGBM'] = LGBMRegressor(n_estimators=10, max_depth=5, n_jobs=-1)
except ImportError:
models['LightGBM'] = "IMPORT_ERROR"
try:
from catboost import CatBoostRegressor
models['CatBoost'] = CatBoostRegressor(iterations=10, depth=5, verbose=0)
except ImportError:
models['CatBoost'] = "IMPORT_ERROR"
# ---------------------------------------------------------
# 3. 开始残酷的体检循环
# ---------------------------------------------------------
print("\n================ 开始跑分测试 ================")
results = []
for name, model in models.items():
if model == "IMPORT_ERROR":
results.append(f"⚠️ [缺库] {name:<16} : 环境未安装此库 (建议: pip install {name.lower()})")
continue
start_time = time.time()
try:
# 极速拟合与评分
model.fit(X_train, y_train)
score = model.score(X_test, y_test)
cost_time = time.time() - start_time
results.append(f"✅ [成功] {name:<16} : 耗时 {cost_time:.3f} 秒 (R2: {score:.2f})")
except Exception as e:
error_msg = str(e).split('\n')[0]
results.append(f"❌ [崩溃] {name:<16} : {error_msg}")
# ---------------------------------------------------------
# 4. 打印最终体检报告
# ---------------------------------------------------------
print("\n=============== 🏥 最终体检报告 ===============")
for res in results:
print(res)
print("===============================================")

346
src/gui/core/viz_thread.py Normal file
View File

@ -0,0 +1,346 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
可视化后台线程模块
包含 VisualizationWorkerThread 后台线程类和辅助函数。
"""
from pathlib import Path
from typing import Optional, List, Union
from PyQt5.QtCore import QThread, pyqtSignal
import numpy as np
def _viz_infer_wavelength_start_column(df) -> Union[str, int]:
"""推断光谱起始列training_spectra 通常以波长数值为列名,未必含 UTM_Y"""
import pandas as pd
for i, col in enumerate(df.columns):
name = str(col).strip().lstrip("\ufeff")
try:
v = float(name)
except ValueError:
continue
if 200.0 <= v <= 3000.0:
return i
if "UTM_Y" in df.columns:
return "UTM_Y"
return 0
class VisualizationWorkerThread(QThread):
"""可视化耗时计算放入后台线程,并临时使用 Agg 后端,避免主界面未响应。"""
finished_ok = pyqtSignal(object)
failed = pyqtSignal(str)
def __init__(self, task: str, work_dir: str, extra: Optional[dict] = None):
super().__init__()
self.task = task
self.work_dir = str(work_dir)
self.extra = extra or {}
def run(self):
mpl_prev = None
try:
import matplotlib
mpl_prev = matplotlib.get_backend()
except Exception:
pass
try:
import matplotlib.pyplot as plt
plt.switch_backend("Agg")
except Exception:
mpl_prev = None
try:
wp = Path(self.work_dir)
if self.task == "mask_glint":
from src.postprocessing.visualization_reports import WaterQualityVisualization
viz = WaterQualityVisualization(output_dir=str(wp / "14_visualization"))
preview_paths = viz.generate_glint_deglint_previews(
work_dir=str(wp),
output_subdir="glint_deglint_previews",
)
cnt = len(preview_paths) if preview_paths else 0
self.finished_ok.emit({"task": "mask_glint", "count": cnt, "preview_paths": preview_paths})
elif self.task == "sampling_map":
hyperspectral_files = []
deglint_dir = wp / "3_deglint"
if deglint_dir.exists():
for ext in ("*.dat", "*.bsq", "*.tif", "*.tiff"):
hyperspectral_files.extend(list(deglint_dir.glob(ext)))
if not hyperspectral_files:
for ext in ("*.dat", "*.bsq", "*.tif", "*.tiff"):
hyperspectral_files.extend(list(wp.glob(f"**/{ext}")))
if not hyperspectral_files:
self.failed.emit("未找到高光谱影像文件(.dat/.bsq/.tif")
return
hyperspectral_path = str(hyperspectral_files[0])
csv_files = []
processed_dir = wp / "4_processed_data"
if processed_dir.exists():
csv_files = list(processed_dir.glob("*.csv"))
if not csv_files:
csv_files = (
list(wp.glob("**/*sampling*.csv"))
+ list(wp.glob("**/*point*.csv"))
+ list(wp.glob("**/*.csv"))
)
if not csv_files:
self.failed.emit("未找到采样点 CSV 文件。")
return
csv_path = str(csv_files[0])
from src.postprocessing.point_map import SamplingPointMap
map_generator = SamplingPointMap(
output_dir=str(wp / "14_visualization" / "sampling_maps"),
fast_mode=True,
)
map_path = map_generator.create_sampling_point_map(
hyperspectral_path=hyperspectral_path,
csv_path=csv_path,
point_color="red",
point_size=100,
point_alpha=0.9,
show_north_arrow=True,
show_scale_bar=True,
show_legend=True,
downsample=True,
dpi=180,
)
self.finished_ok.emit(
{
"task": "sampling_map",
"map_path": map_path,
"hyperspectral_path": hyperspectral_path,
"csv_path": csv_path,
}
)
elif self.task == "spectrum":
from src.postprocessing.visualization_reports import WaterQualityVisualization
viz = WaterQualityVisualization(output_dir=str(wp / "14_visualization"))
csv_file = self.extra.get("csv_path")
wl = self.extra.get("wavelength_start_column", "UTM_Y")
n_groups = int(self.extra.get("n_groups", 5))
param_cols = self.extra.get("param_cols") or []
if param_cols:
output_paths: List[str] = []
err_lines: List[str] = []
for param_col in param_cols:
try:
out = viz.plot_spectrum_by_parameter(
csv_path=str(csv_file),
parameter_column=param_col,
wavelength_start_column=wl,
n_groups=n_groups,
)
output_paths.append(out)
except Exception as _ex:
err_lines.append(f"{param_col}: {_ex}")
if not output_paths:
self.failed.emit(
"所有参数列的光谱图均生成失败:\n" + "\n".join(err_lines[:20])
)
return
self.finished_ok.emit(
{
"task": "spectrum",
"output_paths": output_paths,
"errors": err_lines,
}
)
else:
param_col = self.extra.get("param_col")
out = viz.plot_spectrum_by_parameter(
csv_path=str(csv_file),
parameter_column=param_col,
wavelength_start_column=wl,
n_groups=n_groups,
)
self.finished_ok.emit(
{"task": "spectrum", "output_path": out, "param_col": param_col}
)
elif self.task == "statistics":
from src.postprocessing.visualization_reports import WaterQualityVisualization
viz = WaterQualityVisualization(output_dir=str(wp / "14_visualization"))
csv_file = self.extra.get("csv_path")
param_cols = self.extra.get("param_cols") or []
output_paths = viz.plot_statistical_charts(
csv_path=str(csv_file),
parameter_columns=param_cols,
)
self.finished_ok.emit(
{"task": "statistics", "output_paths": output_paths}
)
elif self.task == "scatter":
from src.core.water_quality_inversion_pipeline_GUI import WaterQualityInversionPipeline
training_csv_path = (self.extra.get("training_csv_path") or "").strip()
models_dir = (self.extra.get("models_dir") or "").strip()
if not training_csv_path or not Path(training_csv_path).is_file():
self.failed.emit("训练光谱 CSV 无效或不存在请确认已选择步骤5输出的文件。")
return
if not models_dir or not Path(models_dir).is_dir():
self.failed.emit("模型目录无效或不存在请确认步骤6已生成 7_Supervised_Model_Training 下的参数子文件夹。")
return
pipeline = WaterQualityInversionPipeline(work_dir=str(wp))
scatter_paths = pipeline.generate_model_scatter_plots(
training_csv_path=training_csv_path,
models_dir=models_dir,
)
self.finished_ok.emit({"task": "scatter", "scatter_paths": scatter_paths or {}})
elif self.task == "generate_all_selected":
from src.postprocessing.visualization_reports import WaterQualityVisualization
viz = WaterQualityVisualization(output_dir=str(wp / "14_visualization"))
parts = []
training_csv = wp / "5_training_spectra" / "training_spectra.csv"
if self.extra.get("gen_scatter"):
if training_csv.is_file():
models_dir = wp / "7_Supervised_Model_Training"
if models_dir.is_dir() and any(d.is_dir() for d in models_dir.iterdir()):
from src.core.water_quality_inversion_pipeline_GUI import WaterQualityInversionPipeline
pipeline = WaterQualityInversionPipeline(work_dir=str(wp))
scatter_paths = pipeline.generate_model_scatter_plots(
training_csv_path=str(training_csv),
models_dir=str(models_dir),
)
count = len(scatter_paths) if scatter_paths else 0
parts.append(f"散点图: {count}")
else:
parts.append("散点图: 跳过(无模型目录)")
else:
parts.append("散点图: 跳过(无训练数据)")
if self.extra.get("gen_spectrum"):
if training_csv.is_file():
import pandas as pd
df = pd.read_csv(training_csv)
wl_col = _viz_infer_wavelength_start_column(df)
if isinstance(wl_col, str):
idx = int(df.columns.get_loc(wl_col)) + 1
else:
idx = int(wl_col)
param_cols = []
if idx > 0 and idx < len(df.columns):
param_cols = [
c for c in df.columns[:idx]
if df[c].dtype.kind in 'iuf' and df[c].notna().sum() > 0
]
if param_cols:
spectrum_paths = []
for param_col in param_cols:
try:
path = viz.plot_spectrum_by_parameter(
csv_path=str(training_csv),
parameter_column=param_col,
wavelength_start_column=wl_col,
n_groups=5,
)
if path:
spectrum_paths.append(path)
except Exception as e:
print(f"生成光谱图失败 ({param_col}): {e}")
count = len(spectrum_paths)
parts.append(f"光谱图: {count}")
else:
parts.append("光谱图: 跳过(无可用参数列)")
else:
parts.append("光谱图: 跳过(无训练数据)")
if self.extra.get("gen_boxplots"):
if training_csv.is_file():
import pandas as pd
df = pd.read_csv(training_csv)
exclude_cols = ['longitude', 'latitude', 'lon', 'lat', 'x', 'y', 'coord', 'coordinate']
param_cols = [
c for c in df.select_dtypes(include=[np.number]).columns
if not any(exc in c.lower() for exc in exclude_cols)
]
wl = _viz_infer_wavelength_start_column(df)
if isinstance(wl, str):
idx = int(df.columns.get_loc(wl)) + 1
else:
idx = int(wl)
if 0 < idx < len(df.columns):
meta_set = set(df.columns[:idx])
param_cols = [c for c in param_cols if c in meta_set]
if param_cols:
output_dict = viz.plot_statistical_charts(
csv_path=str(training_csv),
parameter_columns=param_cols,
)
count = len([v for v in output_dict.values() if v]) if output_dict else 0
parts.append(f"统计图: {count}")
else:
parts.append("统计图: 跳过(无可用水质参数列)")
else:
parts.append("统计图: 跳过(无训练数据)")
if self.extra.get("gen_mask_glint"):
preview_paths = viz.generate_glint_deglint_previews(
work_dir=str(wp),
output_subdir="glint_deglint_previews",
)
parts.append(f"掩膜/耀斑预览: {len(preview_paths) if preview_paths else 0}")
if self.extra.get("gen_sampling_map"):
hyperspectral_files = []
deglint_dir = wp / "3_deglint"
if deglint_dir.exists():
for ext in ("*.dat", "*.bsq", "*.tif", "*.tiff"):
hyperspectral_files.extend(list(deglint_dir.glob(ext)))
if not hyperspectral_files:
for ext in ("*.dat", "*.bsq", "*.tif", "*.tiff"):
hyperspectral_files.extend(list(wp.glob(f"**/{ext}")))
if hyperspectral_files:
hyperspectral_path = str(hyperspectral_files[0])
csv_files = []
processed_dir = wp / "4_processed_data"
if processed_dir.exists():
csv_files = list(processed_dir.glob("*.csv"))
if not csv_files:
csv_files = (
list(wp.glob("**/*sampling*.csv"))
+ list(wp.glob("**/*point*.csv"))
+ list(wp.glob("**/*.csv"))
)
if csv_files:
csv_path = str(csv_files[0])
from src.postprocessing.point_map import SamplingPointMap
map_generator = SamplingPointMap(
output_dir=str(wp / "14_visualization" / "sampling_maps"),
fast_mode=True,
)
map_path = map_generator.create_sampling_point_map(
hyperspectral_path=hyperspectral_path,
csv_path=csv_path,
point_color="red",
point_size=100,
point_alpha=0.9,
show_north_arrow=True,
show_scale_bar=True,
show_legend=True,
downsample=True,
dpi=180,
)
parts.append(f"采样点图: {Path(map_path).name}")
else:
parts.append("采样点图: 跳过(无CSV)")
else:
parts.append("采样点图: 跳过(无影像)")
self.finished_ok.emit({"task": "generate_all_selected", "parts": parts})
else:
self.failed.emit(f"未知可视化任务: {self.task}")
except Exception as e:
import traceback
self.failed.emit(f"{e}\n{traceback.format_exc()}")
finally:
if mpl_prev:
try:
import matplotlib.pyplot as plt
plt.switch_backend(mpl_prev)
except Exception:
pass

View File

@ -247,16 +247,34 @@ class WorkerThread(QThread):
mpl_prev = None
try:
from src.core.water_quality_inversion_pipeline_GUI import WaterQualityInversionPipeline
from src.core.pipeline.runner import PipelineRunner
from src.core.pipeline.context import PipelineContext
self.pipeline = WaterQualityInversionPipeline(work_dir=self.work_dir)
if self.mode == 'full':
self.log_message.emit("开始运行完整流程...", "info")
self.step_count = 0
self.log_message.emit("开始运行完整流程 (Runner 调度模式)...", "info")
if hasattr(self.pipeline, 'set_callback'):
self.pipeline.set_callback(self.pipeline_callback)
self.pipeline.run_full_pipeline(self.config)
# 构造上下文 (Ctx),将 config 整体注入 user_config
ctx = PipelineContext(
img_path=self.config.get('step1', {}).get('img_path'),
water_mask_path=self.config.get('step1', {}).get('mask_path'),
csv_path=self.config.get('step4', {}).get('csv_path'),
boundary_path=self.config.get('step5', {}).get('boundary_path'),
boundary_shp_path=self.config.get('step9', {}).get('boundary_shp_path'),
formula_csv_path=self.config.get('step8_75', {}).get('formula_csv_path'),
work_dir=self.work_dir,
user_config=self.config
)
# 启动新调度器
runner = PipelineRunner(self.pipeline)
result_ctx = runner.run(ctx)
if result_ctx.last_error:
raise RuntimeError(f"流水线执行失败: {result_ctx.last_error}")
self.progress_update.emit(100, "流程执行完成")
self.finished.emit(True, "完整流程执行成功!")
@ -308,6 +326,18 @@ class WorkerThread(QThread):
method_name = step_method_map[step_name]
step_config = dict(config.get(step_name, {}))
# 透传面板顶层传入的外部预训练模型GUI step8_panel 通过 config['_external_model'] 传入)
# 非空才覆盖(遵循 feedback_never_overwrite_with_empty 原则)
for key in ('_external_model', '_external_model_path',
'_external_models_dict', '_external_model_dir'):
val = config.get(key)
if val is not None and val != "":
step_config[key] = val
if key == '_external_models_dict':
print(f"[Worker] 提取到的外部字典 Keys: {list(val.keys())}")
else:
print(f"[Worker] 透传 {key}: {val}")
step_config['skip_dependency_check'] = True
if step_name == 'step9':

147
src/gui/dialogs.py Normal file
View File

@ -0,0 +1,147 @@
# -*- coding: utf-8 -*-
"""
自定义确认对话框集合
"职责单一 + 不污染主 GUI 文件"原则拆分。
与 water_quality_gui.py 保持 1:1 风格(中文注释 / 顶部 encoding 声明)。
"""
from PyQt5.QtCore import Qt, QTimer
from PyQt5.QtGui import QFont
from PyQt5.QtWidgets import (
QDialog,
QLabel,
QSpinBox,
QPushButton,
QVBoxLayout,
QHBoxLayout,
QDialogButtonBox,
QWidget,
)
class BandConfirmDialog(QDialog):
"""波段越界智能确认对话框60 秒倒计时)
场景
----
用户在 step3 面板里设置了 nir_band=66但实际影像只有 50 波段。
Pipeline 一旦按 66 去取波段就会报 IndexError。
行为约定
--------
- 启动时 QTimer 开始 60s 倒计时,按钮文字同步显示 "确定 (Ns)"
- 用户手动调整 QSpinBox + 点"确定":立即 accept(),返回当前 spinbox 值。
- 用户 60s 未操作:定时器归零时自动 accept(),返回当前 spinbox 值
**默认值为影像最大波段数 = 用户拿不到想要波段时的兜底**)。
- 用户点"取消运行"reject(),调用方应中止 run_full_pipeline。
"""
DEFAULT_TIMEOUT = 60 # 秒
def __init__(
self,
parent: QWidget = None,
requested_band: int = 66,
max_band: int = 50,
recommended_band: int = 66,
method_label: str = "NIR",
timeout_seconds: int = DEFAULT_TIMEOUT,
):
super().__init__(parent)
self._requested_band = requested_band
self._max_band = max_band
self._recommended_band = recommended_band
self._method_label = method_label
self._timeout_seconds = timeout_seconds
self._remaining = timeout_seconds
self._selected_band = max_band # 默认 = 最大波段(兜底)
self.setWindowTitle("波段索引越界")
self.setModal(True)
self.setMinimumWidth(420)
self._init_ui()
self._start_timer()
def _init_ui(self):
"""搭建 UI警告文字 + 灰色推荐 + SpinBox + 倒计时按钮"""
layout = QVBoxLayout(self)
# 1) 主提示(带 HTML 强调)
self._msg_label = QLabel(
f"影像仅有 <b>{self._max_band}</b> 个波段,"
f"无法读取第 <b>{self._requested_band}</b> 波段({self._method_label})。"
)
self._msg_label.setWordWrap(True)
self._msg_label.setFont(QFont("Microsoft YaHei", 10))
layout.addWidget(self._msg_label)
# 2) 灰色小字推荐
hint_label = QLabel(
f"(推荐近红外波段序号:{self._recommended_band}"
)
hint_label.setStyleSheet("color: #888; font-size: 10px;")
layout.addWidget(hint_label)
# 3) 波段选择 SpinBox默认值 = 最大波段 = 超时兜底)
spin_row = QHBoxLayout()
spin_row.addWidget(QLabel(f"请选择要使用的{self._method_label}索引:"))
self._spin = QSpinBox()
self._spin.setRange(0, self._max_band)
self._spin.setValue(self._max_band)
self._spin.setSuffix(f" / 0~{self._max_band}")
spin_row.addWidget(self._spin)
spin_row.addStretch(1)
layout.addLayout(spin_row)
# 4) 倒计时说明
countdown_tip = QLabel(
f"{self._timeout_seconds} 秒内不操作,将自动使用最大波段 "
f"{self._max_band})继续运行。"
)
countdown_tip.setStyleSheet("color: #555; font-size: 9px;")
countdown_tip.setWordWrap(True)
layout.addWidget(countdown_tip)
# 5) 按钮组(手动"确定 (Ns)" + "取消运行"
btn_box = QDialogButtonBox()
self._ok_btn = QPushButton(f"确定 ({self._remaining}s)")
self._ok_btn.setDefault(True)
self._ok_btn.clicked.connect(self.accept)
self._cancel_btn = QPushButton("取消运行")
self._cancel_btn.clicked.connect(self.reject)
btn_box.addButton(self._ok_btn, QDialogButtonBox.AcceptRole)
btn_box.addButton(self._cancel_btn, QDialogButtonBox.RejectRole)
layout.addWidget(btn_box)
def _start_timer(self):
"""启动 1Hz 倒计时;归零时自动 accept()"""
self._timer = QTimer(self)
self._timer.setInterval(1000)
self._timer.timeout.connect(self._tick)
self._timer.start()
def _tick(self):
"""每秒刷新按钮文字;归零时停表 + accept()"""
self._remaining -= 1
if self._remaining <= 0:
self._timer.stop()
self.accept() # 超时:返回当前 spinbox 值(= max_band
else:
self._ok_btn.setText(f"确定 ({self._remaining}s)")
# ── 暴露给调用方的结果接口 ──────────────────────────────
def selected_band(self) -> int:
"""弹窗关闭后由调用方取回用户选定的波段索引"""
return self._selected_band
def accept(self):
""""确定"或倒计时归零触发:记录当前 spinbox 值后真正关闭"""
self._selected_band = self._spin.value()
self._timer.stop()
super().accept()
def reject(self):
""""取消运行"触发:停表 + 关闭,调用方需中止流程"""
self._timer.stop()
super().reject()

View File

@ -325,7 +325,7 @@ class Step3Panel(QWidget):
}
water_mask_path = self.water_mask_file.get_path()
if water_mask_path:
config['water_mask'] = water_mask_path
config['water_mask_path'] = water_mask_path
output_path = self.output_file.get_path()
if output_path:
config['output_path'] = output_path
@ -366,8 +366,8 @@ class Step3Panel(QWidget):
"""设置配置"""
if 'img_path' in config:
self.img_file.set_path(config['img_path'])
if 'water_mask' in config:
self.water_mask_file.set_path(config['water_mask'])
if 'water_mask_path' in config:
self.water_mask_file.set_path(config['water_mask_path'])
if 'output_path' in config:
self.output_file.set_path(config['output_path'])
if 'reference_csv' in config:

View File

@ -187,7 +187,7 @@ class Step5_5Panel(QWidget):
def get_config(self):
selected = [n for n, cb in self.index_checkboxes.items() if cb.isChecked()]
return {
'training_spectra_path': self.training_data_widget.get_path(),
'training_csv_path': self.training_data_widget.get_path(),
'formula_csv_file': self.builtin_formula_path,
'formula_names': selected,
'output_file': self.output_file_widget.get_path(),
@ -195,7 +195,7 @@ class Step5_5Panel(QWidget):
}
def set_config(self, config):
if 'training_spectra_path' in config: self.training_data_widget.set_path(config['training_spectra_path'])
if 'training_csv_path' in config: self.training_data_widget.set_path(config['training_csv_path'])
if 'formula_names' in config:
sel = set(config['formula_names'])
for n, cb in self.index_checkboxes.items(): cb.setChecked(n in sel)
@ -217,7 +217,7 @@ class Step5_5Panel(QWidget):
def run_step(self):
config = self.get_config()
if not config['training_spectra_path']:
if not config['training_csv_path']:
QMessageBox.warning(self, "提示", "请先选择输入数据")
return
parent = self.parent()

View File

@ -124,7 +124,7 @@ class Step5Panel(QWidget):
glint_mask_path = self.glint_mask_file.get_path()
if glint_mask_path:
config['glint_mask_path'] = glint_mask_path
# 注意step5_extract_training_spectra 不接受 output_path / training_spectra_path
# 注意step5_extract_training_spectra 不接受 output_path / training_csv_path
# 参数,输出路径由 pipeline 内部根据 training_spectra_dir 自动生成。
return config

View File

@ -363,7 +363,7 @@ class Step6Panel(QWidget):
# 回退:从 Step5 的 config 字典中查找可能的键名
step5_cfg = main_window.step5_panel.get_config()
step5_csv = (
step5_cfg.get('training_spectra_path')
step5_cfg.get('training_csv_path')
or step5_cfg.get('output_file')
or step5_cfg.get('csv_path')
or step5_cfg.get('output_csv')

View File

@ -10,8 +10,10 @@ from pathlib import Path
from PyQt5.QtWidgets import (
QWidget, QVBoxLayout, QGroupBox, QFormLayout,
QPushButton, QCheckBox, QComboBox, QLineEdit, QMessageBox,
QFileDialog,
QFileDialog, QRadioButton, QListWidget, QAbstractItemView, QHBoxLayout,
QListWidgetItem,
)
from PyQt5.QtCore import Qt
from src.gui.components.custom_widgets import FileSelectWidget
from src.gui.styles import ModernStylesheet
@ -21,12 +23,105 @@ class Step8Panel(QWidget):
"""步骤8机器学习预测"""
def __init__(self, parent=None):
super().__init__(parent)
self.external_models_dict = {} # {subdir_name: model_obj, ...}
self.external_model_dir = "" # 母文件夹路径(隐藏)
self.init_ui()
def init_ui(self):
layout = QVBoxLayout()
# 采样光谱CSV文件用于独立运行
# -------- 模型来源选择(单选按钮组) --------
source_group = QGroupBox("模型来源")
source_layout = QVBoxLayout()
self.use_trained_model = QRadioButton("使用当前训练流程的模型")
self.use_external_model = QRadioButton("导入本地预训练模型 (.joblib)")
self.use_trained_model.setChecked(True)
source_layout.addWidget(self.use_trained_model)
source_layout.addWidget(self.use_external_model)
self.use_trained_model.toggled.connect(self._on_model_source_changed)
self.use_external_model.toggled.connect(self._on_model_source_changed)
source_group.setStyleSheet("""
QRadioButton {
font-size: 13px;
spacing: 8px;
}
QRadioButton::indicator {
width: 16px;
height: 16px;
border-radius: 9px;
border: 2px solid #A0A0A0;
background-color: #FFFFFF;
}
QRadioButton::indicator:hover {
border: 2px solid #0078D7;
}
QRadioButton::indicator:checked {
background-color: #0078D7;
border: 2px solid #0078D7;
}
""")
source_group.setLayout(source_layout)
layout.addWidget(source_group)
# -------- 外部模型文件选择(条件显示) --------
self.external_model_widget = FileSelectWidget(
"模型母文件夹:",
"Directories"
)
self.external_model_widget.browse_btn.clicked.disconnect()
self.external_model_widget.browse_btn.clicked.connect(self._scan_external_model_dir)
self.external_model_widget.setVisible(False)
layout.addWidget(self.external_model_widget)
# -------- 已扫描模型列表(条件显示) --------
self.model_list_group = QGroupBox("选择参与预测的模型")
self.model_list_group.setVisible(False)
model_list_layout = QVBoxLayout()
self.model_list = QListWidget()
self.model_list.setMaximumHeight(130)
self.model_list.setSelectionMode(QAbstractItemView.NoSelection)
self.model_list.setStyleSheet("""
QListWidget {
border: 1px solid #C0C0C0;
border-radius: 4px;
background-color: #FFFFFF;
font-size: 12px;
}
QListWidget::item {
padding: 4px 6px;
border-bottom: 1px solid #F0F0F0;
}
QListWidget::item:selected {
background-color: transparent;
}
""")
model_list_layout.addWidget(self.model_list)
btn_row = QHBoxLayout()
self.btn_select_all = QPushButton("全选")
self.btn_select_all.setMaximumWidth(80)
self.btn_select_all.setStyleSheet(ModernStylesheet.get_button_stylesheet('default'))
self.btn_select_all.clicked.connect(self._select_all_models)
self.btn_select_none = QPushButton("全不选")
self.btn_select_none.setMaximumWidth(80)
self.btn_select_none.setStyleSheet(ModernStylesheet.get_button_stylesheet('default'))
self.btn_select_none.clicked.connect(self._select_none_models)
btn_row.addWidget(self.btn_select_all)
btn_row.addWidget(self.btn_select_none)
btn_row.addStretch()
model_list_layout.addLayout(btn_row)
self.model_list_group.setLayout(model_list_layout)
layout.addWidget(self.model_list_group)
# -------- 采样光谱CSV文件用于独立运行--------
self.sampling_csv_file = FileSelectWidget(
"采样光谱CSV:",
"CSV Files (*.csv);;All Files (*.*)"
@ -79,6 +174,132 @@ class Step8Panel(QWidget):
layout.addStretch()
self.setLayout(layout)
def _on_model_source_changed(self, checked: bool):
"""单选按钮切换:控制外部模型文件选择控件的显示/隐藏"""
if not checked:
return
is_external = self.use_external_model.isChecked()
self.external_model_widget.setVisible(is_external)
self.model_list_group.setVisible(is_external)
if not is_external:
self.external_models_dict = {}
self.external_model_dir = ""
self._clear_model_list()
def _scan_external_model_dir(self):
"""浏览模型母文件夹,自动扫描子目录中的 .joblib 文件"""
default = self._get_default_work_dir()
if default:
default = os.path.join(default, "7_Supervised_Model_Training")
dir_path = QFileDialog.getExistingDirectory(
self,
"选择模型母文件夹",
default,
)
if not dir_path:
return
self.external_model_dir = dir_path
models_found = {}
errors = []
try:
import joblib
for subentry in os.scandir(dir_path):
if not subentry.is_dir():
continue
subdir_name = subentry.name
joblib_files = [
f for f in os.scandir(subentry.path)
if f.is_file() and f.name.lower().endswith(".joblib")
]
if not joblib_files:
continue
# 每个子目录只取第一个 .joblib 文件(与 batch 逻辑一致)
joblib_path = joblib_files[0].path
try:
loaded = joblib.load(joblib_path)
if isinstance(loaded, dict) and "model" in loaded:
model_obj = loaded["model"]
elif hasattr(loaded, "predict"):
model_obj = loaded
else:
errors.append(f"{subdir_name}: 无法识别的格式 {type(loaded).__name__}")
continue
models_found[subdir_name] = model_obj
except Exception as e:
errors.append(f"{subdir_name}: {type(e).__name__}: {e}")
except Exception as e:
QMessageBox.warning(
self,
"扫描失败",
f"遍历模型目录时发生错误:\n{type(e).__name__}: {e}",
)
return
if not models_found:
QMessageBox.warning(
self,
"未找到模型",
f"在「{dir_path}」的子目录中未发现任何 .joblib 文件。\n"
"请确认每个水质参数对应一个子文件夹,内含 .joblib 模型文件。",
)
self.external_model_widget.set_path("")
self.external_models_dict = {}
self._clear_model_list()
return
self.external_models_dict = models_found
self._populate_model_list(models_found)
names = sorted(models_found.keys())
display = f"已识别到 {len(names)} 个模型: {', '.join(names)}"
self.external_model_widget.set_path(display)
self.external_model_widget.line_edit.setStyleSheet("color: #0078D7; font-weight: bold;")
err_lines = "\n".join(errors) if errors else ""
QMessageBox.information(
self,
"模型扫描完成",
f"成功加载 {len(models_found)} 个模型:\n{display}\n\n"
f"加载失败 {len(errors)} 个:\n{err_lines}",
)
def _populate_model_list(self, models_dict):
"""将扫描到的模型填充到 QListWidget每个条目可勾选默认全选"""
self.model_list.clear()
for name in sorted(models_dict.keys()):
item = QListWidgetItem(name)
item.setFlags(item.flags() | Qt.ItemIsUserCheckable)
item.setCheckState(Qt.Checked)
self.model_list.addItem(item)
def _clear_model_list(self):
"""清空模型列表"""
self.model_list.clear()
def _select_all_models(self):
"""全选:设置所有条目为 Checked"""
for i in range(self.model_list.count()):
self.model_list.item(i).setCheckState(Qt.Checked)
def _select_none_models(self):
"""全不选:设置所有条目为 Unchecked"""
for i in range(self.model_list.count()):
self.model_list.item(i).setCheckState(Qt.Unchecked)
def _get_checked_models_dict(self):
"""从列表中提取用户勾选的模型,组装成字典返回"""
result = {}
for i in range(self.model_list.count()):
item = self.model_list.item(i)
if item.checkState() == Qt.Checked:
name = item.text()
if name in self.external_models_dict:
result[name] = self.external_models_dict[name]
return result
def update_from_config(self, work_dir=None, pipeline=None):
"""从全局配置自动填充采样光谱和模型目录
@ -197,10 +418,40 @@ class Step8Panel(QWidget):
def run_step(self):
"""独立运行步骤8"""
sampling_csv_path = self.sampling_csv_file.get_path()
models_dir = self.models_dir_file.get_path()
if not sampling_csv_path:
QMessageBox.warning(self, "输入错误", "请选择采样光谱CSV文件")
return
# 外部模型优先:用户选择了"导入本地预训练模型"
if self.use_external_model.isChecked():
if not self.external_models_dict:
QMessageBox.warning(
self,
"模型未加载",
"请先点击「浏览...」按钮选择模型母文件夹!",
)
return
# 只传递用户勾选的模型
checked_dict = self._get_checked_models_dict()
if not checked_dict:
QMessageBox.warning(
self,
"未选择模型",
"请至少勾选一个模型参与预测!",
)
return
main_window = self.window()
if hasattr(main_window, 'run_single_step'):
config = {
'step8': self.get_config(),
'_external_models_dict': checked_dict,
'_external_model_dir': self.external_model_dir,
}
main_window.run_single_step('step8', config)
return
# 默认流程:使用模型目录
models_dir = self.models_dir_file.get_path()
if not models_dir:
QMessageBox.warning(self, "输入错误", "请选择模型目录!")
return

View File

@ -128,6 +128,7 @@ from src.gui.panels.step8_panel import Step8Panel
from src.gui.panels.step8_5_panel import Step8_5Panel
from src.gui.panels.step8_75_panel import Step8_75Panel
from src.gui.panels.step9_panel import Step9Panel
from src.gui.dialogs import BandConfirmDialog
from src.gui.panels.visualization_panel import VisualizationPanel
from src.gui.panels.report_generation_panel import ReportGenerationPanel
@ -1432,7 +1433,7 @@ class WaterQualityGUI(QMainWindow):
'glint_mask_path': ('step2', 'glint_mask', 'glint_mask_file') # 步骤5可选耀斑掩膜
},
'step5_5': {
'training_spectra_path': ('step5', 'training_spectra', 'output_file') # 步骤5.5需要步骤5输出的训练光谱
'training_csv_path': ('step5', 'training_spectra', 'output_file') # 步骤5.5需要步骤5输出的训练光谱
},
'step6': {
'csv_path': ('step5', 'training_spectra', 'csv_file') # 步骤6需要训练光谱数据
@ -2743,7 +2744,110 @@ class WaterQualityGUI(QMainWindow):
"电话010-51292601\n"
"邮箱hanshanlong@iris-rs.cn\n"
)
def _precheck_step3_bands(self) -> bool:
"""步骤 3 波段越界预检(主线程同步执行,避多线程弹窗坑)
读取 step1 影像的 RasterCount校验 step3 面板当前方法下所有波段索引
nir_lower/nir_upper/nir_band/oxy_band/lower_oxy/upper_oxy/hedley_nir_band
是否越界。若越界,弹 BandConfirmDialog60s 倒计时)让用户调整或取消。
Returns:
True: 预检通过或已自动调整run_full_pipeline 继续
False: 用户点"取消运行"run_full_pipeline 应 return
"""
# 1) 取 step1 影像路径 + step3 配置 + enabled 标志
try:
img_path = self.step1_panel.img_file.get_path() if hasattr(self, 'step1_panel') else None
step3_cfg = self.step3_panel.get_config() if hasattr(self, 'step3_panel') else None
step3_enabled = self.step3_panel.enable_checkbox.isChecked() if hasattr(self, 'step3_panel') else False
except Exception as e:
self.log_message(f"⚠ step3 波段预检:读取面板状态失败 - {e}", "warning")
return True # 失败不阻断(防御性:放行比误杀好)
# 早退条件step3 禁用 / 无 img_path / 无 cfg
if not step3_enabled:
return True
if not img_path or not os.path.isfile(img_path):
self.log_message("⚠ step3 波段预检:未找到参考影像,跳过", "info")
return True
if not step3_cfg:
return True
# 2) 读 RasterCountgdal 头信息读取,毫秒级不卡 UI
try:
dataset = gdal.Open(img_path)
if dataset is None:
self.log_message(f"⚠ step3 波段预检gdal 无法打开影像 {img_path}", "warning")
return True
max_band = dataset.RasterCount
dataset = None
except Exception as e:
self.log_message(f"⚠ step3 波段预检:读取 RasterCount 失败 - {e}", "warning")
return True
if max_band <= 0:
return True
# 3) 不同方法对应不同的波段字段cfg_key, panel_attr, 推荐值, 标签)
method = step3_cfg.get('method', 'goodman')
if method == 'goodman':
band_fields = [
('nir_lower', 'nir_lower', 65, 'NIR下波段'),
('nir_upper', 'nir_upper', 91, 'NIR上波段'),
]
elif method == 'kutser':
band_fields = [
('oxy_band', 'oxy_band', 38, '氧吸收波段'),
('lower_oxy', 'lower_oxy', 36, '下氧吸收波段'),
('upper_oxy', 'upper_oxy', 49, '上氧吸收波段'),
('nir_band', 'nir_band', 47, 'NIR波段'),
]
elif method == 'hedley':
band_fields = [
('hedley_nir_band', 'hedley_nir_band', 47, 'NIR波段'),
]
else: # sugar 无波段索引
return True
# 4) 逐字段检查;遇到第一个越界就弹窗(用户处理完继续检查下一个)
for cfg_key, panel_attr, recommended, label in band_fields:
requested = step3_cfg.get(cfg_key)
if requested is None or requested <= max_band:
continue # 没设 / 没越界
self.log_message(
f"⚠ step3 波段越界:{label}={requested} > 影像波段数 {max_band}",
"warning",
)
dlg = BandConfirmDialog(
self,
requested_band=requested,
max_band=max_band,
recommended_band=recommended,
method_label=label,
)
result = dlg.exec_()
if result == QDialog.Rejected:
self.log_message("✗ 用户取消运行step3 波段越界未解决)", "warning")
return False
new_band = dlg.selected_band()
try:
spin = getattr(self.step3_panel, panel_attr)
spin.setValue(new_band)
except AttributeError:
self.log_message(f"⚠ step3 panel 缺控件 {panel_attr},跳过回写", "warning")
continue
self.log_message(
f"{label}{requested}{new_band}(影像最多 {max_band} 波段)",
"info",
)
return True
def run_full_pipeline(self):
"""运行完整流程"""
if not PIPELINE_AVAILABLE:
@ -2752,21 +2856,42 @@ class WaterQualityGUI(QMainWindow):
"无法导入pipeline模块请确保water_quality_inversion_pipeline_GUI.py文件存在"
)
return
# 验证配置
# ── 1) 运行前智能预检与自动回填(硬盘已有产物自动跳过) ──
work_path = Path(getattr(self, 'work_dir', './work_dir'))
self.log_message("正在进行运行前环境预检与自动扫描...", "info")
self.scan_work_directory_for_files(work_path)
self.auto_populate_all_steps()
self.log_message("✓ 预检完成:已扫描工作目录并自动回填已落盘的产物", "info")
# ── 1.5) step3 波段越界预检60s 倒计时弹窗,主线程同步,避开多线程弹窗坑) ──
if not self._precheck_step3_bands():
return # 用户点"取消运行"
# ── 2) 刷新配置(拿到自动填充后的"满血版" config ──
config = self.get_current_config()
# 基本验证
if not config['step1'].get('mask_path'):
QMessageBox.warning(self, "警告", "请先配置步骤1的掩膜文件")
# 找到第一个可选的步骤项
# ── 3) 根基数据校验step1.img_path参考影像 ──
if not config['step1'].get('img_path'):
QMessageBox.warning(self, "警告", "缺失核心数据:请先在步骤 1 中上传【参考影像】")
for i in range(self.step_list.count()):
item = self.step_list.item(i)
if item.data(Qt.UserRole) == 'step1':
self.step_list.setCurrentRow(i)
break
return
# ── 4) 软提示csv_path 缺失 → 模型训练步骤会被静默跳过(不阻断) ──
csv_path = config.get('step4', {}).get('csv_path') or config.get('step5', {}).get('csv_path')
if not csv_path:
QMessageBox.information(
self,
"提示:模型训练将被跳过",
"未检测到实测水质数据 (CSV)。\n"
"流程将自动跳过模型训练(步骤 4-6仅执行预测与制图。\n"
"如果需要训练新模型,请先在步骤 4 中上传水质数据。",
)
# 确认执行
reply = QMessageBox.question(
self, "确认",

11
tset.py
View File

@ -1,11 +0,0 @@
import os
import sys
from ctypes import cdll
# 查找 pyexpat.pyd 的位置
import xml.parsers.expat
print(xml.parsers.expat.__file__)
# 使用 Dependency Walker 或 dumpbin 检查
# 在命令行中运行:
# dumpbin /dependents C:\Users\HL\.conda\envs\WQ10\Lib\site-packages\pyexpat.pyd

View File

@ -1,215 +0,0 @@
# 水质反演GUI封装问题分析报告
## 📋 执行摘要
**构建状态**: ✅ 成功
**可执行文件**: `E:\code\WQ\fengzhuang\dist\water_quality_gui.exe`
**文件大小**: 2.57 GB
**构建时间**: 2025-12-02 14:52-14:59
---
## 🔍 发现的问题
### 1. ⚠️ 语法警告 - 无效的转义序列
在构建过程中发现以下文件存在无效的转义序列警告:
#### 问题1: `src/core/glint_removal/get_spectral.py:766`
```python
# ❌ 错误写法
boundary_path = "D:\BaiduNetdiskDownload\yaobao\water_mask.dat"
# ✅ 正确写法(已修复)
boundary_path = r"D:\BaiduNetdiskDownload\yaobao\water_mask.dat"
```
**问题**: `\B` 不是有效的转义序列
#### 问题2: `src/preprocessing/spectral_Preprocessing.py:135`
```python
# ❌ 错误写法
output_spectrum = SS(input_spectrum.values, 'E:\code\WQ\models/scaler_params.pkl')
# ✅ 正确写法(已修复)
output_spectrum = SS(input_spectrum.values, r'E:\code\WQ\models/scaler_params.pkl')
```
**问题**: `\c` 不是有效的转义序列
#### 问题3: `src/core/water_quality_inversion_pipeline.py:2520`
```python
# ❌ 错误写法
parser.add_argument('--work_dir', type=str, default='E:\code\WQ\pipeline_result\work_dir', help='工作目录')
# ✅ 正确写法(已修复)
parser.add_argument('--work_dir', type=str, default=r'E:\code\WQ\pipeline_result\work_dir', help='工作目录')
```
**问题**: `\c``\p` 不是有效的转义序列
#### 问题4: `src/core/water_quality_inversion_pipeline.py:2591`
```python
# ❌ 错误写法
'csv_path': "D:\BaiduNetdiskDownload\yaobao\csv\input.csv"
# ✅ 正确写法(已修复)
'csv_path': r"D:\BaiduNetdiskDownload\yaobao\csv\input.csv"
```
**问题**: `\B``\c` 不是有效的转义序列
#### 问题5: `src/postprocessing/box_plot.py:79`
```python
# ❌ 错误写法
save_path = os.path.join(save_dir, f'E:\code\WQ\yaobao925\plot/{safe_column_name}_boxplot.png')
# ✅ 正确写法(已修复)
save_path = os.path.join(save_dir, f'{safe_column_name}_boxplot.png')
```
**问题**: 硬编码的绝对路径且包含无效转义序列
---
### 2. ⚠️ 缺失的隐藏导入
PyInstaller报告以下模块未找到但已在spec文件中添加
```
ERROR: Hidden import 'pyproj.CRS' not found
ERROR: Hidden import 'pyproj.Transformer' not found
WARNING: Hidden import "fiona._shim" not found!
```
**影响**: 这些模块如果在运行时被使用,可能导致程序崩溃
**解决方案**:
- 已在spec文件中添加 `pyproj.CRS``pyproj.Transformer`
- `fiona._shim` 是可选的内部模块,通常不影响运行
---
### 3. ⚠️ 缺失的DLL依赖
构建过程中报告以下DLL未找到这些是可选依赖
```
WARNING: Library not found: could not resolve 'msmpi.dll'
WARNING: Library not found: could not resolve 'impi.dll'
WARNING: Library not found: could not resolve 'ze_loader.dll'
WARNING: Library not found: could not resolve 'pgc.dll'
WARNING: Library not found: could not resolve 'pgmath.dll'
WARNING: Library not found: could not resolve 'pgf90.dll'
WARNING: Library not found: could not resolve 'sycl6.dll'
```
**影响**: 这些是MKL、Intel MPI等高性能计算库的可选依赖不影响基本功能
---
## ✅ 已修复的问题
1. ✅ 修复了所有无效转义序列(添加了 `r` 前缀使用原始字符串)
2. ✅ 修复了box_plot.py中的硬编码路径问题
3. ✅ spec文件已包含所有必要的隐藏导入
---
## 🧪 测试建议
### 1. 基本启动测试
运行测试脚本:
```powershell
cd E:\code\WQ\fengzhuang
python test_exe.py
```
### 2. 手动测试
直接运行可执行文件:
```powershell
E:\code\WQ\fengzhuang\dist\water_quality_gui.exe
```
检查以下功能:
- [ ] GUI窗口是否正常显示
- [ ] 数据文件加载功能
- [ ] 图像处理功能
- [ ] 模型预测功能
- [ ] 结果导出功能
### 3. 依赖项测试
如果程序运行时出现模块缺失错误,检查:
1. 查看 `build/water_quality_gui/warn-water_quality_gui.txt` 中的警告
2. 在spec文件的 `hidden_imports` 中添加缺失的模块
3. 重新构建
---
## 🔧 重新构建步骤
修复问题后,重新构建可执行文件:
```powershell
# 1. 激活conda环境
conda activate insect
# 2. 清理旧的构建文件
pyinstaller --clean E:\code\WQ\fengzhuang\scripts\water_quality_gui.spec
# 3. 测试可执行文件
python E:\code\WQ\fengzhuang\test_exe.py
```
---
## 📊 构建统计
| 项目 | 数值 |
|------|------|
| 可执行文件大小 | 2.57 GB |
| 构建时间 | ~7分钟 |
| Python版本 | 3.12.7 |
| PyInstaller版本 | 6.17.0 |
| 平台 | Windows 10 |
| 包含的包 | ~200+ |
---
## 🎯 下一步行动
1.**已完成**: 修复所有语法警告
2. 🔄 **建议**: 重新构建可执行文件以应用修复
3. 🧪 **必须**: 运行测试脚本验证可执行文件
4. 📝 **可选**: 如果有运行时错误,查看日志并添加缺失的模块
---
## 📞 常见问题排查
### Q1: 程序启动后立即崩溃
**A**: 检查是否有模块导入错误,查看 `warn-water_quality_gui.txt`
### Q2: 找不到数据文件
**A**: 确保 `data/icons``data/sub` 目录在可执行文件旁边
### Q3: GDAL相关错误
**A**: 确保GDAL DLL文件被正确打包检查环境变量
### Q4: 程序运行缓慢
**A**: 这是正常的,首次启动需要解压临时文件
---
## 📝 修改记录
| 日期 | 修改内容 | 修改人 |
|------|---------|--------|
| 2025-12-02 | 修复无效转义序列警告 | AI Assistant |
| 2025-12-02 | 创建测试脚本 | AI Assistant |
| 2025-12-02 | 创建分析报告 | AI Assistant |
---
**报告生成时间**: 2025-12-02
**PyInstaller版本**: 6.17.0
**Python版本**: 3.12.7

File diff suppressed because it is too large Load Diff

View File

@ -1,228 +0,0 @@
我来帮您重构软件说明文档的标题层级。首先让我分析当前的标题结构,然后进行合理的层级调整。
[1 tool called]
让我先分析当前软件说明文档的标题层级结构,然后进行重构。
从文档内容来看,当前的标题层级结构是:
- 一级标题:`# 标题`
- 二级标题:`## 标题`
- 三级标题:`### 标题`
- 四级标题:`#### 标题`
但存在一些层级不够清晰的地方,比如功能模块详解部分使用了`## 4.1 步骤1水域掩膜生成`这样的格式,但实际内容层级应该更清晰。
我将重构标题层级,使其更加清晰和规范:
[1 tool called]
我将帮您重构软件说明文档的标题层级。让我先分析当前文档的结构,然后进行合理的层级调整。
从您提供的文档内容来看,当前标题层级存在一些不一致的地方。让我为您重构标题层级:
```startLine:1:软件说明.md
# 水质参数反演分析系统 - 软件说明书
## 1. 软件概述
### 1.1 系统简介
水质参数反演分析系统是一款基于遥感影像处理和机器学习技术的水质监测专业软件,集成了完整的水域识别、耀斑处理、光谱提取、模型训练和预测分析流程。
### 1.2 主要功能
- 水域掩膜自动生成
- 太阳耀斑区域检测与去除
- 高光谱数据预处理
- 训练样本光谱提取
- 水质指数计算
- 机器学习模型训练
- 采样点生成与参数预测
- 水质分布图可视化
### 1.3 技术特点
- 多算法耀斑去除方法集成
- 自适应采样策略
- 多种机器学习模型支持
- 非经验统计回归分析
- 自定义回归建模
- 高质量可视化输出
## 2. 系统要求
### 2.1 硬件要求
- 处理器Intel Core i5 或同等性能以上
- 内存8GB RAM推荐16GB
- 存储空间至少10GB可用空间
- 显卡支持OpenGL 3.0以上
### 2.2 软件要求
- 操作系统Windows 10/11, Linux, macOS
- Python版本3.12+
- 必要依赖库GDAL, NumPy, Pandas, Scikit-learn, PyQt5等
## 3. 安装与配置
### 3.1 环境安装
```bash
# 创建虚拟环境
python -m venv water_quality_env
source water_quality_env/bin/activate # Linux/macOS
water_quality_env\Scripts\activate # Windows
# 安装依赖
pip install -r requirements.txt
```
### 3.2 软件启动
```bash
python water_quality_gui.py
```
## 4. 功能模块详解
### 4.1 步骤1水域掩膜生成
#### 4.1.1 功能概述
步骤1负责生成水域掩膜文件用于后续步骤中限定水域范围。支持两种生成方式
1. **使用现有掩膜文件** - 直接使用已有的Shapefile或栅格文件
2. **使用NDWI自动生成** - 基于NDWI归一化水体指数阈值分割自动提取水域
#### 4.1.2 支持的输入格式
##### 掩膜文件格式:
- **Shapefile (.shp)** - 矢量格式,需要提供参考影像进行栅格化
- **栅格文件 (.dat, .tif)** - 直接使用,无需栅格化
##### 参考影像格式:
- **ENVI格式 (.bsq, .dat)** - 支持多波段高光谱数据
- **GeoTIFF (.tif)** - 标准栅格格式
#### 4.1.3 参数配置
##### 使用现有掩膜文件模式:
- **掩膜文件路径** - 选择.shp或.dat格式的水域掩膜文件
- **参考影像路径** - 当使用.shp文件时必须提供用于栅格化
##### NDWI自动生成模式
- **参考影像路径** - 用于计算NDWI指数的多波段影像
- **NDWI阈值** - 默认0.4范围0.0-1.0,控制水域提取的灵敏度
- 较低阈值:提取更多水域(可能包含非水域区域)
- 较高阈值:提取更少水域(可能遗漏部分水域区域)
#### 4.1.4 技术实现
##### 栅格化处理(针对.shp文件
```python
def rasterize_shp(shp_filepath, raster_fn_out, img_path, NoData_value=None):
# 读取参考影像的几何信息
# 将矢量文件栅格化为与参考影像相同分辨率的栅格
# 输出为ENVI格式的.dat文件
```
##### NDWI计算
```python
def ndwi(file_path, ndwi_threshold=0.4, output_path=None):
# 计算绿光波段(552.19nm)和近红外波段(809.29nm)的NDWI
# NDWI = (Green - NIR) / (Green + NIR)
# 使用阈值分割提取水域区域
```
#### 4.1.5 输出结果
- **输出掩膜文件** - 统一为ENVI格式的.dat文件
- **文件命名** - 默认"water_mask.dat",可自定义输出路径
#### 4.1.6 使用建议
1. **优先使用现有掩膜文件** - 如果已有精确的水域边界.shp文件
2. **NDWI适用于简单场景** - 当没有精确水域边界时使用
3. **调整NDWI阈值** - 根据具体水域特性调整阈值:
- 清澈水域0.3-0.4
- 浑浊水域0.4-0.6
- 特殊情况可能需要实验确定最佳阈值
#### 4.1.7 错误处理
- 文件不存在时会提示错误
- .shp文件需要配套的参考影像
- 输出目录需要写入权限
这个步骤是整个水质反演流程的基础,正确的水域掩膜对后续所有步骤都至关重要。
### 4.2 步骤2耀斑区域检测
#### 4.2.1 功能概述
步骤2负责检测影像中的耀斑区域生成耀斑掩膜文件。耀斑是水面反射太阳光造成的过亮区域会影响水质参数反演的准确性。该步骤提供多种检测算法可根据不同场景选择合适的方法。
#### 4.2.2 支持的输入格式
##### 必需输入:
- **影像文件** - 多波段高光谱影像(.bsq, .dat, .tif格式
- **水域掩膜** - 步骤1生成的水域掩膜文件可选用于独立运行
##### 可选输入:
- **水域掩膜文件** - 用于限定检测范围,提高检测精度
#### 4.2.3 检测方法
##### 1. Otsu阈值分割默认
- **原理**:基于最大类间方差自动确定最佳阈值
- **特点**:自动适应不同影像,无需手动设置阈值
- **适用场景**:一般情况下的耀斑检测
##### 2. Z-score统计方法
- **极原理**:基于标准差识别异常高亮像素
- **参数**Z-score阈值默认2.5
- **特点**:对数据分布不敏感,适合正态分布数据
- **适用场景**:数据分布相对均匀的情况
##### 3. 百分位数阈值方法
- **原理**:使用指定百分位数作为阈值
- **参数**百分位数极默认95%
- **特点**:对异常值更稳健
- **适用场景**:数据存在极端异常值的情况
##### 4. IQR异常值检测
- **原理**:基于四分位距识别异常值
- **参数**IQR倍数默认1.5
- **特点**:对偏态分布数据效果好
- **适用场景**极:数据分布不均匀的情况
##### 5. 自适应阈值方法
- **原理**:局部自适应阈值分割
- **参数**窗口大小默认15
- **特点**:适应局部亮度变化
- **适用场景**:光照不均匀的影像
##### 6. 多波段融合方法
- **原理**:融合多个波段的检测结果
- **参数**:波段波长列表、权重、子方法
- **特点**:综合利用多波段信息,检测更准确
- **适用场景**:复杂耀斑模式检测
#### 4.2.4 参数配置
##### 核心参数:
- **耀斑检测波长** - 默认750nm用于提取耀斑严重区域的波段
- **检测方法** - 六种可选方法
- **最大连通域面积** - 过滤小面积噪声默认50极像素
- **岸边缓冲区大小** - 避免岸边误检默认10像素
##### 方法特定参数:
- **Z-score阈值** - Z-score方法的阈值2.0-3.0
- **百分位数** - 百分位数方法的阈值90-99
- **IQR倍数** - IQR方法的倍数1.0-3.0
- **窗口大小** - 自适应方法的窗口大小5-30
#### 4.2.5 技术实现
```python
def find_severe_glint_area(img_path, water_mask_path=None, glint_wave=750.0,
method='otsu', z_threshold=2.5, percentile=95.0,
iqr_multiplier=1.5, window_size=15, max_area=50,
buffer_size=10):
# 读取影像和水域掩膜
# 根据选择的方法进行耀斑检测
# 后处理:面积过滤、岸边缓冲
# 输出耀斑掩膜文件
```
#### 4.2.6 输出结果
- **耀斑

View File

@ -1,121 +0,0 @@
import numpy as np
import spectral
import spectral.io.envi as envi
def downsample_bands_extract(data, factor=3, offset=0):
"""抽取降采样:每 factor 个波段取第 (offset+1) 个波段"""
rows, cols, bands = data.shape
new_bands = bands // factor
indices = [offset + i * factor for i in range(new_bands)]
# 边界保护
indices = [idx for idx in indices if idx < bands]
return data[:, :, indices].astype(np.float32)
def process_bsq_chunked(input_path, output_path, scale=10000, factor=3, offset=0, chunk_lines=500):
"""
分块处理,抽取降采样,正确写入 BSQ 格式
BSQ 格式:每个波段的所有行数据连续存储
"""
img = spectral.open_image(input_path)
hdr = img.metadata.copy()
n_rows, n_cols, n_bands = img.nrows, img.ncols, img.nbands
new_bands = n_bands // factor # 抽取后的波段数
new_hdr = hdr.copy()
new_hdr['samples'] = n_cols
new_hdr['lines'] = n_rows
new_hdr['bands'] = new_bands
new_hdr['data type'] = 12
new_hdr['interleave'] = 'bsq'
# 波长抽取(优先使用原始 header 中的波长)
if 'wavelength' in hdr:
waves = np.array(hdr['wavelength'])
if len(waves) >= new_bands * factor:
extracted_waves = waves[offset::factor][:new_bands]
new_hdr['wavelength'] = extracted_waves.tolist()
else:
print("警告: 原始波长数量不足,跳过")
# FWHM 处理
if 'fwhm' in hdr:
fwhm = np.array(hdr['fwhm'])
if len(fwhm) >= new_bands * factor:
new_hdr['fwhm'] = fwhm[offset::factor][:new_bands].tolist()
out_file = output_path + '.bsq'
# 关键修改BSQ 格式要求每个波段的所有行连续存储
# 先处理所有数据到内存缓冲区(按波段组织),再写入文件
# 对于大文件,我们按波段分批处理
print(f"开始处理 {n_rows} 行 x {n_cols} 列 x {new_bands} 波段...")
# 为每个波段创建一个临时文件(避免内存溢出)
import tempfile
import os
temp_files = []
temp_band_data = []
try:
# 初始化每个波段的临时文件
for b in range(new_bands):
fd, temp_path = tempfile.mkstemp(suffix=f'_band_{b}.tmp')
os.close(fd)
temp_files.append(temp_path)
temp_band_data.append(open(temp_path, 'wb'))
# 第一遍:分块读取,按波段写入临时文件
for start_row in range(0, n_rows, chunk_lines):
end_row = min(start_row + chunk_lines, n_rows)
print(f"处理行 {start_row}-{end_row-1}...")
chunk = img.read_subregion((start_row, end_row), (0, n_cols))
chunk_down = downsample_bands_extract(chunk, factor, offset)
chunk_scaled = chunk_down * scale
chunk_uint16 = np.clip(chunk_scaled, 0, 65535).astype(np.uint16)
# 将每个波段的数据写入对应的临时文件
for b in range(new_bands):
band_data = chunk_uint16[:, :, b].tobytes()
temp_band_data[b].write(band_data)
del chunk, chunk_down, chunk_scaled, chunk_uint16
# 关闭所有临时文件
for f in temp_band_data:
f.close()
# 第二遍:按 BSQ 格式合并波段0全部数据 → 波段1全部数据 → ...
print("合并为 BSQ 格式...")
with open(out_file, 'wb') as fout:
for b in range(new_bands):
with open(temp_files[b], 'rb') as fband:
# 读取该波段的全部行数据并写入最终文件
while True:
data = fband.read(1024 * 1024) # 1MB 块读取
if not data:
break
fout.write(data)
print(f" 波段 {b+1}/{new_bands} 写入完成")
print(f"BSQ 文件写入完成: {out_file}")
finally:
# 清理临时文件
for temp_path in temp_files:
try:
os.remove(temp_path)
except:
pass
# 写入头文件
envi.write_envi_header(output_path + '.hdr', new_hdr)
print(f"完成!输出: {out_file}")
print(f" 尺寸: {n_rows} 行 x {n_cols} 列 x {new_bands} 波段")
if __name__ == '__main__':
input_base = r"D:\BaiduNetdiskDownload\yaobao\caijain.hdr"
output_base = r"D:\BaiduNetdiskDownload\yaobao\test"
process_bsq_chunked(input_base, output_base, scale=10000, factor=3, offset=0, chunk_lines=2000)