GitHub - omarathon/compression-geospatial-full

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 265 Commits
bench		bench
datasets		datasets
external		external
py		py
scripts		scripts
src		src
tests		tests
.clang-format		.clang-format
.gitignore		.gitignore
.gitmodules		.gitmodules
CLAUDE.md		CLAUDE.md
CMakeLists.txt		CMakeLists.txt
LICENSE.txt		LICENSE.txt
README.txt		README.txt
ablation_bench_carry.sh		ablation_bench_carry.sh
bench_worldclim.sh		bench_worldclim.sh
build_lossy_index.py		build_lossy_index.py
check_nodata_bug.py		check_nodata_bug.py
check_nodata_bug.sh		check_nodata_bug.sh
memusg		memusg
parse_ablation.sh		parse_ablation.sh
parse_bench_comp_results.py		parse_bench_comp_results.py
parse_bench_comp_results_for_agg.py		parse_bench_comp_results_for_agg.py
patch_index_formula_nrmse.py		patch_index_formula_nrmse.py
query_index_std.py		query_index_std.py
results.txt		results.txt
run_benchmarks.py		run_benchmarks.py
run_benchmarks_16bit.py		run_benchmarks_16bit.py
run_benchmarks_sweep.py		run_benchmarks_sweep.py
run_benchmarks_sweep_realdata.py		run_benchmarks_sweep_realdata.py
run_benchmarks_sweep_realdata_parallel.py		run_benchmarks_sweep_realdata_parallel.py
run_comp_bench.sh		run_comp_bench.sh
run_comp_bench_lossy_index.py		run_comp_bench_lossy_index.py
run_comp_bench_originals.sh		run_comp_bench_originals.sh
run_comp_bench_with_lerc.sh		run_comp_bench_with_lerc.sh
run_comp_bench_with_lossy_tif_transform.sh		run_comp_bench_with_lossy_tif_transform.sh
run_with_perf.sh		run_with_perf.sh
todo		todo
verify_fusion.txt		verify_fusion.txt
verify_fusion_latest.sh		verify_fusion_latest.sh
verify_ndvi_2band.py		verify_ndvi_2band.py

Repository files navigation

Cambridge MRes Project "Assessing high-performance lightweight compression formats for Geospatial computation".

Main programs:
* test_comp.cpp: test codecs
* bench_comp.cpp: benchmark codecs
* bench_pipeline.cpp: benchmark geospatial pipelines

All codecs are implemented in the `codecs` directory.

Additional:
* test_remappings.cpp: verifies Morton remappings
* util.h, transformations.h, remappings.h: CPP utilities
* py/*: Python utilities

Setup:
1. obtain the submodules in `external` and build them
2. `make`

Branches (uncleaned):
* agg: with modifications for fusing aggregation into decompression
* hpc: with modifications for running on HPC

Note for fusing aggregation into decompression: use the `agg` branch & re-build `external` with the following forks:
* FastPFor: https://github.com/omarathon/FastPFor
* simdcomp: https://github.com/omarathon/simdcomp

License: CC BY 4.0