COVER-Fish/Reference-Update-Benchmark
COVER-Fish Reference-Update Benchmark This benchmark studies how reference-corpus updates change a frozen recognition system, instantiated on fine-grained fish identification. It publishes the versioned control plane behind COVER-Fish: row-level manifests, gallery states, taxonomy, tensor bindings, transition evidence, protocols and dependency locks. The benchmark does not duplicate the 83 GB Full Payload Archive. Large source archives and frozen tensors remain in the immutable… See the full description on the dataset page: https://huggingface.co/datasets/COVER-Fish/Reference-Update-Benchmark.
0118
1# Datasheet for the COVER-Fish Reference-Update Benchmark2 3## Purpose4 5This is a versioned environment for measuring how changes to a reference corpus6alter a frozen fine-grained recognition system. It separates source-package7membership, aggregation rules and query domains so that gains and regressions8can be reported together and source layers can be rolled back.9 10## Contents11 12- Five source manifests (S0–S4) with provenance, rights and content hashes.13- Eight canonical gallery states from FishBase R0 through +Commons.14- FishBase 25.04 taxonomy and final row/prototype bindings.15- QT26-QC and Q-INT benchmark manifests.16- Transition ledgers and compact result evidence.17- Fixed dependency lock, replay examples and verification tools.18 19The final clean gallery state has 107,722 rows and 19,144 species prototypes.20Frozen tensor and image archives are dependencies, not duplicated payloads.21 22## Tracks23 24Gallery-state replay changes state membership while freezing the scoring stack.25Aggregation replay changes the declared aggregation rule while freezing rows and26embeddings. Neither track treats the public evaluation labels as an acquisition27optimization target.28 29## Rights and access30 31The control files are fully downloadable. Third-party source images retain their32row-level terms and are available through the fixed Full Payload Archive or33source pointers. Licence-filtered TSV files identify rows but are not physical34image subsets. Derived embeddings do not reproduce source pixels; underlying35licence and attribution remain bound through row IDs and manifests.36 37## Known limits38 39The benchmark is instantiated on fish recognition and one frozen BioCLIP 2.540embedding surface. Source packages differ in identity assurance and domain. Some41Level B source rows are pointer-only or permission-required. Cross-domain museum42analysis requires the separately distributed Fish-Vista benchmark.43