Contrib backup/restore scripts

[teletechie]

Thank you for contributing to Harbor!

Comprehensive Summary of your change

backup and restore scripts that are far more robust than the ones that used to exist in the project. These scripts also support much faster backups (using rsync instead of recursive cp) as well as use of rsync to copy the backup directory to a secondary node and then restore from that synced backup directory. Extremely short downtime compared to the old scripts.

Issue being fixed

Contributes upgraded/working backup and restore scripts

Please indicate you've done the following:

• Well Written Title and Summary of the PR
• [] Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
• Accepted the DCO. Commits without the DCO will delay acceptance.
• Made sure tests are passing and test coverage is added if needed.
• Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

[wy65701436]

Thanks, @teletechie, for your contribution! Could you provide more details on this pull request?

• Are the scripts intended for Docker Compose or Kubernetes-based Harbor instances?
• Have you tried leveraging Velero for backup and restore in Kubernetes? Since Harbor already supports Velero as a backup and restore solution.
• Lastly, if this is a new feature for Harbor, please submit a proposal first before filing the PR.

[wy65701436]

@teletechie I assume these scripts are intended for Docker Compose only. I have a few general questions:

• Have you tested them in your environment?
• It seems like you're using an older code base, as ChartMuseum has been removed for some time.
• Would you be able to provide long-term support for the community? Once this is merged into the code base, there will be users who will provide feedback or report bugs. Can you commit to owning those issues?

[teletechie]

• this code is in full production use in our environment. it is used both to sync directly between a primary and secondary instance daily and for fetching backups to our backup system.
• it supports both the last version that had chartmuseum (as we have a site that for now relies on that functionality) as well as the latest branch
• yes, i will provide support as needed. especially initially. over time i hope other people who need this functionality will also chip in and make it a community effort.

[teletechie]

Thanks, @teletechie, for your contribution! Could you provide more details on this pull request?

• Are the scripts intended for Docker Compose or Kubernetes-based Harbor instances?
• Have you tried leveraging Velero for backup and restore in Kubernetes? Since Harbor already supports Velero as a backup and restore solution.
• Lastly, if this is a new feature for Harbor, please submit a proposal first before filing the PR.

• These scripts are primarily for the docker compose based instances. we need to run a registry outside of kubernetes to support the automations that bring up airgapped kubernets clusters.
• Velero is, as you say, not an option outside of kubernetes. I did glance over the instructions, and they seemed very manual. we need something that can be 100% automated.
• It was meant as an upgrade for the backup and restore scripts harbor used to have in the contrib section. i see that those have been removed. as such it is more a re-introduction of lost (unsupported, but needed) functionality. do you want me to make a proposal for that?

[Copilot]

Pull Request Overview

This PR introduces enhanced backup and restore scripts for Harbor to improve performance and error handling by leveraging rsync for faster file transfers and reducing downtime during operations. Key changes include:

• A new restore script (harbor-restore) that starts a temporary database container and restores the Harbor databases and associated data.
• An updated backup script (harbor-backup) that utilizes rsync and offers the option to create a tarball archive.
• Updated documentation in README.md to guide users on how to use these scripts.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
contrib/backup-restore/harbor-restore	New restore script managing database container setup and data restoration.
contrib/backup-restore/harbor-backup	New backup script using rsync with optional tarball creation.
contrib/backup-restore/README.md	Documentation update explaining usage, options, and prerequisites.

[teletechie]

i think the pr needs the label release-note/community but i can not see how to add it.

[wy65701436]

please remove this part since the ChartMuseum has been deprecated for long time.

[teletechie]

i have changed the code to issue warnings when backing up or restoring chartmuseum data, but the script will still do the job. we are probably not the only ones having to still run one old harbor release for "that project" that hasn't yet moved out of chartmuseum. when syslog is enabled these warnings are also issued with warning level in syslog.

[wy65701436]

I think we need to remove this part on chart_museum since it has been removed for several releases and we should not add any code relate with it.

[github-actions]

This PR is being marked stale due to a period of inactivty. If this PR is still relevant, please comment or remove the stale label. Otherwise, this PR will close in 30 days.

[teletechie]

how do i get this moving again?
instead of removing chartmuseum support the change issues warnings when chartmuseum data is detected. it still backs up the data, as a backup and restore script shouldn't willfully lose data.

[wy65701436]

lgtm

[wy65701436]

@teletechie I’m generally okay with this script, thanks. One more question: since we don’t have prior experience with it and it hasn’t been validated in CI, could you provide the following support for community users who might rely on it?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.86%. Comparing base (c8c11b4) to head (6cd1121).
⚠️ Report is 631 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main   #21971       +/-   ##
===========================================
+ Coverage   45.36%   65.86%   +20.49%     
===========================================
  Files         244     1074      +830     
  Lines       13333   116355   +103022     
  Branches     2719     2931      +212     
===========================================
+ Hits         6049    76636    +70587     
- Misses       6983    35470    +28487     
- Partials      301     4249     +3948     

Flag	Coverage Δ
unittests	65.86% <ø> (+20.49%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 989 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

This PR is being marked stale due to a period of inactivty. If this PR is still relevant, please comment or remove the stale label. Otherwise, this PR will close in 30 days.

[teletechie]

how do i get the second review that is required?