Skip to content

Commit

Permalink
[PLAT-2086][Platform] Allow retention of platform logs on replicated …
Browse files Browse the repository at this point in the history
…after upgrades

Summary:
Moved platform logs, so now the platform logs will now be saved under `/opt/yugabyte/yugaware/data/logs` instead of `/opt/yugabyte/yugaware/logs`.
The problem before was that during upgrade with replicated, logs would disappear across upgrades and debugging issues before an upgrade is a huge problem.

The directory `/opt/yugabyte/yugaware/data/` is already stored in a persisted volume in `{storage path}/yugaware/data`, so saving logs in this directory will retain these logs across upgrades

Test Plan:
(A)
1. Set up a replicated environment for dev (vm on gcloud)  using this doc: https://docs.google.com/document/d/1o_sn93VyVP-tnN-Qh2jDQ5OnXDWLn8r9RAkoyu0B0wI/edit#
2. Edit `replicated.yml` with the log directory save location and create a new unstable release
3. Check that logs are now saved in the persisted volume under `{storage path}/yugaware/data`, in this vm, logs are stored in `/opt/yugabyte/yugaware/data/logs` on the vm
4. Create a new unstable release with no little/no change to `replicated.yml`. This should restart all the containers. Go to `<VmIP>:8800` and check for updates and update to newest unstable release.
5. Check that logs from before the upgrade are retained on the vm in `{storage path}/yugaware/data/logs`

(B)
We also do not want the backup script yb_platform_backup.sh to backup the logs directory. We check this by:
1. Looking at the yb_platform_backup.sh (backup should only save `/opt/yugabyte/yugaware/data/certs`, `/opt/yugabyte/yugaware/data/keys`, and `/opt/yugabyte/yugaware/data/provision` directories)
2. Ensure that (A) was completed before the next few steps
3. Copy `/yugabyte-db/managed/devops/bin/yb_platform_backup.sh` to the home directory of the vm
4. Run `chmod +x yb_platform_backup.sh`
5. Run `sudo ./yb_platform_backup.sh create --exclude_prometheus --exclude_releases --db_username postgres --db_port 5432 -o /tmp/{tempfolder}`
6. We should see a backup_{time-creation/unique id}.tgz. Unzip using `sudo tar -xvzf backup_{time-creation/unique id}.tgz`, Make sure that the logs are not saved in this backup (since we know the logs are already stored in `/opt/yugabyte/yugaware/data/logs`

Reviewers: sanketh, sneelakantan

Reviewed By: sneelakantan

Subscribers: jenkins-bot, yugaware

Differential Revision: https://phabricator.dev.yugabyte.com/D18476
  • Loading branch information
charleswang234 committed Jul 22, 2022
1 parent 70ee4fb commit 47feade
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions managed/devops/replicated.yml
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,7 @@ components:
play.http.forwarded.trustedProxies = ["172.17.0.1"]
play.i18n.langs = [ "en" ]
pidfile.path = "/dev/null"
log.override.path = "/opt/yugabyte/yugaware/data/logs"
db {
default.host="{{repl HostPrivateIpAddress "db" "postgres" }}"
Expand Down

0 comments on commit 47feade

Please sign in to comment.