Not all backup requirements are the same, there are simple backups and then there are complex backup and recovery strategies. After working with hundreds of users laying out the ground work, we've decided that although not all installations are the same, all of them run through the same procedure so we have decided to automate most of this things with pyxbackup. This talk will highlight the evolution of the project from the most basic backup implementation to extending capabilities for different scenarios like compression, remote storage, encryption and being a portable, comprehensible backup utility for the novice and experts alike. The pyxbackup project can be found at: https://github.com/dotmanila/pyxbackup

1. Evolving Backups Strategy,
Devploying pyxbackup
Jervin Real
Percona Live 2015 - Amsterdam

2. Managing Backups

3. Percona ♥ Backups

4. Why?
Why not?

5. Why?

6. Everybody ♥ Backups
s9s_backup

7. Everybody ♥ Backups
s9s_backup
holland backup
manager

8. Everybody ♥ Backups
xtrabackup
manager
s9s_backup
holland backup
manager

9. Everybody ♥ Backups
xbackup.sh
xtrabackup
manager
s9s_backup
holland backup
manager

10. Everybody ♥ Backups
xbackup.sh
xtrabackup
manager
s9s_backup
holland backup
manager
fromdual_brman

11. Everybody ♥ Backups
xbackup.sh
xtrabackup
manager
s9s_backup
holland backup
manager
fromdual_brman twindb

12. Everybody ♥ Backups
xbackup.sh
xtrabackup
manager
s9s_backup
holland backup
manager
fromdual_brman twindb
… and more

13. Where things fall short
● dependencies
● portability
● for non-dbas
● babysitting
● validation
● remote streaming
● restore help
● encryption
● expanded retention sets
● full + incremental (+ PS bitmaps)

14. So we began with xbackup.sh
● bash shell
● had most of the initial required features
● difficult to maintain and debug
● rewritten to pyxbackup

15. pyxbackup
https://github.com/dotmanila/pyxbackup

16. Simple backups
xtrabackup
pyxbackup full|incr

17. Saving disk space
xtrabackup
compress
pyxbackup --compress --compress-with=gzip full|incr

18. Validation
xtrabackup
apply-log
compress
pyxbackup --apply-log full|incr

19. Optional encryption
xtrabackup
apply-log
compress
encrypt
pyxbackup --compress --compress-with=gzip full|incr

20. Sending copies to remote
xtrabackup
apply-log
compress
encrypt
stream (scp/nc)
pyxbackup --remote-host --remote-push-only full|incr

21. Validation from remote
xtrabackup
compress
encrypt
stream (scp/nc)
apply-log
pyxbackup apply-last

22. Validation can be run independently
apply-last apply-last
pyxbackup apply-last

23. Restoring a backup (full/incremental)
restore-set restore-set
pyxbackup restore-set

24. Binary log streaming to support PITR
binary log stream pyxbackup
pyxbackup binlog-stream

25. Binlog streaming from master
binary log stream pyxbackup
stream (nc/scp)S
M
pyxbackup --binlog-from-master binlog-stream

26. Limitations
● python <= 2.7

27. Limitations
● python <= 2.7
● xtrabackup backup only (can be extended for logical dumps, per
database/table restore)

28. Limitations
● python <= 2.7
● xtrabackup backup only (can be extended for logical dumps, per
database/table restore)
● local or remote Linux servers only (SSH + Python + xtrabackup)

29. Future
● xrabackup 2.3

30. Future
● xrabackup 2.3
● python 3.x

31. Future
● xrabackup 2.3
● python 3.x
● multiple storage locations (round-robin)

32. Future
● xrabackup 2.3
● python 3.x
● multiple storage locations (round-robin)
● point in time recovery (some)

33. Future
● xrabackup 2.3
● python 3.x
● multiple storage locations (round-robin)
● point in time recovery (some)
● restore/test database/table

34. Future
● xrabackup 2.3
● python 3.x
● multiple storage locations (round-robin)
● point in time recovery (some)
● restore/test database/table
● logical dumps

35. Future
● xrabackup 2.3
● python 3.x
● multiple storage locations (round-robin)
● point in time recovery (some)
● restore/test database/table
● logical dumps
● unit tests (ahem)

36. Questions?
Yes, #jervinisreal

38. xtrabackup
apply-log
compress
encrypt
stream (scp/nc)